gprof displays profile data for an object file. To use gprof I first needed to add “-g -pg” to my g++ options – i.e.

$ g++ -g -pg main.cpp

Then I simply needed to run the output executable (./a.out) which produced a file named gmon.out in the current directory.

Executing gprof is now enough to get some basic profiling information – e.g.:


% cumulative self self total
time seconds seconds calls s/call s/call name
100.52 9.73 9.73 5 1.95 1.95 run()
0.00 9.73 0.00 1 0.00 0.00 init()

The app ran for almost 10 seconds, made 5 calls to run which ran, on average, about 2 seconds per call.

Pretty simple high-level profiling.

The sample app was nothing more than a busy-wait (I wanted to busy wait otherwise all of the samples ended up in the call to sleep).

It’s code is:


#include
#include
#include

void init() {
srand(time(0));
}

void run() {
int limit = rand();

for(int i = 0; i

Some interesting command line options include:

-i Prints out some basic summary info (record counts)
-p Print basic call graph info
-z Display unused functions
-p Print basic call graph info
-A Print annotated source code

I intend to look more into gcov in the next few days to see what it has to offer.

While I'm on the topic of programming tools - what's available in the way of static analysis (i.e. lint) tools?

“more” displays the named file(s) one screenful at a time. Simple enough and quite useful – but it’s got a few handy little features I was unaware of prior to getting to more in Linux in a Nutshell.

The straight-forward usage is something like:

$ more file.txt

The result of this will be the first screenful of file.txt with a prompt indicating what percentage of the file was being displayed.Up to today that was the extent of how I used more.

Usually when I’m using more I’m looking for something but I’m not necessarily sure what so grep isn’t what I want. But I’ve often got a pretty decent idea of what I want – perhaps a keyword.

more supports jumping right to the first instance of the pattern you care about. If more is already running you can enter:

/pattern

And more will skip forward to the next instance of the pattern string. This can also be done upon invoking more using the command line syntax:

$ more +/pattern file.txt

If the pattern is found more will present the first match on the displayed screenful. If the pattern is not found more will indicate as much.

Now that you are at the first pattern match perhaps it was not the one you wanted. You can skip to the next match by hitting ‘n’ at the prompt. This is a great way to skip forward in the file quickly. If can skip more than one by typing a number before hitting ‘n’. For example at the more prompt if you type “12n” it will take you to the twelfth instance of the repeated pattern match.

Finally you can work with multiple files in more. The commands for moving between the files are “:n” to move the next file and “:p” to move to the previous. To help keep track of where you are you can use “:f” to display the file name and currently displayed line number.

There’s more going on than I previously realized.

I’m sorry. I couldn’t help it.

In the last couple of posts I’ve looked at how I can get the information I used to get from Windows Task Manager – CPU, memory, process stats and also how I can get disk information. Basically the commands I need to get a snapshot of my general system health.

Today I’m looking at a command to tell me how much disk space I have available on any given partition. The command to do this (or one of the many, I’m sure) is “df”.

Run without any arguments it reports the amount of free disk space on all mount file systems.

$ df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda2             30961696   2751648  26637288  10% /
varrun                  501248       100    501148   1% /var/run
varlock                 501248         0    501248   0% /var/lock
procbususb              501248       136    501112   1% /proc/bus/usb
udev                    501248       136    501112   1% /dev
devshm                  501248         0    501248   0% /dev/shm
lrm                     501248     33788    467460   7% /lib/modules/2.6.20-15-generic/volatile
/dev/sda1            113276220  40018800  73257420  36% /media/sda1
/dev/sda5              9444976       168   9444808   1% /media/sda5
/dev/scd1               628846    628846         0 100% /media/cdrom1

The -m option returns the values in megabytes which is a little more readable

$ df -m
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/sda2                30237      2688     26013  10% /
varrun                     490         1       490   1% /var/run
varlock                    490         0       490   0% /var/lock
procbususb                 490         1       490   1% /proc/bus/usb
udev                       490         1       490   1% /dev
devshm                     490         0       490   0% /dev/shm
lrm                        490        33       457   7% /lib/modules/2.6.20-15-generic/volatile
/dev/sda1               110622     39081     71541  36% /media/sda1
/dev/sda5                 9224         1      9224   1% /media/sda5
/dev/scd1                  615       615         0 100% /media/cdrom1

I can tack a “-T” on to that to get the file system type as well

$ df -Tm
Filesystem    Type   1M-blocks      Used Available Use% Mounted on
/dev/sda2     ext3       30237      2688     26013  10% /
varrun       tmpfs         490         1       490   1% /var/run
varlock      tmpfs         490         0       490   0% /var/lock
procbususb   usbfs         490         1       490   1% /proc/bus/usb
udev         tmpfs         490         1       490   1% /dev
devshm       tmpfs         490         0       490   0% /dev/shm
lrm          tmpfs         490        33       457   7% /lib/modules/2.6.20-15-generic/volatile
/dev/sda1     ntfs      110622     39081     71541  36% /media/sda1
/dev/sda5     vfat        9224         1      9224   1% /media/sda5
/dev/scd1  iso9660         615       615         0 100% /media/cdrom1

-h can be used to make the output even more “human” readable:

$ df -Th
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/sda2     ext3     30G  2.7G   26G  10% /
varrun       tmpfs    490M  100K  490M   1% /var/run
varlock      tmpfs    490M     0  490M   0% /var/lock
procbususb   usbfs    490M  136K  490M   1% /proc/bus/usb
udev         tmpfs    490M  136K  490M   1% /dev
devshm       tmpfs    490M     0  490M   0% /dev/shm
lrm          tmpfs    490M   33M  457M   7% /lib/modules/2.6.20-15-generic/volatile
/dev/sda1     ntfs    109G   39G   70G  36% /media/sda1
/dev/sda5     vfat    9.1G  168K  9.1G   1% /media/sda5
/dev/scd1  iso9660    615M  615M     0 100% /media/cdrom1

“-t” and “-x” can be used to include or exclude file systems of a specific type (respectively). “-i” provides information on inodes. “-l” only returns local file systems (since all of mine are local this does not create an interesting demo).

There are a few other options but this tool provides the answer to the basic question “how much disk space is available?”

Top provides information about the processes currently running sorted by their CPU usage. Like Windows Task Manager it refreshes frequently and gives basic stats on CPU and memory usage. Another similarity is that it can be used to adjust process priority, sort by other criteria (pid, etc), etc.

A sample probably says it best:

Tasks: 103 total,   2 running, 100 sleeping,   0 stopped,   1 zombie 
Cpu(s):  8.5%us,  2.0%sy,  0.8%ni, 79.6%id,  8.8%wa,  0.1%hi,  0.2%si,  0.0%st 
Mem:   1002500k total,   442988k used,   559512k free,    60144k buffers 
Swap:  2096472k total,        0k used,  2096472k free,   236868k cached    

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND 
 5103 root      15   0 62556  21m 6476 S  4.0  2.2   0:15.16 Xorg 
    1 root      18   0  2912 1844  524 S  0.0  0.2   0:01.09 init 
    2 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0 
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/0 
    4 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 watchdog/0 
    5 root      10  -5     0    0    0 S  0.0  0.0   0:00.01 events/0 
    6 root      16  -5     0    0    0 S  0.0  0.0   0:00.00 khelper 
    7 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kthread 
   30 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kblockd/0 
   31 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 kacpid 
   32 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 kacpi_notify 
  139 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kseriod 
  163 root      16   0     0    0    0 S  0.0  0.0   0:00.00 pdflush 
  164 root      15   0     0    0    0 S  0.0  0.0   0:00.00 pdflush 
  165 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 kswapd0 
  166 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 aio/0 
 1991 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 ksuspend_usbd 
 1992 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 khubd

It’s worth noting that to get that output I needed to run top in batch mode and redirect to a file – the command line option to do that is ‘-b’ (i.e. top -b > output.txt).

When running top in non-batched mode there are a ton of ways to filter. Some that I found helpful were:

z Toggle color/mono mode
F Select filter criteria (it brings up a screen of filter options)
h Display help
i Toggle suppression of idle processes
P Sort by CPU usage (this is the default)
T Sort by time or cumulative time.
m Toggle display of memory stats
W Write out current settings to the ~/.toprc file
k Prompted for pid to kill
r Apply renice to a process (renice can adjust the process priority)

This is just the tip of the iceberg. I had no idea how much was available with the top command.

After getting everything installed and working for a few days I wanted to get a feel for some basic metrics:

  1. Memory consumption
  2. Disk space usage
  3. Running processes
  4. CPU usage

Basically I wanted to have the info I get from the Windows Task Manager + disk usage.

I’ll take those in order – today is “free”.

Free displays memory usage stats in bytes, kilobytes or megabytes. Also it can display on a timer (every N seconds). Pretty straight forward and easy to follow.

@desktop:~$ free
total       used       free     shared    buffers     cached
Mem:       1002500     475980     526520          0      60156     267344
-/+ buffers/cache:     148480     854020
Swap:      2096472          0    2096472

Next CotD will cover disk space usage.

This one is pretty cool – fuser identifies the process id of any processes that are using a specific file or filesystem. This is something I need frequently on Windows when debugging service or multi-user tests. Invariably there will be a race condition that is causing cleanup to not succeed because the cleanup thread has gotten ahead of the app or test thread(s). When that happens the cleanup thread is unable to move or delete log files, temp data, etc.

I’m left wondering who had the file open. There are many ways to find this but I normally use a SysInternals tool to check which process has the open handle. Problem is this isn’t easy to automate so by the time I’m looking for the file lock it’s already gone.

Using fuser I can add this functionality right into my test suites.

To test fuser I performed a more operation on a file in my home directory and left the more process blocked with the file open. That command was:

username@desktop:~$ more menu.lst

In another terminal window I then executed:

username@desktop:~$ fuser -u menu.lst
menu.lst:             5829(username)

So process 5829 being run by the user “username” has the file menu.lst open.

Now I can use ps to figure out what command that user is running (and on what tty).

username@desktop:~$ ps 5829
PID TTY      STAT   TIME COMMAND
5829 pts/1    S+     0:00 more menu.lst

I think I need to try piping all of that together into one operation.

To kick off the “Command of the Day” series I decided to start with addr2line.

Addr2line, part of GNU Binutils, will translate a hex address into the appropriate filename and line number for the code at that address. So an example usage would be if a program crashes at a specific location you can easily find out which line of code in which file the crash occurred at. This is quite useful if you get the crash report via an email or forum post and don’t have access to a core dump or the ability to hook up a debugger (assuming the other person is running a well known pre-built binary and have proper debug symbols).

A concrete example helped for me. I wrote this simple program:


#include <stdio.h>
void test_method() {
  printf("%pn", &test_method);
}

int main(int argc, char **argv) {
  test_method();
  return 0;
}

When compiled (gcc a2ltest.c -ggdb) the produced executable (a.out) prints out the pointer to the function test_method in hex format. An example run is:

..@desktop:~/cotd/addr2line$ ./a.out
0x8048374

Now I can pipe that output to addr2line and see the expected results:

..@desktop:~/cotd/addr2line$ ./a.out  | addr2line
/home/../cotd/addr2line/addr2line.c:3

On Windows I could get similar info from various methods such as using map files (though you have to work through the relative addresses on your own and hope that the assembly loaded where you think it did)

Very useful tool.

Follow

Get every new post delivered to your Inbox.