Tweaking theme to find the best presentation of code/output snippets

The theme I was using previously was causing command output to be trimmed which made the demo output more difficult to read.  I’m experimenting with other themes to see if I can find one that works better – in the mean time the look of the site may change a few times.

Sorry.

Command of the Day: df

In the last couple of posts I’ve looked at how I can get the information I used to get from Windows Task Manager – CPU, memory, process stats and also how I can get disk information. Basically the commands I need to get a snapshot of my general system health.

Today I’m looking at a command to tell me how much disk space I have available on any given partition. The command to do this (or one of the many, I’m sure) is “df”.

Run without any arguments it reports the amount of free disk space on all mount file systems.

$ df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda2             30961696   2751648  26637288  10% /
varrun                  501248       100    501148   1% /var/run
varlock                 501248         0    501248   0% /var/lock
procbususb              501248       136    501112   1% /proc/bus/usb
udev                    501248       136    501112   1% /dev
devshm                  501248         0    501248   0% /dev/shm
lrm                     501248     33788    467460   7% /lib/modules/2.6.20-15-generic/volatile
/dev/sda1            113276220  40018800  73257420  36% /media/sda1
/dev/sda5              9444976       168   9444808   1% /media/sda5
/dev/scd1               628846    628846         0 100% /media/cdrom1

The -m option returns the values in megabytes which is a little more readable

$ df -m
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/sda2                30237      2688     26013  10% /
varrun                     490         1       490   1% /var/run
varlock                    490         0       490   0% /var/lock
procbususb                 490         1       490   1% /proc/bus/usb
udev                       490         1       490   1% /dev
devshm                     490         0       490   0% /dev/shm
lrm                        490        33       457   7% /lib/modules/2.6.20-15-generic/volatile
/dev/sda1               110622     39081     71541  36% /media/sda1
/dev/sda5                 9224         1      9224   1% /media/sda5
/dev/scd1                  615       615         0 100% /media/cdrom1

I can tack a “-T” on to that to get the file system type as well

$ df -Tm
Filesystem    Type   1M-blocks      Used Available Use% Mounted on
/dev/sda2     ext3       30237      2688     26013  10% /
varrun       tmpfs         490         1       490   1% /var/run
varlock      tmpfs         490         0       490   0% /var/lock
procbususb   usbfs         490         1       490   1% /proc/bus/usb
udev         tmpfs         490         1       490   1% /dev
devshm       tmpfs         490         0       490   0% /dev/shm
lrm          tmpfs         490        33       457   7% /lib/modules/2.6.20-15-generic/volatile
/dev/sda1     ntfs      110622     39081     71541  36% /media/sda1
/dev/sda5     vfat        9224         1      9224   1% /media/sda5
/dev/scd1  iso9660         615       615         0 100% /media/cdrom1

-h can be used to make the output even more “human” readable:

$ df -Th
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/sda2     ext3     30G  2.7G   26G  10% /
varrun       tmpfs    490M  100K  490M   1% /var/run
varlock      tmpfs    490M     0  490M   0% /var/lock
procbususb   usbfs    490M  136K  490M   1% /proc/bus/usb
udev         tmpfs    490M  136K  490M   1% /dev
devshm       tmpfs    490M     0  490M   0% /dev/shm
lrm          tmpfs    490M   33M  457M   7% /lib/modules/2.6.20-15-generic/volatile
/dev/sda1     ntfs    109G   39G   70G  36% /media/sda1
/dev/sda5     vfat    9.1G  168K  9.1G   1% /media/sda5
/dev/scd1  iso9660    615M  615M     0 100% /media/cdrom1

“-t” and “-x” can be used to include or exclude file systems of a specific type (respectively). “-i” provides information on inodes. “-l” only returns local file systems (since all of mine are local this does not create an interesting demo).

There are a few other options but this tool provides the answer to the basic question “how much disk space is available?”

Command of the Day: top

Top provides information about the processes currently running sorted by their CPU usage. Like Windows Task Manager it refreshes frequently and gives basic stats on CPU and memory usage. Another similarity is that it can be used to adjust process priority, sort by other criteria (pid, etc), etc.

A sample probably says it best:

Tasks: 103 total,   2 running, 100 sleeping,   0 stopped,   1 zombie
Cpu(s):  8.5%us,  2.0%sy,  0.8%ni, 79.6%id,  8.8%wa,  0.1%hi,  0.2%si,  0.0%st
Mem:   1002500k total,   442988k used,   559512k free,    60144k buffers
Swap:  2096472k total,        0k used,  2096472k free,   236868k cached    

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 5103 root      15   0 62556  21m 6476 S  4.0  2.2   0:15.16 Xorg
    1 root      18   0  2912 1844  524 S  0.0  0.2   0:01.09 init
    2 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/0
    4 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    5 root      10  -5     0    0    0 S  0.0  0.0   0:00.01 events/0
    6 root      16  -5     0    0    0 S  0.0  0.0   0:00.00 khelper
    7 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kthread
   30 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kblockd/0
   31 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 kacpid
   32 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 kacpi_notify
  139 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kseriod
  163 root      16   0     0    0    0 S  0.0  0.0   0:00.00 pdflush
  164 root      15   0     0    0    0 S  0.0  0.0   0:00.00 pdflush
  165 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 kswapd0
  166 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 aio/0
 1991 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 ksuspend_usbd
 1992 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 khubd

It’s worth noting that to get that output I needed to run top in batch mode and redirect to a file – the command line option to do that is ‘-b’ (i.e. top -b > output.txt).

When running top in non-batched mode there are a ton of ways to filter. Some that I found helpful were:

z	Toggle color/mono mode
F	Select filter criteria (it brings up a screen of filter options)
h	Display help
i	Toggle suppression of idle processes
P	Sort by CPU usage (this is the default)
T	Sort by time or cumulative time.
m	Toggle display of memory stats
W	Write out current settings to the ~/.toprc file
k	Prompted for pid to kill
r	Apply renice to a process (renice can adjust the process priority)

This is just the tip of the iceberg. I had no idea how much was available with the top command.

Command of the Day: free

After getting everything installed and working for a few days I wanted to get a feel for some basic metrics:

1. Memory consumption
2. Disk space usage
3. Running processes
4. CPU usage

Basically I wanted to have the info I get from the Windows Task Manager + disk usage.

I’ll take those in order – today is “free”.

Free displays memory usage stats in bytes, kilobytes or megabytes. Also it can display on a timer (every N seconds). Pretty straight forward and easy to follow.

@desktop:~$ free
total       used       free     shared    buffers     cached
Mem:       1002500     475980     526520          0      60156     267344
-/+ buffers/cache:     148480     854020
Swap:      2096472          0    2096472

Next CotD will cover disk space usage.

Command of the Day: fuser

This one is pretty cool – fuser identifies the process id of any processes that are using a specific file or filesystem. This is something I need frequently on Windows when debugging service or multi-user tests. Invariably there will be a race condition that is causing cleanup to not succeed because the cleanup thread has gotten ahead of the app or test thread(s). When that happens the cleanup thread is unable to move or delete log files, temp data, etc.

I’m left wondering who had the file open. There are many ways to find this but I normally use a SysInternals tool to check which process has the open handle. Problem is this isn’t easy to automate so by the time I’m looking for the file lock it’s already gone.

Using fuser I can add this functionality right into my test suites.

To test fuser I performed a more operation on a file in my home directory and left the more process blocked with the file open. That command was:

username@desktop:~$ more menu.lst

In another terminal window I then executed:

username@desktop:~$ fuser -u menu.lst
menu.lst:             5829(username)

So process 5829 being run by the user “username” has the file menu.lst open.

Now I can use ps to figure out what command that user is running (and on what tty).

username@desktop:~$ ps 5829
PID TTY      STAT   TIME COMMAND
5829 pts/1    S+     0:00 more menu.lst

I think I need to try piping all of that together into one operation.

Command of the Day: addr2line

To kick off the “Command of the Day” series I decided to start with addr2line.

Addr2line, part of GNU Binutils, will translate a hex address into the appropriate filename and line number for the code at that address. So an example usage would be if a program crashes at a specific location you can easily find out which line of code in which file the crash occurred at. This is quite useful if you get the crash report via an email or forum post and don’t have access to a core dump or the ability to hook up a debugger (assuming the other person is running a well known pre-built binary and have proper debug symbols).

A concrete example helped for me. I wrote this simple program:

#include <stdio.h>

void test_method() {
  printf("%pn", &test_method);
}

int main(int argc, char **argv) {
  test_method();
  return 0;
}

When compiled (gcc a2ltest.c -ggdb) the produced executable (a.out) prints out the pointer to the function test_method in hex format. An example run is:

..@desktop:~/cotd/addr2line$ ./a.out

0x8048374

Now I can pipe that output to addr2line and see the expected results:

..@desktop:~/cotd/addr2line$ ./a.out  | addr2line

/home/../cotd/addr2line/addr2line.c:3

On Windows I could get similar info from various methods such as using map files (though you have to work through the relative addresses on your own and hope that the assembly loaded where you think it did)

Very useful tool.

Fixing my GRUB mistakes

A few generous folks left comments with some good advice on editing the GRUB configuration so that kernel updates would not over-write my customizations.

I reverted to the original /boot/grub/menu.lst file (glad I made that backup!) and change the default to 4 (Windows XP), enabled hidden menus and turned on pretty colors (just because I could). Rebooted and things worked great. Thanks for the help.

I’ve spent some time considering what my next step should be and I have three things I want to accomplish:

1. Become proficient at developing using a fairly standard toolset (this is mostly about shell and editor familiarity) – vi, emacs and bash.

2. Develop a simple Firefox extension (theory is that this will cover a good bit of territory while still being a very approachable project).

3. Go into “command a day” mode where I learn about a single command each day (I picked up Linux in a Nutshell) and write a test script (or at least come up with a scenario) where I use that command.

Nothing crazy – but something that will produce at least one new thing per day (even if it’s just a command).

I’m pretty sure I know what the Firefox extension will be but I want a night to mull it over so I don’t get into more than I want right now.  The goal is success not to overwhelm myself.

Future posts will be about Linux – but to address some recent comments …

Unexpected results with the Digg post. I don’t want to dwell on the post too much but there were some comments I wanted to address in a single post. To do that I’ll briefly dissect a few lines of my post:

What was able to drain the marrow of life from my body and leave only hollowed-out shell of the man who once occupied this space?

Six years ago I began working for Microsoft.

I’m not drawing a 100% causal relationship here. Meaning – all of the issues don’t stem from my job (certainly not – kids, home, life, etc – all played a significant role) and I’m not implying that had I worked somewhere else the same thing wouldn’t have happened. It is probably a personality fault in me that allowed this – but when searching for the trigger event this was the landmark event.  Perhaps I shot my foot there. Not sure. I hope not.

I hope people also weigh the positive things I said too.  If I didn’t want to be at MS I would quit.

But there is another side to Microsoft. The side that has held my hand, shielded my eyes and who I allowed to slowly lead me down this path.

Note here that I said “I allowed” – as in everything that has changed in my life changed because I allowed it to. I let myself get into projects and schedules and deadlines and politics when I could of, or should of, had the foresight or backbone to avoid them.

The side where you aren’t given time to be creative or perform research that does not address an immediate work item on your schedule.

Note “given time”. I’m not saying that Microsoft does not value creativity or research – I’m saying that I never had the privilege of working on a team where it was actively fostered in the rank-and-file. Yes – this it the nature of business. No – I don’t think I would have gotten more chances at many other employers. Again – I’m stating my experience.

Where you are discouraged from thinking about things other than what is right in front of you at that moment. Where you have 6 weeks to do 16 weeks of work.

These two are really related. In any software development project there are unrealistic goals and deadlines and political pressures. But in a large company with multi-thousand person development teams working on 2-4 year timelines the needs of the individual can get lost. As is so often the case in our industry deadlines are a product of marketing and sales needs – not honest estimates of the feature’s cost. Being asked for an estimate is often only a courtesy because the dates have already been decided upon by others. This is an industry problem.

Where months of hard work can be thrown away because the lawyers won’t let you ship it even though the customers are screaming for it (talk about getting punched in the gut).

This is true at any company but perhaps magnified at one such as Microsoft. This really isn’t about lawyers and more about feeling like “Why couldn’t you have just told us that upfront so we could have worked on something else?” It’s about knowing that you have wasted months of your time and that your customers still aren’t getting what they need.  You don’t get that time back.  Time is the most valuable resource any of us have.

I love my current job. My role is great. I work on a team of smart people who have a passion for what we are doing. As individuals, and as a team, we have our customer’s needs on our minds constantly. Almost every decision comes down to them. I would recommend this team to anyone who is considering a career at Microsoft.

But this isn’t my first role in MS (and I pray it won’t be my last!) – some previous ones were not nearly so enjoyable. Not every job is rainbows and butterflies. Not at any employer. Not every team is right for every person. Not every manager is the right fit for every employee. I was on a team that was not the right fit for me. I used the proper process to find a new role. But to deny that I was left felling burnt out and a bit like damaged goods would be disingenuous.

Several years of working on a team that is not a match for you becomes a grind. It wears you down.

Oh – and general comments:

1. I’ve apparently made a mistake in my usage of GRUB. I’ll look into that.

2. I have several typos that change the meanings of the phrases entirely. I will leave them as-is. But for the record my wife does not have a glandular problem. There are certainly typos in this post too.

3. While I did have comment moderation enabled I did not reject or fail to publish any.

4. I do not speak for my employer. This blog is about my personal thoughts and feelings.

5. I hope that those of you who suggest I could or should be fired are wrong.

6. I have disabled the Snap preview pages. I did not know they were on. It is a WordPress setting that was enabled (I think I did it while signing up).

7. I am still running XP at home. I intend to continue doing that for quite some time. It is important to my family to have a familiar experience.

8. I did not bash XP, Vista, etc – the move to Linux was, as I said, mostly symbolic. My OS does not define me. It will not change my life. I haven’t become suddenly more creative since installing Linux. This was about letting my curiosity explore something new (new to me, anyway), researching something and expressing myself creatively (this blog – whether you like it or not – that’s the intent).

9. This post was about me – not my employer. The fact that I let my career become such an overwhelming force in my life is my fault – not my employers.

I appreciate the comments. I may respond further in the comment section on some posts but future posts will be about what’s going on and not rehashing this further.  My intent is to talk about my experience in moving to Linux – not about the politics of work.

And seriously – if you work at MS and are thinking “this guy should be fired!” – sleep on it.  Microsoft needs more people like me – not fewer (sorry Mini – I just encouraged hiring).  Microsoft needs more people who have woken up and realized that we can and must do better.  That work/life balance is important – not just paying it lip-service – but actually realizing it in a healthy way.  That giving people time to think about things other than their day-to-day tasks will improve every employee in the long run (which improves the company as a whole).  And that Linux, and OSS in general, are something we all need to be looking at because you can bet that the Linux community is looking hard at each and every one of us.

It’s been a long day.  Good night.

Changing GRUB boot order to boot Windows XP before Ubuntu

One goal of installing Ubuntu is to help ease my wife and kids onto it – not to create a huge problem with the family. After the Ubuntu install the default boot ed OS was Ubuntu – not Windows. That’s going to be a problem.

So I went on a mission to figure out how to change the boot order to load XP by default.

I knew that Ubuntu was using the GRUB boot loader. I wasn’t familiar with GRUB so I don’t know why I knew this – it must have been stated during the install sometime. But anyway – I knew it was GRUB.

I Googled for “change GRUB boot order” and the first hit was exactly what I needed. I needed to edit /boot/grub/menu.lst.

I copied the file into my home dir and used chmod (it’s been so long since I’ve used a *nix CLI that I had to lookup chmod – I knew it was “ch” something) so I could write to it then opened it in xemacs.

I simply copied the XP settings ahead of the Ubuntu settings in the item list and left everything else the same.

The relevant parts of the original menu.lst were:

—————————- /boot/grub/menu.lst —————————————-

default 0
timeout 10

## ## End Default Options ##

title Ubuntu, kernel 2.6.20-15-generic
root (hd0,1)
kernel /boot/vmlinuz-2.6.20-15-generic root=UUID=12de9aee-c011-429e-b2a9-0ed83b3eb727 ro quiet splash
initrd /boot/initrd.img-2.6.20-15-generic
quiet
savedefault

title Ubuntu, kernel 2.6.20-15-generic (recovery mode)
root (hd0,1)
kernel /boot/vmlinuz-2.6.20-15-generic root=UUID=12de9aee-c011-429e-b2a9-0ed83b3eb727 ro single
initrd /boot/initrd.img-2.6.20-15-generic

title Ubuntu, memtest86+
root (hd0,1)
kernel /boot/memtest86+.bin
quiet

### END DEBIAN AUTOMAGIC KERNELS LIST

# This is a divider, added to separate the menu items below from the Debian ones.

title Other operating systems:
root

# This entry automatically added by the Debian installer for a non-linux OS
# on /dev/sda1

title Microsoft Windows XP Home Edition
root (hd0,0)
savedefault
makeactive

—————————–

I referred to the online GRUB manual (http://www.gnu.org/software/grub) and confirmed that the “0” in default was the list item to boot by default (zero-indexed) and that if it were changed to “saved” the previously loaded item with a “savedefault” entry would be used. I didn’t want that. I want it to boot to XP by default regardless of what I used most recently.

I had two choices – change the default value to 4 (the menu divider is an option as well) or leave it at 0 and reorder the items.

I decided to reorder the items for one reason – I want the top menu item to be the default because that is how my family will expect it to work.

There are some notes about an automatically generated section that could be over-written so I did back up the file before making the change and I decided it was worth the hassle of losing the customizations I made (and possibly the Window’s item) to make it work the way we need for now. I don’t plan to change it often.

So I simply moved the XP section to the top, moved the divider below it, saved the local copy and copied it over the original.

Rebooting brought up XP after a 10 second delay – just as I had hoped.

Note – I reverted the changes and used a simplier approach that kernel updates won’t overwrite.

Step 3: Install Ubuntu

Finally it was time to install Ubuntu to the newly created partitions. I booted to the Ubuntu CD and …

wait … what’s going on? Why am I in 640×480 mode? It’s always been fine before.

Did I accidentally choose safe graphics mode? Sure – that’s what I must have done.

So I rebooted and made doubly sure I selected the proper startup option. This is looking much better … wait … the screen just flashed. I’m in 640×480 again. Perhaps it’s just a configuration problem. System -> Preferences -> Screen Resolution … yup … 640×480 – so I’ll just select the drop-down l…

There are no other options.

Why do I care so much? Because the Ubuntu installer does not run properly in 640×480 because the control buttons (forward, back, etc) are not visible – the windows are too large and they do not have the option to scroll.

I went through this dance twice before actually choosing to start in the graphics safe mode.

Well in safe graphics mode it worked as expected! 1280×1024 resolution. I have no idea what happened. I’m going to call it an anomaly and not worry about it. I don’t know enough about Ubuntu to assume it was anything other than user error.

Since I was in a decent resolution I ran the installer by using the “Install” link on the desktop.

Installing was a breeze. I had to pick my timezone, keyboard info, create an account for myself, and manually configure the partitions I created earlier. Then about 20 minutes later the installation was complete.

I rebooted and was greeted by a new boot loader that let me choose between a few Ubuntu options and Windows XP. Exactly what I hoped for.

I booted to Ubuntu, logged in using the account I created, opened a terminal window and created a symbolic link from a directory under /media/sda5 (the shared drive – it took me a little while to find the proper location) into my home directory. Now I can easily share data between the two systems.

Oh – and my NTFS drive is mounted too so I guess that was unnecessary but I’m glad to have done it since it was a good learning experience.

Now that it’s working – its time to move on to the next pet project.