Space War

Let say it is a late Christmas present for Science Fiction fans. Especially ones who like a lot of science in SF. Two articles discussion how space warfare can look. Have fun.

• www.foreignpolicy.com/articles/2012/09/28/aircraft_carriers_in_space
• http://gizmodo.com/5426453/the-physics-of-space-battles

Even more threads counting

This is small extension to one of my previous posts. This time a loop is enriched by load values (from /proc/loadavg) as well as measurement time (date).  ps command uses the same option, but there is small improvement in awk call.  Rather than count only processes per state it concatenate state with last string in command arguments — I was mostly interested in few java application and jar name was the last parameter for each one. Also awk counts only processes actually running or in uninterruptible sleep ($2 ~ /(D|R)/ at the beginning of awk command).

while [ 1 ];
do
    date;
    cat /proc/loadavg;
    ps -Leo pid,state,args |
     awk ' $2 ~ /(D|R)/ {state[$2 " - (" $1 ") " $NF]++} \
      END{ for (j in state) {printf "%s - %d\n", j, state[j]}}' |
      sort -k 2;
    echo "---";
    sleep 5;
done

Debian, Vim and alternatives

I don't know why, but default editor in Debian is not Vim, but Nano. I learned once that it can be change with a command, but has never remembered the actual name of it and since than had to check it each time in install Debian (not so often). Today decided to write it down, not to search too long next time. Maybe it's going to be useful to others:

aptitude install vim.nox
update-alternatives --set editor vim.nox

e17 and stable Debian

If you are interested in using e17 with Debian Wheezy, the best way is to use package provided by John Holland at http://www.vin-dit.org/.

Number of threads per state

awk with ps creates powerful Linux combo. For example there is one, or more, multi-threaded application on your system. You would like to know what states all those threads are. Standard ps show information about processes, but you can learn more about threads with "-L" switch. You probably hit another issue - number of threads, you don't want to analyse, let say, 300 lines. Lucky, awk is very good for such tasks. Following 'one-liner' shows number of threads and processes per state (running, sleeping, uninterrupted sleep), every 5 seconds.

while [ 1 ];
do
    ps -Leo pid,state,args| \
     awk '{state[$2]++} \
      END{ for (j in state) {printf "%s - %d\n", j, state[j]}}';
    echo "---";
    sleep 5;
done

First line ensures that command runs forever. ps uses '-e' (all processes), '-L' (include threads), '-o' (user specify output). There are 3 lines in ps output: PID, state and command arguments. (These arguments are useful for my other commands. In this case you can leave only 'state', but you has to modify awk command). Next awk takes every line and counts each state using state array with argument equal to state representation (i.e.: S for sleep, D for uninterrupted sleep, R for running). At the end of execution it prints each argument with it value. After that script displays '---'  and waits 5 seconds.
Such output can be valuable extension to top command results. It shows instant changes to number of busy processes, which affects system load.

CPU affinity, interrupts and old kernel bug

This is an old story and AFAIK were address ages ago in Linux kernel, but there are still plenty of CentOS/RHEL 5 running and you still might be hit by the problem with kernel not properly balancing interrupts from hardware, so the that below list of the links my be helpful. In bug report network card are mentioned, but it might affect other components (i.e. raid controllers from HP - cciss drivers).

If you would like to learn more there is a set of good links:

1. https://bugzilla.redhat.com/show_bug.cgi?id=432451
2. http://docs.oracle.com/cd/E24290_01/coh.371/e22838/tune_perftune.htm
3. http://www.alexonlinux.com/smp-affinity-and-proper-interrupt-handling-in-linux
4. http://www.alexonlinux.com/msi-x-the-right-way-to-spread-interrupt-load
5. http://www.alexonlinux.com/why-interrupt-affinity-with-multiple-cores-is-not-such-a-good-thing

Regexp Groups in Python

Let say that we have a string consist from a numeric continent code  (i.e. 1 for Europe) , followed by a country code  (i.e. 44 for UK)  and ended with a region code made in similar manner (i.e. Cambridge equal 65). A continent and a country are preceded with the letter C and a location with the letter L.

So we have the string C1C44L65 and we need to know the continent, counter and region codes, but numbers might have different length (i.e. 5, but also 55 or 555) so we cannot just grab selected characters from a string, but in Python we can use following regular expression to save all that information into "groups" for further use, in example in an mathematical operations (as integers).

info = re.search(r"^C(?P[^\d]+)C(?P\d+)L(?P\d+)$", full_id)
continent = int(info.group("continent"))
country = int(info.group("country"))
region = int(info.group("region"))

Ubuntu and mix of NFS, NIS and autofs in post from past

Some time ago I had a problem with autofs, NIS and NFS during Ubuntu start. As far as I remember autofs was responsible for serving users home directories, define with NIS and  provided from remote location with  NFS. The problem that autofs started before network and NIS, so home directories weren't ready.

When I started look into the problem found that small change to autofs.conf should resolve the situation. I didn't deploy the fix on big scale (I were using CRUX on my desktop), but saved the patch to blogger and today finally found some time to share it. Maybe it will be useful for anyone.

root@test-upstart:/etc/init# diff -u autofs.conf ~wawrzekn/autofs.conf
--- autofs.conf2010-08-17 11:34:47.000000000 +0100 
+++ /home/wawrzekn/autofs.conf2011-02-06 12:19:48.678663000 +0000 
@@ -10,6 +10,15 @@ 
 respawn 
  
 pre-start script 
+while : 
+do 
+ypwhich 
+if [ $? = 0 ]  
+then  
+break 
+fi 
+sleep 4 
+done 
 if [ -f /etc/default/autofs ]; then 
 . /etc/default/autofs 
 fi

PS2 and friends

This post won't be about PlayStation 2, but about my (not so) recent problem with the UNIX shell prompt.

I had a problem with prompt on some remote Linux system. Everything looked fine until the command (with arguments) fitted one line. I was using iTerm2 on MacOSX with full screen mode, so line could be long, but anyway it started to be very annoying. When I tried to move cursor back or jump to the beginning of the line it could end up in the middle of existing text.

I spent a lot of time trying to debug problem. First I tried to find something wrong with my iTerm2 settings, but could not find a thing. Next I started to play with tput (see link 1 and 2). For a moment believed that I had broken my PS2 setting (see 3) and finally, after reading link 4, figured out that I hadn't properly escaped non printing character in colour settings. My 'remote' bash prompt looks currently as printer below and you can see a lot of \[ before not printing characters and \] after them.

\[\e[32m\]\h \[\e[0m\]: \[\e[33m\]\w \[\e[0m\]#>

Reading:

1. http://tldp.yolinux.com/HOWTO/Bash-Prompt-HOWTO.html
2. http://tldp.org/LDP/abs/html/terminalccmds.html
3. http://www.thegeekstuff.com/2008/09/bash-shell-take-control-of-ps1-ps2-ps3-ps4-and-prompt_command/
4. http://www.ibm.com/developerworks/linux/library/l-tip-prompt/