Remote diff

When you are working as a Linux SysAdmin quite often you have to compare files from two different machines. I found (here) the script which made my life easier, but after some time decided to customize and extent it a bit. Usually I compare file in the same location therefore the first argument of my script is file path. I also gave chance user to pass an argument for the diff command (4th argument, the default is '-b').

#!/bin/bash
#
# this acts as a remote diff program, accepting two files and displaying
# a diff for them.  Zero, one, or both files can be remote.  File paths
# must be in a format `scp` understands: [[user@]host:]file

[ -n "$1" ] || [ -n "$2" ] || [ -n "$3" ] || \
{ echo "Usage: `basename $0` file1 server1 server2" && exit 1;}

if test -e $4 
then
       opt="-b"
else
       opt=$4
fi

scp "$2:$1" rdiff.1 >& /dev/null
scp "$3:$1" rdiff.2 >& /dev/null
diff $opt rdiff.1 rdiff.2
rm -f rdiff.1 rdiff.2

How to create pictures thums in the one line

The following command can be paste as a one line

for i in `find /path/to/directory/with/pictures  -iname "*.JPG"`;\
do\
convert $i -resize 800x600 `dirname $i`/thumb-`basename $i`;\
done

The find command return a whole path to a file. But we want to add thumb- before the actual name of a file. Therefore `dirname $i` ensure that convert get the proper path and `basename $i` the actual file name (preceded by thumb-). It is also worth to note the iname option of find command. It is case insensitive version of the name.

Another Awk one-liner

This one-liner printing third word from the lines beginning with "Mem:" (precisely the lines which first word is "Mem:") but adding the serial number before the word.

awk 'BEGIN {a=1} \
{if ( $1 == "Mem:" ) \
  {printf "%4d  %s\n",  a, $3; a++}}' \
free-prefork.log >mem-prefork.log

BTW. This script is to help me to plot a gnuplot graph based on "used memory" number from free command. plot "mem-prefork.log" will do the rest of job. UPDATE Mike Hommey post force me to rethink my scripts and I found easy way to eliminate if clause:

awk 'BEGIN {a=1} \
/^Mem:/  {printf "%4d  %s\n",  a, $3; a++}' \
free-prefork.log >mem-prefork.log

Gnuplot with readline on MacOSX

Recently, I tried to use gnuplot on a Mac, and, of course, it wasn't working properly. Apple prepared and shipped with MacOS its own (broken) version of readline library with didn't work with gnuplot (known bug). So I grabbed the sources of readline, applied all of patches and built mine version of readline. Next, I added the proper option to gnuplot configure file, but it wasn't pass to makefile. I looked into the Makefiles and found the the TERMLIBS option had to be change. I.e. using such command:

find . -name Makefile \
-exec sed -i.old "s/TERMLIBS\ =/TERMLIBS = -L\/usr\/local\/lib/" {} \; 

Two more things about gnuplot and MacOSX.

1. I started to think to make a gnuplot.app for MacOSX, but sure how it should work.
2. I found that X11 term is much better then Aqua, in particular, it's allow to rotate 3D (s)plots.

Header for bonnie++ csv file

Recently I started to use bonnie++ to perform some disk tests. One of the thing annoy me is that bonnie++ output has no the 'csv header' describing what is in which column. It makes overview of results in a spreadsheet rather hard. Therefore I created my own one.

,,Sequential,Output,,,,,Sequential,Input,,,Random,,Sequential,Create,,,,,Random,Create,
,,Per Chr, ,Block, ,Rewrite, ,Per Chr, ,Block, ,Seeks, ,Create, ,Read, ,Delete, ,Create, ,Read, ,Delete
Machine,Size,K/sec,%CP,K/sec,%CP,K/sec,%CP,K/sec,%CP,K/sec,%CP,/sec,%CP,files,/sec,%CP,/sec,%CP,/sec,%CP,/sec,%CP,/sec,%CP,/sec,%CP

You can save above lines in the file called i.e. bonnie-header.csv and then cat it before csv part of bonnie.out file (of course it can have different name), by:

cat bonnie-header.csv `tail -1 bonnie.out` >bonnie.csv

After that the output should looks similar to this one: BTW. I found that default number of files creating for metadata benchmarks is low, so I increased it to 128.

BigPicture

I don't know how I might not to add this earlier but I didn't. The Big Picture it's a on-line addition to Boston Globe (I guess it only on-line, but I don't reader Boston Globe, as I'm leaving in proper Cambridge, at least at the moment). I saw it first time on Bad Astronomy Blog. That story was about picture of our sky but made from above. The pictures were amazing, but what more important that Big Picture brings new great pictures month after month. It's quite interesting how many of them are space/astronomy related, i.e. this one:

Polish UK X11 keybord layout

The UK and US keyboard layout are different. Polish one, as probably most other ones based on American. It's fine in Poland, but not so fine in UK. Therefore, I decided to create Polish layout, but based on UK keyboard. I based on Mikoła Kosmulski article describing how to prepare mixture of Polish and German layouts. My kebord setting looks this way:

partial default alphanumeric_keys
xkb_symbols "basic" {

    include "latin"

    name[Group1]="Poland based on GB";

    key   { [         q,          Q ] };
    key   { [         w,          W ] };
    key   { [         e,          E,      eogonek,      Eogonek ] };
    key   { [         o,          O,       oacute,       Oacute ] };

    key   { [         a,          A,      aogonek,      Aogonek ] };
    key   { [         s,          S,       sacute,       Sacute ] };
    key   { [         f,          F ] };

    key   { [         z,          Z,    zabovedot,    Zabovedot ] };
    key   { [         x,          X,       zacute,       Zacute ] };
    key   { [         c,          C,       cacute,       Cacute ] };
    key   { [         n,          N,       nacute,       Nacute ] };

    key   { [         2,   quotedbl,  twosuperior,    oneeighth ] };
    key   { [         3,   sterling, threesuperior,    sterling ] };
    key   { [         4,     dollar,     EuroSign,   onequarter ] };

    key   { [apostrophe,         at, dead_circumflex, dead_caron] };
    key   { [     grave,    notsign,          bar,          bar ] };

    key   { [numbersign, asciitilde,   dead_grave,   dead_breve ] };
    key   { [ backslash,        bar,          bar,    brokenbar ] };

    include "kpdl(comma)"

    include "level3(ralt_switch)"
};

Trash in Ubuntu

My father had a strange problem. He couldn't delete few files from his Trash. First we figured out that he didn't have 'write' rights to them, even if he saved them from a mail. My first suggestion was to change permissions, but he couldn't (using Nautilus). Today I had a chance to sit in front of dad's computer, but I couldn't find where is the Trash. I decided to check if there were any similar bug report in the launchpad. I found this suggestion, which work for me (of course I need to make chmod -R +w ~/.local/share/Trash/files/*). And the point I would like to save here is that the trash in Ubuntu is in:

~/.local/share/Trash/files/

Cordless USB phone - not working with Ubuntu

This time no-success story. I bought the WP-01 VoIP Dect Cordless Handset on ebay (from shop4USB). In Product Description there was an info that it works with "Linux2.4 above". In reality it was working only partly, very partly. Indeed the only part working was sound (as normal sound card), it means I could hear and speak after /choose a number/pick up a call/ using Skype software. Anything ( ring, Keyboard and LCD) wasn't working. Phone was recognize by system and asign to USB HID module:
dmesg |grep hid [ 33.859714] usbcore: registered new interface driver hiddev [ 33.866820] hiddev96hidraw0: USB HID v1.00 Device [HID 06e6:c31c] on usb-0000:00:1f.4-2.2
I tried to make yealink module controlling phone.
rmmod ubhid modprobe yealink
But, how I expected, it didn't help. Finally, the phone USB info was/is
06e6:c31c Tiger Jet Network, Inc.

PS. The seller was OK and gave me my money back.

Swap - VMware effects and parallelization

VMare and swap

To present our software marketing and scientific represents use Windows laptopts with Linux in VMWare. They need Linux because our web based product (Relibase+, IsoStar and incoming WebCSD) working only on it. Using virtualization shouldn't be a problem as a machine has 2GB of memory and we can assign 1GB to guest OS. However, recently we couldn't start WebCSD, not only guest was affected but also host froze. VMWare has problem with I/O operation so we were suspicious about disk usage, but the server didn't need to much of it. Anyway I went I/O trace and decided to turn off the swap. After that server started to work as a rocket!

Parallelization of a swap partitions

I was browins through IBM developersWorks and found info that you can parallelize a swap partition.

Amazingly, all modern Linux kernels, by default (with no special kernel options or patches) allow you to parallelize swap, just like a RAID 0 stripe. By using the pri option in /etc/fstab to set multiple swap partitions to the same priority, we tell Linux to use them in parallel:

/dev/sda2 none swap sw,pri=3 0 0
/dev/sdb2 none swap sw,pri=3 0 0
/dev/sdc2 none swap sw,pri=3 0 0
/dev/sdd2 none swap sw,pri=1 0 0

Creating service starting script

One of our customer ask what to do to start IsoStar server during system start. Our customer was using Centos (RedHat derivative) so I suggested to use the command line tool called chkconfig [1, 2]. TO use this tool you need a special script starting Isostar Apache server. The sample of such script can be found here [3]. Here is the example instruction. Let me assume that isostar is going to /opt directory.

1. Copy the script from link [3] to your HDD and call it isostar_server
2. Change line:

   /path/to/command/to/start/new-service

   to:

   /opt/csd/isostar/APACHE/bin/ccdc_apache start

   and line:

   /path/to/command/to/stop/new-service

   to:

   /opt/csd/isostar/APACHE/bin/ccdc_apache stop

3. Remove following lines (from both start and stop subsection):

         #Or to run it as some other user:
   /bin/su - username -c /path/to/command/to/start/new-service
   echo "."

4. Change 'new-service' in 'echo -n' lines to 'isostar_server'.
5. Now as a root copy isostar_server file into /etc/init.d/
6. Again as a root invoke chkconfig and add isostar_server: /sbin/chkconfig --add isostar_server

There are very similar tool for Debian (Ubuntu etc.) called update-rc.d [4, 5]. You should be able to use following command with the same (very similar script): update-rc.d isostar_server start 90 2 3 4 5 . stop 10 0 1 6.

Links

[1] http://linux.die.net/man/8/chkconfig
[2] http://spiralbound.net/2006/11/15/controlling-services-with-chkconfig
[3] http://spiralbound.net/2007/07/23/example-linux-init-script
[4] http://wiki.linuxquestions.org/wiki/Update-rc.d
[5] http://www.annodex.net/cgi-bin/man/man2html?update-rc.d+8

Count a sum of sizes of selected files in a directory

I would like to count how much space taking Qt4 libraries for the MacOSX universal build of one from our application. So I used following command:

du -m * | sort -nr | grep Qt | awk '{sum+=$1} END {print sum}'

• du -m - print used size in megabytes (for directory do not forgot about -s option);
• sort -nr - sort in reverse, numerical order;
• grep Qt - left only lines with Qt (you can also first grep and next sort lines);
• {sum+=$1} - adding a value from first column of each line to variable sum;
• END {print sum} - printing variable sum on after going through all of lines.

The electronic structure of selected betaine dyes. A quantum chemical study

This thesis presents electronic absorption spectra, non linear optical properties and geometrical parameters of betaine dyes obtained by quantum chemical calculations.

Four betaines [4-(1-piridinium-phenolan), 3-(1-piridinium-phenolan), 2-(1-piridinium-phenolan) and 4-(1-piridinium-thiophenolan)] were selected for the study.

During the research various ab initio methods were applied. The Hartee-Fock method (HF) and the second order Møller-Plesset perturbation theory (MP2) were used to determine a geometrical and NLO properties. Moreover, the NLO were obtained using chosen variants of the coupled cluster metod (CC2, CCSD) and the geometry optimizations were perform using the Density Functional Theory (DFT/B3LYP) as well as complete active space methods (CASSCF and CASPT2). In the case of spectroscopic properties the CC and CASSCF/CASPT2 methods along with Time-Dependant DFT (with B3LYP, PBE0 and CAM-B3LYP functionals) and the Configuration Interaction with Singles (CIS and CIS(D)) were used.

Results obtained during the study indicate that the correct description of betaine dyes' electronic structure is an unusually demanding test for present quantum chemical methods. It is safe to say, that, for all of the investigated parameters, the electron correlation has to be take into account. It is also worth to notify that basis set selection is less important. However, diffuse and polarisation functions should be included in the case of spectroscopic and optical properties.

The presented computational result confirmed the very strong interaction between a betaine molecule and its environment. One of the outcome of this phenomena is a large difference between experimental results (usually obtained in condense phases) and theoretical data (calculated in vacuum). Another observation verified during project is the significant increase of the betaines' NLO by the conformational shifting. Finally, it is worth notify that the largest NLO response was obtained for 4-(1-piridinium-thiophenolan).

How you can quest I'm a doctor now! I defended my thesis (abstract above) 19th of June 2008 and my degree was confirmed by faculty of Chemistry board 26th of June. Thesis was written in Polish, so it isn't very useful for most of the world. However, there is the appendix with all available theoretical data of geometrical and spectroscopic parameters of betaine dyes. Additionally, some of results was published in following articles: JMM-11, JMM-13 and LETT-411 (I hope to write one, maybe more). PDF with thesis can be download here.

If I find some time I will might prepare English version of mentioned appendix, and of course, I will share my LaTeX, gnuplot, computational experience.

GRUB and why root!=root

It one of the this 'keep in mind notes'. I tried to install the Fedora 9 along the Ubuntu 8.04. The Fedora went onto the sdb3 partition or (hd1,2) in the GRUB notation. I added following lines to the Ubuntu /boot/grub/menu.lst file. (I didn't install GRUB from Fedora).

title        Fedora 9
root        (hd1,2)
kernel        /boot/vmlinuz-2.6.25-14.fc9.i686 root=/dev/sdb3 ro
initrd        /boot/initrd-2.6.25-14.fc9.i686.img

But it wanted to work. All the time, I got Error 2 : Bad file or directory type. After trying many things I figured out that problem is lack of the GRUB files at the Fedora partition, so I updated grub config file.

title        Fedora 9
root        (hd0,0)
kernel        (hd1,2)/boot/vmlinuz-2.6.25-14.fc9.i686 root=/dev/sdb3 ro
initrd        (hd1,2)/boot/initrd-2.6.25-14.fc9.i686.img

Finally, the Fedora started to boot.

Yet Another Gnuplot Script

I'm preparing presentation for the Public Defence of my PhD thesis. I'm creating few plots and of course I'm using gnuplot for them. One of plots is seems to be really simple: three 'columns' each in other colour, but it took me few hours to make it. I'm glad that I finally did it so I'm sharing my scripts with you (I needed Postscript for Greek's symbols):

set term postscript eps enhanced color
set output "nlo-rhb-cc.eps"
set ylabel '{/Symbol b} [10^{-30} esu]'
set style data histogram
set style histogram cluster gap 1
set style fill solid border -1
set boxwidth
unset xtics

plot [-0.5:.7][-15000:0] "cc.csv" using 1 ti col, '' using 2 ti col, '' using 3 ti col

If you would like to try use the data below: and data:

HF/FF MP2/FF CCSD
-14720.51945800 -6960.7083277 -10843.140 

BTW. I noted three new interesting website related to the gnuplot:

• http://www.cs.ucr.edu/~titus/pyxTutorial/
• http://www.ibm.com/developerworks/library/l-gnuplot/
• http://www.duke.edu/~hpgavin/gnuplot.html

Ian Foster blog

When I was preparing to the exam from GRID computing one of me friend sent me a link to the Ian Foster blog, and I think that it is great that you can follow thoughts one of the GRID fathers. At the moment he is at the "Open Source Grid & Cluster Conference". It is very interesting that you can grab pdf/videos/ppt from all point of programme.

And Now For Something Not-Completely Different

Sed one liners

I'm using this set of sed one liners very hardly. Today I found that I hasn't added it to my blog so I cannot find it easily. (You know I use this blog as a some kind of memory extension ;) I was trying to find sed command capitalized a variable, I couldn't find it at mentioned page. Farther searching pointed to very interesting thread at Unix .com forum. The was script doing something similar in python, perl, sed and awk. I chosen perl:

echo $myvar | perl -p -e '$_ = ucfirst'

However, python looks also nice:

echo $myvar | python -c "print raw_input().capitalize()"

Awk one is also nice, but a bit more complicated (one line):

echo $myvar |awk '{(sub("^.",substr(toupper($1),1,1),$1)); print }'

Very useful MacOSX key shortcut

The apple+tab key combination change between apps on MacOSX, but it works for one windows per application. What if you have more than one window for app? It is quite common when you are using X11 applications or try to use more than one terminal. To change between active window of one app you can use apple+', but it seems works only for windows on one desktop. It also not work for X11 app, but for them you can use apple+left/right arrows. One more thing. In some instruction related to MacOSX apple key is called comm and alt - option.

Povray export in Mercury CSD

One of the nice, new feature of Mercury CSD 2.0 is export to Povray. You can create high quality picture like one below. I used refcode MATCQI04 and Display->Voids and Calculate->Packing&Slicing option to produce the picture. If you are interesting in these as well other features of Mercury please check the article in J. Appl. Cryst. [DOI: 10.1107/S0021889807067908].