Vim substitution

Let say that you want to add string 'bprdp' and the end of each line beginning with string 'bc' and ending with comma you should use following command:

:% g/^bc/s/\,$/, bprdp/

% means the whole file g/ for each line with pattern after '/' in above case pattern is ^bc line beginning with bc s/\,$/, bprdb/ substitute comma (\,) followed by end of line character ($) with ', bprdb'.

I wrote this message based on Vim regular expression and Vim Command Cheat Sheet.

Control the Vim from the edited file

One of the very nice Vim feature I've learnt recently is the possibility of controlling the Vim from a edited file. Chosen Vim commands may to be put in one of the first (specially formatted) file lines. The line format is describe in 'modeline' help keyword (:help modeline). It worth to remember that text before and after main part has to be commenting out directive. Therefore, for example the line in HTML might looks similar to:

<-- vim: set tabstop=4 noexpandtab:-->

for python:

# vim: tabstop=4 noexpandtab:

If you like to learn more please check the modeline keyword in Vim help.

update: I forgot to add that you need to set modeline in .vimrc file.

My first Perl script

It's nothing big, but it's the first one and, as Perl is write only language, I'd better add the short description. The script takes a list of files passed as arguments to the command; reads all lines (http addresses) from them and creates the list of unique domains names.

#!/usr/bin/env perl

%seen = ();
foreach (@ARGV)
{
open (LFILE,"$_");

for $line ()
{
       @sline=split(/\//,$line);
       print ("@sline[2]\n") unless $seen{@sline[2]}++;
}

close LFILE;
}

Perl tutorial from tizag.com was helpful.

DarwinPorts via proxy

Recently, I needed a perl module not present on my MacOSX computer, which was behind a proxy. The friend suggested to use the Darwin ports rather the Perl from Apple. I downloaded and installed it to found I cannot install any port. The problem was due to the combination of using a proxy and the sudo rather then the root user. I guess such combination is rather common among MacOSX-perl users. So below I present the command which allows to use the Darwin ports from a normal MacOSX account. Generally note, you have to export both the RSYNC_PROXY as well as the http_proxy in the sudo environment.

sudo sh -c "export RSYNC_PROXY=proxy.server:port; \
export http_proxy=http://proxy.server:port; \
port install perl5.10 "

How to find the line not starting with "X" in Vim

I don't know why but most of Vim's search examples say nothing about how to find the line not starting with string "X". Finally, I found how to do this. I.e. following line find anything not started from del:

^[^d][^e][^l].*

For people not advance in regex. The consecutive signs means:

• ^ - a line starting with
• [^d] - character other than d;
• [^e] - character other than e;
• [^l] - character other than l;
• .* - any string (any character repeated any times).

Total size of quite new files

Following command count the size of all files (-type f) newer than 10 days (-mtime -10). The size is printed in megabytes. "%k" argument of printf returns size in kilobytes, but "a/1024" in awk change it to megabytes.

find -type f -mtime -10 -printf "%k\n"| \
 awk 'BEGIN {a=0} {a=a+$1} END {print a/1024}'

Remote diff

When you are working as a Linux SysAdmin quite often you have to compare files from two different machines. I found (here) the script which made my life easier, but after some time decided to customize and extent it a bit. Usually I compare file in the same location therefore the first argument of my script is file path. I also gave chance user to pass an argument for the diff command (4th argument, the default is '-b').

#!/bin/bash
#
# this acts as a remote diff program, accepting two files and displaying
# a diff for them.  Zero, one, or both files can be remote.  File paths
# must be in a format `scp` understands: [[user@]host:]file

[ -n "$1" ] || [ -n "$2" ] || [ -n "$3" ] || \
{ echo "Usage: `basename $0` file1 server1 server2" && exit 1;}

if test -e $4 
then
       opt="-b"
else
       opt=$4
fi

scp "$2:$1" rdiff.1 >& /dev/null
scp "$3:$1" rdiff.2 >& /dev/null
diff $opt rdiff.1 rdiff.2
rm -f rdiff.1 rdiff.2

How to create pictures thums in the one line

The following command can be paste as a one line

for i in `find /path/to/directory/with/pictures  -iname "*.JPG"`;\
do\
convert $i -resize 800x600 `dirname $i`/thumb-`basename $i`;\
done

The find command return a whole path to a file. But we want to add thumb- before the actual name of a file. Therefore `dirname $i` ensure that convert get the proper path and `basename $i` the actual file name (preceded by thumb-). It is also worth to note the iname option of find command. It is case insensitive version of the name.

Another Awk one-liner

This one-liner printing third word from the lines beginning with "Mem:" (precisely the lines which first word is "Mem:") but adding the serial number before the word.

awk 'BEGIN {a=1} \
{if ( $1 == "Mem:" ) \
  {printf "%4d  %s\n",  a, $3; a++}}' \
free-prefork.log >mem-prefork.log

BTW. This script is to help me to plot a gnuplot graph based on "used memory" number from free command. plot "mem-prefork.log" will do the rest of job. UPDATE Mike Hommey post force me to rethink my scripts and I found easy way to eliminate if clause:

awk 'BEGIN {a=1} \
/^Mem:/  {printf "%4d  %s\n",  a, $3; a++}' \
free-prefork.log >mem-prefork.log

Gnuplot with readline on MacOSX

Recently, I tried to use gnuplot on a Mac, and, of course, it wasn't working properly. Apple prepared and shipped with MacOS its own (broken) version of readline library with didn't work with gnuplot (known bug). So I grabbed the sources of readline, applied all of patches and built mine version of readline. Next, I added the proper option to gnuplot configure file, but it wasn't pass to makefile. I looked into the Makefiles and found the the TERMLIBS option had to be change. I.e. using such command:

find . -name Makefile \
-exec sed -i.old "s/TERMLIBS\ =/TERMLIBS = -L\/usr\/local\/lib/" {} \; 

Two more things about gnuplot and MacOSX.

1. I started to think to make a gnuplot.app for MacOSX, but sure how it should work.
2. I found that X11 term is much better then Aqua, in particular, it's allow to rotate 3D (s)plots.

Header for bonnie++ csv file

Recently I started to use bonnie++ to perform some disk tests. One of the thing annoy me is that bonnie++ output has no the 'csv header' describing what is in which column. It makes overview of results in a spreadsheet rather hard. Therefore I created my own one.

,,Sequential,Output,,,,,Sequential,Input,,,Random,,Sequential,Create,,,,,Random,Create,
,,Per Chr, ,Block, ,Rewrite, ,Per Chr, ,Block, ,Seeks, ,Create, ,Read, ,Delete, ,Create, ,Read, ,Delete
Machine,Size,K/sec,%CP,K/sec,%CP,K/sec,%CP,K/sec,%CP,K/sec,%CP,/sec,%CP,files,/sec,%CP,/sec,%CP,/sec,%CP,/sec,%CP,/sec,%CP,/sec,%CP

You can save above lines in the file called i.e. bonnie-header.csv and then cat it before csv part of bonnie.out file (of course it can have different name), by:

cat bonnie-header.csv `tail -1 bonnie.out` >bonnie.csv

After that the output should looks similar to this one: BTW. I found that default number of files creating for metadata benchmarks is low, so I increased it to 128.

BigPicture

I don't know how I might not to add this earlier but I didn't. The Big Picture it's a on-line addition to Boston Globe (I guess it only on-line, but I don't reader Boston Globe, as I'm leaving in proper Cambridge, at least at the moment). I saw it first time on Bad Astronomy Blog. That story was about picture of our sky but made from above. The pictures were amazing, but what more important that Big Picture brings new great pictures month after month. It's quite interesting how many of them are space/astronomy related, i.e. this one:

Polish UK X11 keybord layout

The UK and US keyboard layout are different. Polish one, as probably most other ones based on American. It's fine in Poland, but not so fine in UK. Therefore, I decided to create Polish layout, but based on UK keyboard. I based on Mikoła Kosmulski article describing how to prepare mixture of Polish and German layouts. My kebord setting looks this way:

partial default alphanumeric_keys
xkb_symbols "basic" {

    include "latin"

    name[Group1]="Poland based on GB";

    key   { [         q,          Q ] };
    key   { [         w,          W ] };
    key   { [         e,          E,      eogonek,      Eogonek ] };
    key   { [         o,          O,       oacute,       Oacute ] };

    key   { [         a,          A,      aogonek,      Aogonek ] };
    key   { [         s,          S,       sacute,       Sacute ] };
    key   { [         f,          F ] };

    key   { [         z,          Z,    zabovedot,    Zabovedot ] };
    key   { [         x,          X,       zacute,       Zacute ] };
    key   { [         c,          C,       cacute,       Cacute ] };
    key   { [         n,          N,       nacute,       Nacute ] };

    key   { [         2,   quotedbl,  twosuperior,    oneeighth ] };
    key   { [         3,   sterling, threesuperior,    sterling ] };
    key   { [         4,     dollar,     EuroSign,   onequarter ] };

    key   { [apostrophe,         at, dead_circumflex, dead_caron] };
    key   { [     grave,    notsign,          bar,          bar ] };

    key   { [numbersign, asciitilde,   dead_grave,   dead_breve ] };
    key   { [ backslash,        bar,          bar,    brokenbar ] };

    include "kpdl(comma)"

    include "level3(ralt_switch)"
};

Trash in Ubuntu

My father had a strange problem. He couldn't delete few files from his Trash. First we figured out that he didn't have 'write' rights to them, even if he saved them from a mail. My first suggestion was to change permissions, but he couldn't (using Nautilus). Today I had a chance to sit in front of dad's computer, but I couldn't find where is the Trash. I decided to check if there were any similar bug report in the launchpad. I found this suggestion, which work for me (of course I need to make chmod -R +w ~/.local/share/Trash/files/*). And the point I would like to save here is that the trash in Ubuntu is in:

~/.local/share/Trash/files/

Cordless USB phone - not working with Ubuntu

This time no-success story. I bought the WP-01 VoIP Dect Cordless Handset on ebay (from shop4USB). In Product Description there was an info that it works with "Linux2.4 above". In reality it was working only partly, very partly. Indeed the only part working was sound (as normal sound card), it means I could hear and speak after /choose a number/pick up a call/ using Skype software. Anything ( ring, Keyboard and LCD) wasn't working. Phone was recognize by system and asign to USB HID module:
dmesg |grep hid [ 33.859714] usbcore: registered new interface driver hiddev [ 33.866820] hiddev96hidraw0: USB HID v1.00 Device [HID 06e6:c31c] on usb-0000:00:1f.4-2.2
I tried to make yealink module controlling phone.
rmmod ubhid modprobe yealink
But, how I expected, it didn't help. Finally, the phone USB info was/is
06e6:c31c Tiger Jet Network, Inc.

PS. The seller was OK and gave me my money back.

Swap - VMware effects and parallelization

VMare and swap

To present our software marketing and scientific represents use Windows laptopts with Linux in VMWare. They need Linux because our web based product (Relibase+, IsoStar and incoming WebCSD) working only on it. Using virtualization shouldn't be a problem as a machine has 2GB of memory and we can assign 1GB to guest OS. However, recently we couldn't start WebCSD, not only guest was affected but also host froze. VMWare has problem with I/O operation so we were suspicious about disk usage, but the server didn't need to much of it. Anyway I went I/O trace and decided to turn off the swap. After that server started to work as a rocket!

Parallelization of a swap partitions

I was browins through IBM developersWorks and found info that you can parallelize a swap partition.

Amazingly, all modern Linux kernels, by default (with no special kernel options or patches) allow you to parallelize swap, just like a RAID 0 stripe. By using the pri option in /etc/fstab to set multiple swap partitions to the same priority, we tell Linux to use them in parallel:

/dev/sda2 none swap sw,pri=3 0 0
/dev/sdb2 none swap sw,pri=3 0 0
/dev/sdc2 none swap sw,pri=3 0 0
/dev/sdd2 none swap sw,pri=1 0 0

Creating service starting script

One of our customer ask what to do to start IsoStar server during system start. Our customer was using Centos (RedHat derivative) so I suggested to use the command line tool called chkconfig [1, 2]. TO use this tool you need a special script starting Isostar Apache server. The sample of such script can be found here [3]. Here is the example instruction. Let me assume that isostar is going to /opt directory.

1. Copy the script from link [3] to your HDD and call it isostar_server
2. Change line:

   /path/to/command/to/start/new-service

   to:

   /opt/csd/isostar/APACHE/bin/ccdc_apache start

   and line:

   /path/to/command/to/stop/new-service

   to:

   /opt/csd/isostar/APACHE/bin/ccdc_apache stop

3. Remove following lines (from both start and stop subsection):

         #Or to run it as some other user:
   /bin/su - username -c /path/to/command/to/start/new-service
   echo "."

4. Change 'new-service' in 'echo -n' lines to 'isostar_server'.
5. Now as a root copy isostar_server file into /etc/init.d/
6. Again as a root invoke chkconfig and add isostar_server: /sbin/chkconfig --add isostar_server

There are very similar tool for Debian (Ubuntu etc.) called update-rc.d [4, 5]. You should be able to use following command with the same (very similar script): update-rc.d isostar_server start 90 2 3 4 5 . stop 10 0 1 6.

Links

[1] http://linux.die.net/man/8/chkconfig
[2] http://spiralbound.net/2006/11/15/controlling-services-with-chkconfig
[3] http://spiralbound.net/2007/07/23/example-linux-init-script
[4] http://wiki.linuxquestions.org/wiki/Update-rc.d
[5] http://www.annodex.net/cgi-bin/man/man2html?update-rc.d+8

Count a sum of sizes of selected files in a directory

I would like to count how much space taking Qt4 libraries for the MacOSX universal build of one from our application. So I used following command:

du -m * | sort -nr | grep Qt | awk '{sum+=$1} END {print sum}'

• du -m - print used size in megabytes (for directory do not forgot about -s option);
• sort -nr - sort in reverse, numerical order;
• grep Qt - left only lines with Qt (you can also first grep and next sort lines);
• {sum+=$1} - adding a value from first column of each line to variable sum;
• END {print sum} - printing variable sum on after going through all of lines.

The electronic structure of selected betaine dyes. A quantum chemical study

This thesis presents electronic absorption spectra, non linear optical properties and geometrical parameters of betaine dyes obtained by quantum chemical calculations.

Four betaines [4-(1-piridinium-phenolan), 3-(1-piridinium-phenolan), 2-(1-piridinium-phenolan) and 4-(1-piridinium-thiophenolan)] were selected for the study.

During the research various ab initio methods were applied. The Hartee-Fock method (HF) and the second order Møller-Plesset perturbation theory (MP2) were used to determine a geometrical and NLO properties. Moreover, the NLO were obtained using chosen variants of the coupled cluster metod (CC2, CCSD) and the geometry optimizations were perform using the Density Functional Theory (DFT/B3LYP) as well as complete active space methods (CASSCF and CASPT2). In the case of spectroscopic properties the CC and CASSCF/CASPT2 methods along with Time-Dependant DFT (with B3LYP, PBE0 and CAM-B3LYP functionals) and the Configuration Interaction with Singles (CIS and CIS(D)) were used.

Results obtained during the study indicate that the correct description of betaine dyes' electronic structure is an unusually demanding test for present quantum chemical methods. It is safe to say, that, for all of the investigated parameters, the electron correlation has to be take into account. It is also worth to notify that basis set selection is less important. However, diffuse and polarisation functions should be included in the case of spectroscopic and optical properties.

The presented computational result confirmed the very strong interaction between a betaine molecule and its environment. One of the outcome of this phenomena is a large difference between experimental results (usually obtained in condense phases) and theoretical data (calculated in vacuum). Another observation verified during project is the significant increase of the betaines' NLO by the conformational shifting. Finally, it is worth notify that the largest NLO response was obtained for 4-(1-piridinium-thiophenolan).

How you can quest I'm a doctor now! I defended my thesis (abstract above) 19th of June 2008 and my degree was confirmed by faculty of Chemistry board 26th of June. Thesis was written in Polish, so it isn't very useful for most of the world. However, there is the appendix with all available theoretical data of geometrical and spectroscopic parameters of betaine dyes. Additionally, some of results was published in following articles: JMM-11, JMM-13 and LETT-411 (I hope to write one, maybe more). PDF with thesis can be download here.

If I find some time I will might prepare English version of mentioned appendix, and of course, I will share my LaTeX, gnuplot, computational experience.

GRUB and why root!=root

It one of the this 'keep in mind notes'. I tried to install the Fedora 9 along the Ubuntu 8.04. The Fedora went onto the sdb3 partition or (hd1,2) in the GRUB notation. I added following lines to the Ubuntu /boot/grub/menu.lst file. (I didn't install GRUB from Fedora).

title        Fedora 9
root        (hd1,2)
kernel        /boot/vmlinuz-2.6.25-14.fc9.i686 root=/dev/sdb3 ro
initrd        /boot/initrd-2.6.25-14.fc9.i686.img

But it wanted to work. All the time, I got Error 2 : Bad file or directory type. After trying many things I figured out that problem is lack of the GRUB files at the Fedora partition, so I updated grub config file.

title        Fedora 9
root        (hd0,0)
kernel        (hd1,2)/boot/vmlinuz-2.6.25-14.fc9.i686 root=/dev/sdb3 ro
initrd        (hd1,2)/boot/initrd-2.6.25-14.fc9.i686.img

Finally, the Fedora started to boot.