Wednesday, June 25, 2014

Python for SysAdmins

Preparing for a interview some time ago I made a list of python module interesting for SysAdmins. Today looking for something else I found that half baked note and decided to polish it enough to put it on the blog. It's mostly for myself as quick reference.

Each module is describe by one, two sentences (from Python documentation) and have a link to official online document. At the end there is a list of example function, object.

Python modules for SysAdmin

import sys

This module provides access to some variables used or maintained by the interpreter and to functions that interact strongly with the interpreter. It is always available.

examples:

  • argv,
  • exit(),
  • path,
  • modules,
  • exec().

 

import os

This module provides a portable way of using operating system dependent functionality.

https://docs.python.org/2/library/os.html

examples:

  • chdir(),
  • getuid(),
  • uname(),
  • listdir(),
  • stat(),
  • rename(),
  • access().

 

import os.path

This module implements some useful functions on pathnames.

https://docs.python.org/2/library/os.path.html

examples:

  • isdir(), 
  • isfile(), 
  • exist(), 
  • getmtime(), 
  • abspath(), 
  • join(), 
  • basename(), 
  • dirname().

 

import time

This module provides various time-related functions.

https://docs.python.org/2/library/time.html

examples:

  • time(), 
  • ctime(), 
  • sleep(), 
  • strftime(), 
  • strptime().

 

import glob

The glob module finds all the pathnames matching a specified pattern according to the rules used by the Unix shell. No tilde expansion is done, but *, ?, and character ranges expressed with [] will be correctly matched.

https://docs.python.org/2/library/glob.html

examples:

  • glob(),
  • iglob().

 

import fnmatch

This module provides support for Unix shell-style wildcards, which are not the same as regular expressions (which are documented in the re module).

https://docs.python.org/2/library/fnmatch.html

examples:

  •  fnmatch().

 

import re

This module provides regular expression matching operations similar to those found in Perl. Both patterns and strings to be searched can be Unicode strings as well as 8-bit strings.

https://docs.python.org/2/library/re.html

examples:

  • compile(), 
  • match(), 
  • search(), 
  • split(), 
  • findall(), 
  • sub(),
  • group().

MatchObject

Match objects always have a boolean value of True. Since match() and search() return None when there is no match, you can test whether there was a match with a simple if statement:

match = re.search(pattern, string)
if match:
    process(match)


Friday, March 28, 2014

Zombie

Usually zombie process is not a big problem, but sometimes... just look at the screenshot below. It wants the whole machine as fried eggs! or maybe boiled?

Tuesday, March 25, 2014

Multiprocessing (in Python)


I needed to do some multi-threading in Python. As I needed effect quick I decided to use standard threading module. However, every time I had to use it I felt it was rather complicated beast. At threading module documentation page there is a link to multiprocessing module. I was tired with threading, on the one hand, but didn't have enough time to learn about greenlet or another competing project, on the other, so I decided to take a quick glance at multiprocessing module...
... And my life became much easier, sky bluer, grass, greener, oh and scripts faster ;-).
I don't do anything special with it, so just one simple code example, but this is very good tutorial you can find much more: http://pymotw.com/2/multiprocessing/communication.html.

Main part of nearly all my scripts looks the same:

import multiprocessing 


SIZE = 30
pool = multiprocessing.Pool(processes=SIZE)
pool_output = pool.map(get_values, servers)
pool.close() # no more tasks
pool.join()

Where servers is a list with servers I need get information from, and get_values is a function (sometimes with a different name).  Simple, isn't it?

Tuesday, February 04, 2014

shell, history and substitution

One of the most known tricks in using shell history is to use:

^old^new
 
to replace string old by new in last command. The only problem is that it replaces only first appearance. But there is another command replacing all string old by new:

!:gs/old/new/

Saturday, December 28, 2013

Space War

Let say it is a late Christmas present for Science Fiction fans. Especially ones who like a lot of science in SF. Two articles discussion how space warfare can look. Have fun.

Friday, December 27, 2013

Even more threads counting

This is small extension to one of my previous posts. This time a loop is enriched by load values (from /proc/loadavg) as well as measurement time (date).  ps command uses the same option, but there is small improvement in awk call.  Rather than count only processes per state it concatenate state with last string in command arguments — I was mostly interested in few java application and jar name was the last parameter for each one. Also awk counts only processes actually running or in uninterruptible sleep ($2 ~ /(D|R)/ at the beginning of awk command).

while [ 1 ];
do
    date;
    cat /proc/loadavg;
    ps -Leo pid,state,args |
     awk ' $2 ~ /(D|R)/ {state[$2 " - (" $1 ") " $NF]++} \
      END{ for (j in state) {printf "%s - %d\n", j, state[j]}}' |
      sort -k 2;
    echo "---";
    sleep 5;
done

Wednesday, December 04, 2013

Debian, Vim and alternatives

I don't know why, but default editor in Debian is not Vim, but Nano. I learned once that it can be change with a command, but has never remembered the actual name of it and since than had to check it each time in install Debian (not so often). Today decided to write it down, not to search too long next time. Maybe it's going to be useful to others:
aptitude install vim.nox
update-alternatives --set editor vim.nox

Search This Blog

Loading...