Fabric as a python module

Introduction

Fabric is a powerful tool to automate running the same command on many remote server. It is written in python, but authors concentrate on usage as command line tool rather as a python module.

There are some documentation how to do this. You can also find some examples in the internet, but  examples never too much, so below another one.

Scenario

We would like to run a command on each of our webservers, ideally in batches, but in a specific order. Our webserver are grouped in 'cells'. Each cell consist of 3 servers in different 'zones' (zone are called a, b, and c). For convince there is following convention of naming servers: "role-cell#-zone" i.e. web-04-a. To make things more interesting some cells are missing, so we cannot make simple loop. On the other hand, there is a service holding various servers information in json file accessible over http. Additionally, we have a python module called 'server' to locally access those information from our workstation. In the 'server' module we have class 'webserver' to deal with webservers. One of its method is 'get_all_server'. It return a list of all webserver.

Script

Servers

First we create the list of all servers using our external tool.

from servers import webserver
allservers = webserver.get_all_severs()


The order mentioned above is to run first all 'c' boxes next 'b' and finally 'a'. To achieve that, let creates 3 lists (in a function). Nothing magic, a bit of 're'.

def make_lists(servers):
    import re # if not imported in the main part

    wa = []
    wb = []
    wc = []
   
    for server in servers:
        if re.search('web-\d+-a', server):
            wa.append(server)
        elif re.search('web-\d+-b', server):
            wb.append(server)
        elif re.search('web-\d+-c', server):
            wc.append(server)

    separation = (wa, wb, wc)
    return separation


Fabric

So now we can start to use fabric. Let import necessary pieces:

# Fabric part
from fabric.api import *
from fabric.network import disconnect_all

set up environment

env.skip_bad_hosts = True
env.warn_only = True
env.parallel = True


and finally call the function:


@parallel(pool_size=10)
def run_remotely():
    with hide('running', 'status'):
       sudo(task)


Main part

Prepare the lists of servers (see paragraph servers):

lists = make_lists(get_servers())

Get the command to run. I don't remember why, but writing my script decided not to pass variable to 'fabric; function, but use global variable.

global task
task = raw_input("Type/paste command to run >")

Now run the provided command in selected order:

for i in [3, 2, 1]:
    execute (run_remotely, hosts=lists[i-1])


For any case disconnect all connection. (It might be not necessary).

disconnect_all