Fabric Recipes

This page is intended to let users share small snippets of code with each other. The snippets should be small, generally useful (ie. not too specific) and self-contained.

They may show of cute hacks, extend existing functionality or simply be functions that do something useful.

Fabric's Own Fabfile

Fabric has a fabfile of its own which is used for tagging, releasing, pushing docs, running tests and so forth. While many of its operations are local-only, it should still serve as a decent introduction to a real fabfile and what the syntax looks like. Check it out on GitHub or code.fabfile.org.

Files and Directories

Recipes for working with files and directories.

Improved Directory Upload

Improvement on fabric.contrib.project.upload_project, for cases where one cannot use rsync for some reason. Use the function like put_dir("/localpath","/remotepath"). /remotepath gets silently removed if it exists, a bit dangerous but necessary for re-deployment. Format strings use Python 2.6 .format method, and I suppose adversary could spoof /tmp/fabtemp.tgz.

   1 def _put_dir(local_dir, remote_dir):
   2     """Copy a local directory to a remote one, using tar and put. Silently
   3     overwrites remote directory if it exists, creates it if it does not
   4     exist."""
   5     local_tgz = "/tmp/fabtemp.tgz"
   6     remote_tgz = os.path.basename(local_dir) + ".tgz"
   7     local('tar -C "{0}" -czf "{1}" .'.format(local_dir, local_tgz))
   8     put(local_tgz, remote_tgz)
   9     local('rm -f "{0}"'.format(local_tgz))
  10     run('rm -Rf "{0}"; mkdir "{0}"; tar -C "{0}" -xzf "{1}" && rm -f "{1}"'\
        .format(remote_dir, remote_tgz))

   1 from fabric.api import run, abort
   2 
   3 def compare_versions(x, y):
   4     """
   5     Expects 2 strings in the format of 'X.Y.Z' where X, Y and Z are
   6     integers. It will compare the items which will organize things
   7     properly by their major, minor and bugfix version.
   8     ::
   9     
  10         >>> my_list = ['1.13', '1.14.2', '1.14.1', '1.9', '1.1']
  11         >>> sorted(my_list, cmp=compare_versions)
  12         ['1.1', '1.9', '1.13', '1.14.1', '1.14.2']
  13 
  14     """
  15     def version_to_tuple(version):
  16         version_list = version.split('.', 2)
  17         if len(version_list) <= 3:
  18             [version_list.append(0) for x in range(3 - len(version_list))]
  19         try:
  20             return tuple(map(int, version_list))
  21         except ValueError: # not an integer, so it goes to the bottom
  22             return (0,0,0)
  23 
  24     x_major, x_minor, x_bugfix = version_to_tuple(x)
  25     y_major, y_minor, y_bugfix = version_to_tuple(y)
  26     return cmp(x_major, y_major) or cmp(x_minor, y_minor) or cmp(x_bugfix, y_bugfix)
  27 
  28 def symlink_latest(base_dir=None, cmp=None, destination=None):
  29     """
  30     Given a `base_dir` which contains folders, sort using `cmp` and
  31     symlink the first to `destination`.
  32     """
  33     if not destination and not base_dir:
  34         abort('You must specify both base_dir and destination as keyword arguments.')
  35     base_dir = base_dir.rstrip('/')
  36     dirs = run("ls %s/" % base_dir).split('\n')
  37     if len(dirs) > 1:
  38         dirs = sorted(dirs, cmp=cmp)
  39     # remove existing symlink if it exists
  40     run("if [ -z %(dest)s ]; then rm %(dest)s;  fi" % {'dest': destination})
  41     run("ln -s %s %s" % (os.path.join(base_dir, dirs[-1]), destination))

Wait for line

This is a function that uses tail -F and grep to read from a certain file (on the remote host) until the specified line appears. This is useful if you want to wait for some event to happen.

An example use case could be, that you have a command/task that reboots some service, but you want to wait for the given service to be fully operational before you start rebooting the next. You could then use this function to wait for the service to print "Ready!" or some such in its log-file.

   1 def waitForLine(fname, linePattern, grepArgs=''):
   2   run("tail -F '%s' | grep -m 1 %s '%s'" % (fname, grepArgs, linePattern))

Misc.

Recipes that don't fit elsewhere.

Recursive string interpolation

With the move from version 0.1.x to 0.9.x, Fabric lost its special recursive string interpolation syntax. This function uses string.Template for the interpolation and adds to it our beloved recursive application. You need to call it explicitly, though, as it isn't applied auto-magically like in ye olden days.

   1 from string import Template
   2 from fabric.api import env
   3 
   4 def fmt(s, **kwargs):
   5     """
   6     Recursively interpolate values into the given format string, using
   7     ``string.Template``.
   8 
   9     The values are drawn from the keyword arguments and ``env``, in that order.
  10     The recursion means that if a value to be interpolated is itself a format
  11     string, then it will be processed as well.
  12 
  13     If a name cannot be found in the keyword arguments or ``env``, then that
  14     format-part will be left untouched.
  15 
  16     Examples::
  17 
  18         >>> fmt("$shell 'echo $PWD'")
  19         "/bin/bash -l -c 'echo $PWD'"
  20 
  21         >>> fmt("echo $$shell")
  22         'echo $shell'
  23 
  24         >>> fmt("a is ($a)", a="b is ($b/$c)", b=1, c=2)
  25         'a is (b is (1/2))'
  26     """
  27     data = {}
  28     data.update(env)
  29     data.update(kwargs)
  30     for k, v in data.items():
  31         if isinstance(v, basestring) and '$' in v:
  32             data[k] = fmt(v, **data)
  33     return Template(s).safe_substitute(data)

Load hosts from a file

   1 def hostsfile(group):
   2     """Loads a group of hosts from a config file.
   3 
   4     group: name of the group file, one host per line
   5     """
   6     base_dir = '~/.dsh/group/'
   7 
   8     from os.path import join, abspath, expanduser
   9 
  10     filename = abspath(expanduser(join(base_dir, group)))
  11     try:
  12         fhosts = open(filename)
  13     except IOError:
  14         abort('file not found: %s' % filename)
  15 
  16     def has_data(line):
  17         """'line' is not commented out and not just whitespace."""
  18         return line.strip() and not line.startswith('#'):
  19 
  20     config.fab_hosts = [ line.strip() for line in fhosts
  21                         if has_data(line)]

Run command

   1 def runcmd(arg):
   2     """
   3     A general execute function for running specified command on hosts.
   4     """
   5     if env.user != "root":
   6         sudo("%s" % arg)
   7     else:
   8         run("%s" % arg)

Interactive command

This is a bit of a hack for situations that require an interactivity. Non-linux users may have to change the TERM value.

def manual(cmd):
    pre_cmd = "ssh -p %(port)s %(user)s@%(host)s export " \
              "TERM=linux && " % env
    local(pre_cmd + cmd, capture=False)

- Jesse Heitler

Background command

On remote *nix type machines I use this to run background jobs. It uses the lightweight dtach command, so make sure that is installed.

def runbg(cmd, sockname="dtach"):
    return run('dtach -n `mktemp -u /tmp/%s.XXXX` %s'  % (sockname,cmd))

- Erich Heine

Recipes (last edited 2010-06-23 15:35:46 by ErichHeine)