autobot
=======

buildbot for the A*Team


What is autobot?
----------------

autobot is a continuous integration solution for the Automation and
Tools Team.  We have a lot of software.  We're really talented, so
usually it doesn't break.  But we're not infalliable.  Our robot ally,
autobot, is there to test things for us.  Let's me autobot!


Installing autobot
------------------

autobot may be installed using the Install script::

 curl http://k0s.org/mozilla/hg/autobot/raw-file/tip/INSTALL.sh | bash

This will create a virtualenv and install autobot for development
($VIRTUAL_ENV/src/autobot). 


Setting up a buildmaster and slave
----------------------------------

Once you have autobot installed and the virtualenv activated, you'll
want to create a buildmaster and a buildslave.

You can create a master-slave pair by running ``create-autobot`` after
activating the virtualenv. This is mostly useful for autobot
development. The scripts ``create-autobot-master`` and 
``create-autobot-slave`` are also available.  The scripts will prompt you
for a factory. The factories are from ``autobot.projects`` and its
subdirectories.  You can list the available factories with
``create-autobot --list-factories`` (or ``create-autobot-master
--list-factories``).

Running ``create-autobot --help`` will give the variables you can set
when it makes a new bot for you (``create-autobot-master`` and
``create-autobot-slave`` also have a ``--help``, the variables in
``create-autobot`` being a superset of these).

If you don't specify a variable, the default will be used to create
your particular bot.  You can (probably) change it later.

 
Using autobot
-------------

The buildmaster and buildslave are started with 
``buildbot start master`` and ``buildslave stop slave``. Respective
``stop`` commands also exist.  If you used the ``create-autobot``
command the generated bot will have a ``restart_buildbot.py`` script
that will stop and start both the master and slave and (if debug is
set) remove the log as well.

The generated ``master.cfg`` file reads values from an ``master.ini``
file in the same directory.  The master.ini contains a number of
different sections:

* ``[:master:]`` contains the top level configuration for the master
  as well as default settings for slaves
* sections like ``[slave:slavename]`` indicate a slave of name
  ``slavename``
* other sections are construction parameters for factories

The format of the ``.ini`` file is detailed in the subsequent
section.  You may change the .ini file used by editing the
``master.cfg`` file, or, if for whatever reason you don't want to use
an .ini file you can construct the appropriate configuration therein
yourself.

An overview of the build status is detailed at the waterfall display,
by default at http://localhost:8010/waterfall . To force a build,
click on the desired builder and there will be a force build button
towards the bottom.

It is important to remember that continuous integration is a safety
net, not a first line of defense.  In other words, continue to check your code
*before* sending it to autobot.

.ini file format
----------------

As stated above, a ``master.ini`` file contains three different kinds
of sections:

* ``[:master:]`` contains the top level configuration for the master
  as well as default settings for slaves
* sections like ``[slave:slavename]`` indicate a slave of name
  ``slavename``
* other sections are construction parameters for factories

What goes in each of these?

``:master:`` The master section contains parameters appropriate to the
buildmaster:

* slaveport: which port to listen for the slaves on
* htmlport: which port to serve the waterfall display on
* pollInterval: how often to poll the repositories, in seconds [DEFAULT: 60]
* treeStable: how long to wait before the tree settles down before
  building [DEFAULT: 60]
  
The other defaults may be seen by running ``create-autobot-master --help``.

Other parameters given in the ``[:master:]`` section are used as the
baseline defaults for each slave.  They may be overridden in each
``slave:`` section

----

``slave:`` parameters for each slave:

* password: each slave *must* have a password.  Unless there's a
  reason to have a password per slave, its probably better to put this
  in the ``:master:`` section

* factories: space-separated list of factories to run on that slave.
  If the special value ``*`` is used, all factories will be run.  You
  can view the factories available with 
  ``create-autobot --list-factories``. The factory name corresponds to
  the directory (or module) name in ``autobot.projects``.

* os: the operating system of the slave. Should be one of linux, win,
  or mac (though see TODO about future use of MozInfo making this
  obselescent). You probably *shouldn't* have a default key for this
  in the ``:master:`` section (though autobot won't try to stop you!).

----

factory sections:  Each parameter in a factory section is used as a
constructor (``__init__``) keyword argument when they are created in
the ``master.cfg``. So a factory section like::

 [foo]
 bar = fleem
 baz = another parameter

will invoke the foo factory (lets say it maps to ``MyFooFactory``) on
each slave like::

 MyFooFactory(**dict(bar='fleem', baz='another parameter'))

In addition, if a factory has a special key, ``platform``, the slave
will pass its platform information when instantiating a factory.
Currently, this is a dict with a single key, ``os``, but more may be
added as needed.  As noted in the TODO below, ideally this would be
deprecated entirely by MozInfo but such is the interim solution.


Projects
--------

What does autobot test?

* logparser      [WORKING]
* profilemanager [IN FLIGHT]
* mozmill        [IN FLIGHT]
* devicemanager  [IN FLIGHT]
* firebug        [TODO]
* jetpack        [TODO]

The projects are obtained from any factories in subdirectories of
``autobot.projects`` (use 
``python -c 'from autobot import projects; print projects.__file__'`` 
to see this location).


Adding a New Project
--------------------

Occassionally, you'll need to add a new project to test.  You can add
a ``__init__.py`` file


Is your autobot being feisty?
-----------------------------

Let me know! I'd like to make autobot a solution that works for all
stake-holders, and if you're reading this, that means you!

jhammel@mozilla.com


TODO
----

No software of any size is ever finished.  Here are a few things I
would like to add:

* singular checkout of repos on slaves: the slaves should have a
  singular master repo that is checked out once for each repo URL,
  branch pair.  It is then updated as the slaves need and (e.g.)
  cloned from there.  This should effectively minimize fetch time.

* use (as yet non-existent) MozInfo package on the slave: instead of
  specifying the operating system, etc on the master, MozInfo should
  determine the applicable information from introspection