annotate README.txt @ 302:8bc0160993a2

remove reference to autolog
author Jeff Hammel <k0scist@gmail.com>
date Sun, 11 May 2014 09:53:00 -0700
parents b6d0ce3cf430
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
59
e66165f2f31b complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
1 autobot
e66165f2f31b complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
2 =======
e66165f2f31b complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
3
288
b6d0ce3cf430 dereference + repurposes (round1)
Jeff Hammel <k0scist@gmail.com>
parents: 277
diff changeset
4 buildbot automation
59
e66165f2f31b complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
5
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
6
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
7 What is autobot?
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
8 ----------------
59
e66165f2f31b complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
9
288
b6d0ce3cf430 dereference + repurposes (round1)
Jeff Hammel <k0scist@gmail.com>
parents: 277
diff changeset
10 autobot is a continuous integration solution built as a front-end to
b6d0ce3cf430 dereference + repurposes (round1)
Jeff Hammel <k0scist@gmail.com>
parents: 277
diff changeset
11 buildbot. We have a lot of software. We're talented, so
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
12 usually it doesn't break. But we're not infalliable. Our robot ally,
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
13 autobot, is there to test things for us. Let's meet autobot!
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
14
59
e66165f2f31b complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
15
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
16 Installing autobot
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
17 ------------------
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
18
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
19 autobot may be installed using the install script::
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
20
288
b6d0ce3cf430 dereference + repurposes (round1)
Jeff Hammel <k0scist@gmail.com>
parents: 277
diff changeset
21 curl http://k0s.org/hg/autobot/raw-file/tip/INSTALL.sh | bash
59
e66165f2f31b complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
22
e66165f2f31b complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
23 This will create a virtualenv and install autobot for development
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
24 ($VIRTUAL_ENV/src/autobot). You can also use the script as
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
25 instructions and install it in the same manner as any other python package.
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
26
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
27
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
28 Setting up a buildmaster and slave
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
29 ----------------------------------
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
30
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
31 Once you have autobot installed and the virtualenv activated, you'll
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
32 want to create a buildmaster and a buildslave.
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
33
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
34 You can create a master-slave pair by running ``create-autobot`` after
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
35 activating the virtualenv. This is mostly useful for autobot
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
36 development. The scripts ``create-autobot-master`` and
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
37 ``create-autobot-slave`` are also available and are more useful for
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
38 production, as normally you will not want to keep the master and the
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
39 slave on the same machine except for development and testing. The
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
40 scripts will prompt you for a factory or you can specify one or more
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
41 from the commandline using ``--factory`` or ``-f``. The factories are
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
42 from ``autobot.projects`` and its subdirectories. You can list the
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
43 available factories with ``create-autobot --list-factories`` (or
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
44 ``create-autobot-master --list-factories``).
59
e66165f2f31b complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
45
186
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
46 Running ``create-autobot --help`` will give the variables you can set
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
47 when it makes a new bot for you (``create-autobot-master`` and
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
48 ``create-autobot-slave`` also have a ``--help``, the variables in
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
49 ``create-autobot`` being a superset of these).
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
50
186
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
51 If you don't specify a variable, the default will be used to create
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
52 your particular bot. You can change these in the resultant
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
53 configuration later.
186
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
54
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
55
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
56 Using autobot
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
57 -------------
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
58
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
59 The buildmaster and buildslave are started with
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
60 ``buildbot start master`` and ``buildslave stop slave`` from the bot's
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
61 directories. Respective ``stop`` commands also exist. If you used the
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
62 ``create-autobot`` command the generated bot will have a
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
63 ``restart_buildbot.py`` script that will stop and start both the
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
64 master and slave and (if debug is set) remove the log as well.
187
bba97450cbc2 * update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents: 186
diff changeset
65
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
66 The generated ``master.cfg`` file reads values from a ``master.ini``
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
67 file in the same directory. The master.ini contains a number of
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
68 different sections:
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
69
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
70 * ``[:master:]`` contains the top level configuration for the master
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
71 as well as default settings for slaves
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
72 * sections like ``[slave:slavename]`` indicate a slave of name
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
73 ``slavename``
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
74 * all other sections are construction parameters for factories
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
75
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
76 The format of the ``.ini`` file is detailed in the subsequent
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
77 section. You may change which .ini file is used by editing the
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
78 ``master.cfg`` file, or, if for whatever reason you don't want to use
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
79 an .ini file as a driver you can construct the appropriate
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
80 configuration therein yourself.
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
81
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
82 An overview of the build status is detailed at the waterfall display,
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
83 by default at http://localhost:8010/waterfall . To force a build,
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
84 click on the desired builder and there will be a force build button
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
85 towards the bottom.
187
bba97450cbc2 * update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents: 186
diff changeset
86
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
87 It is important to remember that continuous integration is a safety
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
88 net, not a first line of defense. In other words, continue to check
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
89 your code *before* sending it to autobot.
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
90
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
91
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
92 .ini file format
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
93 ----------------
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
94
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
95 As stated above, a ``master.ini`` file contains three different types
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
96 of sections:
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
97
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
98 * ``[:master:]`` contains the top level configuration for the master
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
99 as well as default settings for slaves
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
100 * sections like ``[slave:slavename]`` indicate a slave of name
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
101 ``slavename``
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
102 * other sections are construction parameters for factories
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
103
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
104 What goes in each of these?
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
105
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
106 ``:master:`` The master section contains parameters appropriate to the
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
107 buildmaster:
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
108
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
109 * slaveport: which port to listen for the slaves on
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
110 * htmlport: which port to serve the waterfall display on
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
111 * pollInterval: how often to poll the repositories, in seconds [DEFAULT: 60]
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
112 * treeStable: how long to wait before the tree settles down before
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
113 building [DEFAULT: 60]
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
114
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
115 The other defaults may be seen by running ``create-autobot-master --help``.
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
116
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
117 Other parameters given in the ``[:master:]`` section are used as the
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
118 baseline defaults for each slave. They may be overridden in each
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
119 ``slave:`` section
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
120
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
121 ----
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
122
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
123 ``slave:`` parameters for each slave:
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
124
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
125 * password: each slave *must* have a password. Unless there's a
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
126 reason to have a different password per slave, its probably better
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
127 to put this in the ``:master:`` section
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
128
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
129 * factories: space-separated list of factories to run on that slave.
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
130 If the special value ``*`` is used, all factories will be run. You
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
131 can view the factories available with
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
132 ``create-autobot --list-factories``. The factory name corresponds to
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
133 the directory (or module) name in ``autobot.projects``. If
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
134 every slave is meant to run the same factories, you can put this in
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
135 the ``:master:`` section
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
136
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
137 * os: the operating system of the slave. Should be one of ``linux``,
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
138 ``win``, or ``mac`` (though see TODO about future use of MozInfo
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
139 making this obselescent). You probably *shouldn't* have a default
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
140 key for this in the ``:master:`` section (though autobot won't try
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
141 to stop you!).
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
142
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
143 ----
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
144
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
145 factory sections: All other sections are used to build the factories.
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
146 Each parameter in a factory section is used as a constructor
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
147 (``__init__``) keyword argument when they are created in the
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
148 ``master.cfg``. So a factory section like::
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
149
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
150 [foo]
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
151 bar = fleem
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
152 baz = another parameter
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
153
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
154 will invoke the foo factory (lets say it maps to ``MyFooFactory``) on
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
155 each slave like::
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
156
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
157 MyFooFactory(**dict(bar='fleem', baz='another parameter'))
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
158
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
159 In addition, if a factory has a special key, ``platform``, the slave
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
160 will pass its platform information when instantiating a factory.
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
161 Currently, this is a dict with a single key, ``os``, but more may be
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
162 added as needed. As noted in the TODO below, ideally this would be
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
163 deprecated entirely by MozInfo but such is the interim solution.
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
164
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
165
208
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
166 Sources
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
167 -------
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
168
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
169 ``autobot.process.factory:SourceFactory`` is an abstract base class
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
170 for specifying sources. Sources live as a member on the instance, named
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
171 (oddly) ``sources``. This is a dict with the key being the source
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
172 type and the value being the source to use. Source types are
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
173 currently "git" and "hg", though this is easily extensible.
208
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
174
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
175 Internally, an individual source is stored as a 2-tuple with the first
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
176 item being the URL of the source and the second item being the
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
177 branch. If the branch is ``None`` or not specified, this is assumed
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
178 to equate to ``master`` for git or ``default`` for hg. However, you
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
179 may also use a string in the form of ``URL#branch``. In addition, you
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
180 may specify a whitespace separated source, which ``SourceFactory``
288
b6d0ce3cf430 dereference + repurposes (round1)
Jeff Hammel <k0scist@gmail.com>
parents: 277
diff changeset
181 will split into a list of sources.
208
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
182
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
183 The generated ``master.cfg`` polls for changes on the given source,
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
184 then uses ``buildbot.schedulers.filter:Changefilter`` in conjunction
209
98e3594f2928 notes on the pollers
Jeff Hammel <jhammel@mozilla.com>
parents: 208
diff changeset
185 with a ``Scheduler`` to trigger the appropriate builds. This is done
98e3594f2928 notes on the pollers
Jeff Hammel <jhammel@mozilla.com>
parents: 208
diff changeset
186 by ``GitPoller`` and ``HgPoller`` in
98e3594f2928 notes on the pollers
Jeff Hammel <jhammel@mozilla.com>
parents: 208
diff changeset
187 ``autobot.changes.poller``. Again, more pollers can be added as
98e3594f2928 notes on the pollers
Jeff Hammel <jhammel@mozilla.com>
parents: 208
diff changeset
188 needed or the provided buildbot change sources may be utilized.
208
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
189
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
190 While, in general, the canonical sources should be specified at the
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
191 class level, if an appropriate argument (e.g. *hg* or *git*) is passed
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
192 in to the ``SourceFactory`` constructor, this will override that
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
193 source type. This is useful for testing changes on non-canonical URLs
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
194 and/or branches.
2d5b78d3411c explain how sources work
Jeff Hammel <jhammel@mozilla.com>
parents: 206
diff changeset
195
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
196
184
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
197 Adding a New Project
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
198 --------------------
a5709ef5e275 additional documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 70
diff changeset
199
187
bba97450cbc2 * update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents: 186
diff changeset
200 Occassionally, you'll need to add a new project to test. You can add
206
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
201 a ``__init__.py`` file to a directory under ``autobot.projects`` and,
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
202 if you have a factory therein, autobot will find it and add it to the
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
203 list of factories it knows about.
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
204
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
205 There is also a script, ``create-autobot-project``, that can create
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
206 this stub for you::
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
207
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
208 Usage: create-autobot-project [options] project <output>
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
209
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
210 ``project`` is the name of the project and ``output``, if specified,
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
211 is the path of the file to create. If output is not specified, it
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
212 will find its way to a directory named for the project under
3e5d1c1e302d * documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 204
diff changeset
213 ``autobot.projects``.
186
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
214
210
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
215 Several factories (``BuildFactory`` descendents) are in
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
216 ``autobot.process`` to make building a new project easier:
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
217
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
218 * ``SourceFactory``: as described above, this processes the sources
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
219 and gives a method (``checkout()``) for downloading them.
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
220 ``master.cfg`` looks in each factory and gleems its needed
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
221 repository from its ``sources`` attribute (if any) in the internal
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
222 storage mechanism of ``SourceFactory`` and sets up schedulers
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
223 accordingly.
210
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
224
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
225 * ``VirtualenvFactory``: inherits from ``SourceFactory``. creates a
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
226 python virtualenv and provides build properties ``%(virtualenv)s``,
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
227 ``%(python)s``, and ``%(scripts)s`` (location of ``bin`` on unix or
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
228 ``Scripts`` on windows with respect to the virtualenv. It also has
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
229 a ``findScript()`` method which accepts the unix-style script name
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
230 (no extension) and returns the path to the script on the platform.
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
231
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
232 * ``PythonSourceFactory``: inherits from ``VirtualenvFactory``. It
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
233 treats *all* (for now, can be changed) sources as sources of python
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
234 packages. In addition to checking them out in
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
235 ``%(virtualenv)s/src``, it will also install them (using
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
236 ``%(python)s setup.py install``)
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
237
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
238 See the ``autobot.process.factory`` file for more details. These
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
239 classes are intended as mix-ins (not my favorite pattern, but it seems
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
240 to basically be what buildbot wants you to do). More work needs to be
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
241 done to provide for the breadth of use-cases, but what exists now is
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
242 considered a "good start" versus a "final form". More factories may
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
243 be added per necessity.
6606238e2b07 finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents: 209
diff changeset
244
186
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
245
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
246 Is your autobot being feisty?
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
247 -----------------------------
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
248
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
249 Let me know! I'd like to make autobot a solution that works for all
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
250 stake-holders, and if you're reading this, that means you!
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
251
288
b6d0ce3cf430 dereference + repurposes (round1)
Jeff Hammel <k0scist@gmail.com>
parents: 277
diff changeset
252 k0scist@gmail.com
186
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
253
204
5f188d2a36aa write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents: 187
diff changeset
254
186
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
255 TODO
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
256 ----
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
257
187
bba97450cbc2 * update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents: 186
diff changeset
258 No software of any size is ever finished. Here are a few things I
bba97450cbc2 * update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents: 186
diff changeset
259 would like to add:
186
c7172ca54dff more docs
Jeff Hammel <jhammel@mozilla.com>
parents: 184
diff changeset
260
270
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
261 * email notifications
c77b946e4496 minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents: 267
diff changeset
262
187
bba97450cbc2 * update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents: 186
diff changeset
263 * singular checkout of repos on slaves: the slaves should have a
bba97450cbc2 * update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents: 186
diff changeset
264 singular master repo that is checked out once for each repo URL,
bba97450cbc2 * update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents: 186
diff changeset
265 branch pair. It is then updated as the slaves need and (e.g.)
bba97450cbc2 * update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents: 186
diff changeset
266 cloned from there. This should effectively minimize fetch time.
bba97450cbc2 * update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents: 186
diff changeset
267