Mercurial > hg > autobot
annotate README.txt @ 206:3e5d1c1e302d
* documentation
* fix syntax error
author | Jeff Hammel <jhammel@mozilla.com> |
---|---|
date | Fri, 04 Feb 2011 08:28:21 -0800 |
parents | 5f188d2a36aa |
children | 2d5b78d3411c |
rev | line source |
---|---|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
1 autobot |
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
2 ======= |
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
3 |
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
4 buildbot for the A*Team |
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
5 |
184 | 6 |
7 What is autobot? | |
8 ---------------- | |
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
9 |
184 | 10 autobot is a continuous integration solution for the Automation and |
11 Tools Team. We have a lot of software. We're really talented, so | |
12 usually it doesn't break. But we're not infalliable. Our robot ally, | |
206 | 13 autobot, is there to test things for us. Let's meet autobot! |
184 | 14 |
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
15 |
184 | 16 Installing autobot |
17 ------------------ | |
18 | |
206 | 19 autobot may be installed using the install script:: |
184 | 20 |
21 curl http://k0s.org/mozilla/hg/autobot/raw-file/tip/INSTALL.sh | bash | |
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
22 |
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
23 This will create a virtualenv and install autobot for development |
206 | 24 ($VIRTUAL_ENV/src/autobot). You can also use the script as |
25 instructions and install it in the same manner as any other python package. | |
184 | 26 |
27 | |
28 Setting up a buildmaster and slave | |
29 ---------------------------------- | |
30 | |
31 Once you have autobot installed and the virtualenv activated, you'll | |
32 want to create a buildmaster and a buildslave. | |
33 | |
34 You can create a master-slave pair by running ``create-autobot`` after | |
35 activating the virtualenv. This is mostly useful for autobot | |
36 development. The scripts ``create-autobot-master`` and | |
206 | 37 ``create-autobot-slave`` are also available and are more useful for |
38 production, as normally you will not want to keep the master and the | |
39 slave on the same machine except for development and testing. The | |
40 scripts will prompt you for a factory or you can specify one or more | |
41 from the commandline using ``--factory`` or ``-f``. The factories are | |
42 from ``autobot.projects`` and its subdirectories. You can list the | |
43 available factories with ``create-autobot --list-factories`` (or | |
44 ``create-autobot-master --list-factories``). | |
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
45 |
186 | 46 Running ``create-autobot --help`` will give the variables you can set |
47 when it makes a new bot for you (``create-autobot-master`` and | |
48 ``create-autobot-slave`` also have a ``--help``, the variables in | |
49 ``create-autobot`` being a superset of these). | |
184 | 50 |
186 | 51 If you don't specify a variable, the default will be used to create |
206 | 52 your particular bot. You can change these in the resultant |
53 configuration later. | |
186 | 54 |
55 | |
184 | 56 Using autobot |
57 ------------- | |
58 | |
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
59 The buildmaster and buildslave are started with |
206 | 60 ``buildbot start master`` and ``buildslave stop slave`` from the bot's |
61 directories. Respective ``stop`` commands also exist. If you used the | |
62 ``create-autobot`` command the generated bot will have a | |
63 ``restart_buildbot.py`` script that will stop and start both the | |
64 master and slave and (if debug is set) remove the log as well. | |
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
65 |
206 | 66 The generated ``master.cfg`` file reads values from a ``master.ini`` |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
67 file in the same directory. The master.ini contains a number of |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
68 different sections: |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
69 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
70 * ``[:master:]`` contains the top level configuration for the master |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
71 as well as default settings for slaves |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
72 * sections like ``[slave:slavename]`` indicate a slave of name |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
73 ``slavename`` |
206 | 74 * all other sections are construction parameters for factories |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
75 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
76 The format of the ``.ini`` file is detailed in the subsequent |
206 | 77 section. You may change which .ini file is used by editing the |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
78 ``master.cfg`` file, or, if for whatever reason you don't want to use |
206 | 79 an .ini file as a driver you can construct the appropriate |
80 configuration therein yourself. | |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
81 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
82 An overview of the build status is detailed at the waterfall display, |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
83 by default at http://localhost:8010/waterfall . To force a build, |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
84 click on the desired builder and there will be a force build button |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
85 towards the bottom. |
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
86 |
184 | 87 It is important to remember that continuous integration is a safety |
206 | 88 net, not a first line of defense. In other words, continue to check |
89 your code *before* sending it to autobot. | |
90 | |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
91 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
92 .ini file format |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
93 ---------------- |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
94 |
206 | 95 As stated above, a ``master.ini`` file contains three different types |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
96 of sections: |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
97 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
98 * ``[:master:]`` contains the top level configuration for the master |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
99 as well as default settings for slaves |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
100 * sections like ``[slave:slavename]`` indicate a slave of name |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
101 ``slavename`` |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
102 * other sections are construction parameters for factories |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
103 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
104 What goes in each of these? |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
105 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
106 ``:master:`` The master section contains parameters appropriate to the |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
107 buildmaster: |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
108 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
109 * slaveport: which port to listen for the slaves on |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
110 * htmlport: which port to serve the waterfall display on |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
111 * pollInterval: how often to poll the repositories, in seconds [DEFAULT: 60] |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
112 * treeStable: how long to wait before the tree settles down before |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
113 building [DEFAULT: 60] |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
114 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
115 The other defaults may be seen by running ``create-autobot-master --help``. |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
116 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
117 Other parameters given in the ``[:master:]`` section are used as the |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
118 baseline defaults for each slave. They may be overridden in each |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
119 ``slave:`` section |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
120 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
121 ---- |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
122 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
123 ``slave:`` parameters for each slave: |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
124 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
125 * password: each slave *must* have a password. Unless there's a |
206 | 126 reason to have a different password per slave, its probably better |
127 to put this in the ``:master:`` section | |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
128 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
129 * factories: space-separated list of factories to run on that slave. |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
130 If the special value ``*`` is used, all factories will be run. You |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
131 can view the factories available with |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
132 ``create-autobot --list-factories``. The factory name corresponds to |
206 | 133 the directory (or module) name in ``autobot.projects``. If |
134 every slave is meant to run the same factories, you can put this in | |
135 the ``:master:`` section | |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
136 |
206 | 137 * os: the operating system of the slave. Should be one of ``linux``, |
138 ``win``, or ``mac`` (though see TODO about future use of MozInfo | |
139 making this obselescent). You probably *shouldn't* have a default | |
140 key for this in the ``:master:`` section (though autobot won't try | |
141 to stop you!). | |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
142 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
143 ---- |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
144 |
206 | 145 factory sections: All other sections are used to build the factories. |
146 Each parameter in a factory section is used as a constructor | |
147 (``__init__``) keyword argument when they are created in the | |
148 ``master.cfg``. So a factory section like:: | |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
149 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
150 [foo] |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
151 bar = fleem |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
152 baz = another parameter |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
153 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
154 will invoke the foo factory (lets say it maps to ``MyFooFactory``) on |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
155 each slave like:: |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
156 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
157 MyFooFactory(**dict(bar='fleem', baz='another parameter')) |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
158 |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
159 In addition, if a factory has a special key, ``platform``, the slave |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
160 will pass its platform information when instantiating a factory. |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
161 Currently, this is a dict with a single key, ``os``, but more may be |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
162 added as needed. As noted in the TODO below, ideally this would be |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
163 deprecated entirely by MozInfo but such is the interim solution. |
184 | 164 |
165 | |
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
166 Projects |
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
167 -------- |
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
168 |
184 | 169 What does autobot test? |
170 | |
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
171 * logparser [WORKING] |
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
172 * profilemanager [IN FLIGHT] |
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
173 * mozmill [IN FLIGHT] |
70 | 174 * devicemanager [IN FLIGHT] |
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
175 * firebug [TODO] |
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
176 * jetpack [TODO] |
70 | 177 |
186 | 178 The projects are obtained from any factories in subdirectories of |
179 ``autobot.projects`` (use | |
180 ``python -c 'from autobot import projects; print projects.__file__'`` | |
181 to see this location). | |
182 | |
184 | 183 |
206 | 184 |
184 | 185 Adding a New Project |
186 -------------------- | |
187 | |
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
188 Occassionally, you'll need to add a new project to test. You can add |
206 | 189 a ``__init__.py`` file to a directory under ``autobot.projects`` and, |
190 if you have a factory therein, autobot will find it and add it to the | |
191 list of factories it knows about. | |
192 | |
193 There is also a script, ``create-autobot-project``, that can create | |
194 this stub for you:: | |
195 | |
196 Usage: create-autobot-project [options] project <output> | |
197 | |
198 ``project`` is the name of the project and ``output``, if specified, | |
199 is the path of the file to create. If output is not specified, it | |
200 will find its way to a directory named for the project under | |
201 ``autobot.projects``. | |
186 | 202 |
203 | |
204 Is your autobot being feisty? | |
205 ----------------------------- | |
206 | |
207 Let me know! I'd like to make autobot a solution that works for all | |
208 stake-holders, and if you're reading this, that means you! | |
209 | |
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
210 jhammel@mozilla.com |
186 | 211 |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
212 |
186 | 213 TODO |
214 ---- | |
215 | |
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
216 No software of any size is ever finished. Here are a few things I |
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
217 would like to add: |
186 | 218 |
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
219 * singular checkout of repos on slaves: the slaves should have a |
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
220 singular master repo that is checked out once for each repo URL, |
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
221 branch pair. It is then updated as the slaves need and (e.g.) |
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
222 cloned from there. This should effectively minimize fetch time. |
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
223 |
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
224 * use (as yet non-existent) MozInfo package on the slave: instead of |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
225 specifying the operating system, etc on the master, MozInfo should |
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
226 determine the applicable information from introspection |