Mercurial > hg > autobot

comparison README.txt @ 270:c77b946e4496

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
minor fixes to the README
author Jeff Hammel <jhammel@mozilla.com>
date Tue, 31 Jul 2012 09:49:57 -0700
parents afa37f032c39
children 22bb1ac50eb7
comparison
equal deleted inserted replaced
269:8f9862788dce 270:c77b946e4496
31 Once you have autobot installed and the virtualenv activated, you'll 31 Once you have autobot installed and the virtualenv activated, you'll
32 want to create a buildmaster and a buildslave. 32 want to create a buildmaster and a buildslave.
33 33
34 You can create a master-slave pair by running ``create-autobot`` after 34 You can create a master-slave pair by running ``create-autobot`` after
35 activating the virtualenv. This is mostly useful for autobot 35 activating the virtualenv. This is mostly useful for autobot
36 development. The scripts ``create-autobot-master`` and 36 development. The scripts ``create-autobot-master`` and
37 ``create-autobot-slave`` are also available and are more useful for 37 ``create-autobot-slave`` are also available and are more useful for
38 production, as normally you will not want to keep the master and the 38 production, as normally you will not want to keep the master and the
39 slave on the same machine except for development and testing. The 39 slave on the same machine except for development and testing. The
40 scripts will prompt you for a factory or you can specify one or more 40 scripts will prompt you for a factory or you can specify one or more
41 from the commandline using ``--factory`` or ``-f``. The factories are 41 from the commandline using ``--factory`` or ``-f``. The factories are
42 from ``autobot.projects`` and its subdirectories. You can list the 42 from ``autobot.projects`` and its subdirectories. You can list the
43 available factories with ``create-autobot --list-factories`` (or 43 available factories with ``create-autobot --list-factories`` (or
44 ``create-autobot-master --list-factories``). 44 ``create-autobot-master --list-factories``).
45 45
46 Running ``create-autobot --help`` will give the variables you can set 46 Running ``create-autobot --help`` will give the variables you can set
47 when it makes a new bot for you (``create-autobot-master`` and 47 when it makes a new bot for you (``create-autobot-master`` and
48 ``create-autobot-slave`` also have a ``--help``, the variables in 48 ``create-autobot-slave`` also have a ``--help``, the variables in
50 50
51 If you don't specify a variable, the default will be used to create 51 If you don't specify a variable, the default will be used to create
52 your particular bot. You can change these in the resultant 52 your particular bot. You can change these in the resultant
53 configuration later. 53 configuration later.
54 54
55 55
56 Using autobot 56 Using autobot
57 ------------- 57 -------------
58 58
59 The buildmaster and buildslave are started with 59 The buildmaster and buildslave are started with
60 ``buildbot start master`` and ``buildslave stop slave`` from the bot's 60 ``buildbot start master`` and ``buildslave stop slave`` from the bot's
61 directories. Respective ``stop`` commands also exist. If you used the 61 directories. Respective ``stop`` commands also exist. If you used the
62 ``create-autobot`` command the generated bot will have a 62 ``create-autobot`` command the generated bot will have a
63 ``restart_buildbot.py`` script that will stop and start both the 63 ``restart_buildbot.py`` script that will stop and start both the
64 master and slave and (if debug is set) remove the log as well. 64 master and slave and (if debug is set) remove the log as well.
109 * slaveport: which port to listen for the slaves on 109 * slaveport: which port to listen for the slaves on
110 * htmlport: which port to serve the waterfall display on 110 * htmlport: which port to serve the waterfall display on
111 * pollInterval: how often to poll the repositories, in seconds [DEFAULT: 60] 111 * pollInterval: how often to poll the repositories, in seconds [DEFAULT: 60]
112 * treeStable: how long to wait before the tree settles down before 112 * treeStable: how long to wait before the tree settles down before
113 building [DEFAULT: 60] 113 building [DEFAULT: 60]
114 114
115 The other defaults may be seen by running ``create-autobot-master --help``. 115 The other defaults may be seen by running ``create-autobot-master --help``.
116 116
117 Other parameters given in the ``[:master:]`` section are used as the 117 Other parameters given in the ``[:master:]`` section are used as the
118 baseline defaults for each slave. They may be overridden in each 118 baseline defaults for each slave. They may be overridden in each
119 ``slave:`` section 119 ``slave:`` section
126 reason to have a different password per slave, its probably better 126 reason to have a different password per slave, its probably better
127 to put this in the ``:master:`` section 127 to put this in the ``:master:`` section
128 128
129 * factories: space-separated list of factories to run on that slave. 129 * factories: space-separated list of factories to run on that slave.
130 If the special value ``*`` is used, all factories will be run. You 130 If the special value ``*`` is used, all factories will be run. You
131 can view the factories available with 131 can view the factories available with
132 ``create-autobot --list-factories``. The factory name corresponds to 132 ``create-autobot --list-factories``. The factory name corresponds to
133 the directory (or module) name in ``autobot.projects``. If 133 the directory (or module) name in ``autobot.projects``. If
134 every slave is meant to run the same factories, you can put this in 134 every slave is meant to run the same factories, you can put this in
135 the ``:master:`` section 135 the ``:master:`` section
136 136
137 * os: the operating system of the slave. Should be one of ``linux``, 137 * os: the operating system of the slave. Should be one of ``linux``,
138 ``win``, or ``mac`` (though see TODO about future use of MozInfo 138 ``win``, or ``mac`` (though see TODO about future use of MozInfo
139 making this obselescent). You probably *shouldn't* have a default 139 making this obselescent). You probably *shouldn't* have a default
140 key for this in the ``:master:`` section (though autobot won't try 140 key for this in the ``:master:`` section (though autobot won't try
141 to stop you!). 141 to stop you!).
142 142
143 ---- 143 ----
144 144
145 factory sections: All other sections are used to build the factories. 145 factory sections: All other sections are used to build the factories.
146 Each parameter in a factory section is used as a constructor 146 Each parameter in a factory section is used as a constructor
147 (``__init__``) keyword argument when they are created in the 147 (``__init__``) keyword argument when they are created in the
148 ``master.cfg``. So a factory section like:: 148 ``master.cfg``. So a factory section like::
149 149
150 [foo] 150 [foo]
151 bar = fleem 151 bar = fleem
152 baz = another parameter 152 baz = another parameter
153 153
166 Projects 166 Projects
167 -------- 167 --------
168 168
169 What does autobot test? 169 What does autobot test?
170 170
171 * mozbase [WORKING]
171 * logparser [WORKING] 172 * logparser [WORKING]
172 * profilemanager [IN FLIGHT] 173 * profilemanager [IN FLIGHT]
173 * mozmill [IN FLIGHT] 174 * mozmill [IN FLIGHT]
174 * devicemanager [IN FLIGHT] 175 * devicemanager [IN FLIGHT]
175 * firebug [TODO] 176 * firebug [TODO]
176 * jetpack [TODO] 177 * jetpack [TODO]
178 * talos [TODO]
177 179
178 The projects are obtained from any factories in subdirectories of 180 The projects are obtained from any factories in subdirectories of
179 ``autobot.projects`` (use 181 ``autobot.projects`` (use
180 ``python -c 'from autobot import projects; print projects.__file__'`` 182 ``python -c 'from autobot import projects; print projects.__file__'``
181 to see this location). 183 to see this location).
182 184
183 185
184 Sources 186 Sources
185 ------- 187 -------
186 188
187 ``autobot.process.factory:SourceFactory`` is an abstract base class 189 ``autobot.process.factory:SourceFactory`` is an abstract base class
188 for specifying sources. Sources live as a member on the instance, named 190 for specifying sources. Sources live as a member on the instance, named
189 (oddly) ``sources``. This is a dict with the key being the source 191 (oddly) ``sources``. This is a dict with the key being the source
190 type and the value being the source to use. Source types are 192 type and the value being the source to use. Source types are
191 currently "git" and "hg", though this is easily extensible. 193 currently "git" and "hg", though this is easily extensible.
192 194
193 Internally, an individual source is stored as a 2-tuple with the first 195 Internally, an individual source is stored as a 2-tuple with the first
194 item being the URL of the source and the second item being the 196 item being the URL of the source and the second item being the
195 branch. If the branch is ``None`` or not specified, this is assumed 197 branch. If the branch is ``None`` or not specified, this is assumed
196 to equate to ``master`` for git or ``default`` for hg. However, you 198 to equate to ``master`` for git or ``default`` for hg. However, you
243 * ``SourceFactory``: as described above, this processes the sources 245 * ``SourceFactory``: as described above, this processes the sources
244 and gives a method (``checkout()``) for downloading them. 246 and gives a method (``checkout()``) for downloading them.
245 ``master.cfg`` looks in each factory and gleems its needed 247 ``master.cfg`` looks in each factory and gleems its needed
246 repository from its ``sources`` attribute (if any) in the internal 248 repository from its ``sources`` attribute (if any) in the internal
247 storage mechanism of ``SourceFactory`` and sets up schedulers 249 storage mechanism of ``SourceFactory`` and sets up schedulers
248 accordingly. 250 accordingly.
249 251
250 * ``VirtualenvFactory``: inherits from ``SourceFactory``. creates a 252 * ``VirtualenvFactory``: inherits from ``SourceFactory``. creates a
251 python virtualenv and provides build properties ``%(virtualenv)s``, 253 python virtualenv and provides build properties ``%(virtualenv)s``,
252 ``%(python)s``, and ``%(scripts)s`` (location of ``bin`` on unix or 254 ``%(python)s``, and ``%(scripts)s`` (location of ``bin`` on unix or
253 ``Scripts`` on windows with respect to the virtualenv. It also has 255 ``Scripts`` on windows with respect to the virtualenv. It also has
259 packages. In addition to checking them out in 261 packages. In addition to checking them out in
260 ``%(virtualenv)s/src``, it will also install them (using 262 ``%(virtualenv)s/src``, it will also install them (using
261 ``%(python)s setup.py install``) 263 ``%(python)s setup.py install``)
262 264
263 * ``FirefoxDownloaderFactory``: downloads and unpacks a Firefox 265 * ``FirefoxDownloaderFactory``: downloads and unpacks a Firefox
264 bundle, using 266 bundle, using
265 `GetLatestTinderbox <http://hg.mozilla.org/automation/getlatest-tinderbox>`_ . 267 `GetLatestTinderbox <http://hg.mozilla.org/automation/getlatest-tinderbox>`_ .
266 It also provides three properties: ``%(firefox_url)s``, 268 It also provides three properties: ``%(firefox_url)s``,
267 ``%(firefox_bundle)s``, and ``%(firefox)s`` -- the last, and most 269 ``%(firefox_bundle)s``, and ``%(firefox)s`` -- the last, and most
268 interesting, being the absolute path to the Firefox executable. You 270 interesting, being the absolute path to the Firefox executable. You
269 (currently) *must* pass a ``platform`` argument with 271 (currently) *must* pass a ``platform`` argument with
270 ``platform['os']`` set to "linux", "win", or "mac" in order for 272 ``platform['os']`` set to "linux", "win", or "mac" in order for
304 add a 'server' keyword to the AutologTestGroup constructor, set to 306 add a 'server' keyword to the AutologTestGroup constructor, set to
305 10.2.76.100:9200, that will cause the data to be written to 307 10.2.76.100:9200, that will cause the data to be written to
306 brasstacks test server. you can omit the restserver keyword, it will 308 brasstacks test server. you can omit the restserver keyword, it will
307 use the correct default if you skip it 309 use the correct default if you skip it
308 310
311 * email notifications
312
309 * get FirefoxDownloaderFactory working on Mac with .dmg files! 313 * get FirefoxDownloaderFactory working on Mac with .dmg files!
310 314
311 * singular checkout of repos on slaves: the slaves should have a 315 * singular checkout of repos on slaves: the slaves should have a
312 singular master repo that is checked out once for each repo URL, 316 singular master repo that is checked out once for each repo URL,
313 branch pair. It is then updated as the slaves need and (e.g.) 317 branch pair. It is then updated as the slaves need and (e.g.)
314 cloned from there. This should effectively minimize fetch time. 318 cloned from there. This should effectively minimize fetch time.
315 319
316 * use MozInfo package 320 * use MozInfo package for slave information