Mercurial > hg > autobot
annotate README.txt @ 270:c77b946e4496
minor fixes to the README
| author | Jeff Hammel <jhammel@mozilla.com> |
|---|---|
| date | Tue, 31 Jul 2012 09:49:57 -0700 |
| parents | afa37f032c39 |
| children | 22bb1ac50eb7 |
| rev | line source |
|---|---|
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
1 autobot |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
2 ======= |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
3 |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
4 buildbot for the A*Team |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
5 |
| 184 | 6 |
| 7 What is autobot? | |
| 8 ---------------- | |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
9 |
| 184 | 10 autobot is a continuous integration solution for the Automation and |
| 11 Tools Team. We have a lot of software. We're really talented, so | |
| 12 usually it doesn't break. But we're not infalliable. Our robot ally, | |
| 206 | 13 autobot, is there to test things for us. Let's meet autobot! |
| 184 | 14 |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
15 |
| 184 | 16 Installing autobot |
| 17 ------------------ | |
| 18 | |
| 206 | 19 autobot may be installed using the install script:: |
| 184 | 20 |
| 21 curl http://k0s.org/mozilla/hg/autobot/raw-file/tip/INSTALL.sh | bash | |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
22 |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
23 This will create a virtualenv and install autobot for development |
| 206 | 24 ($VIRTUAL_ENV/src/autobot). You can also use the script as |
| 25 instructions and install it in the same manner as any other python package. | |
| 184 | 26 |
| 27 | |
| 28 Setting up a buildmaster and slave | |
| 29 ---------------------------------- | |
| 30 | |
| 31 Once you have autobot installed and the virtualenv activated, you'll | |
| 32 want to create a buildmaster and a buildslave. | |
| 33 | |
| 34 You can create a master-slave pair by running ``create-autobot`` after | |
| 35 activating the virtualenv. This is mostly useful for autobot | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
36 development. The scripts ``create-autobot-master`` and |
| 206 | 37 ``create-autobot-slave`` are also available and are more useful for |
| 38 production, as normally you will not want to keep the master and the | |
| 39 slave on the same machine except for development and testing. The | |
| 40 scripts will prompt you for a factory or you can specify one or more | |
| 41 from the commandline using ``--factory`` or ``-f``. The factories are | |
| 42 from ``autobot.projects`` and its subdirectories. You can list the | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
43 available factories with ``create-autobot --list-factories`` (or |
| 206 | 44 ``create-autobot-master --list-factories``). |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
45 |
| 186 | 46 Running ``create-autobot --help`` will give the variables you can set |
| 47 when it makes a new bot for you (``create-autobot-master`` and | |
| 48 ``create-autobot-slave`` also have a ``--help``, the variables in | |
| 49 ``create-autobot`` being a superset of these). | |
| 184 | 50 |
| 186 | 51 If you don't specify a variable, the default will be used to create |
| 206 | 52 your particular bot. You can change these in the resultant |
| 53 configuration later. | |
| 186 | 54 |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
55 |
| 184 | 56 Using autobot |
| 57 ------------- | |
| 58 | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
59 The buildmaster and buildslave are started with |
| 206 | 60 ``buildbot start master`` and ``buildslave stop slave`` from the bot's |
| 61 directories. Respective ``stop`` commands also exist. If you used the | |
| 62 ``create-autobot`` command the generated bot will have a | |
| 63 ``restart_buildbot.py`` script that will stop and start both the | |
| 64 master and slave and (if debug is set) remove the log as well. | |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
65 |
| 206 | 66 The generated ``master.cfg`` file reads values from a ``master.ini`` |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
67 file in the same directory. The master.ini contains a number of |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
68 different sections: |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
69 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
70 * ``[:master:]`` contains the top level configuration for the master |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
71 as well as default settings for slaves |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
72 * sections like ``[slave:slavename]`` indicate a slave of name |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
73 ``slavename`` |
| 206 | 74 * all other sections are construction parameters for factories |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
75 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
76 The format of the ``.ini`` file is detailed in the subsequent |
| 206 | 77 section. You may change which .ini file is used by editing the |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
78 ``master.cfg`` file, or, if for whatever reason you don't want to use |
| 206 | 79 an .ini file as a driver you can construct the appropriate |
| 80 configuration therein yourself. | |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
81 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
82 An overview of the build status is detailed at the waterfall display, |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
83 by default at http://localhost:8010/waterfall . To force a build, |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
84 click on the desired builder and there will be a force build button |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
85 towards the bottom. |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
86 |
| 184 | 87 It is important to remember that continuous integration is a safety |
| 206 | 88 net, not a first line of defense. In other words, continue to check |
| 89 your code *before* sending it to autobot. | |
| 90 | |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
91 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
92 .ini file format |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
93 ---------------- |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
94 |
| 206 | 95 As stated above, a ``master.ini`` file contains three different types |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
96 of sections: |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
97 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
98 * ``[:master:]`` contains the top level configuration for the master |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
99 as well as default settings for slaves |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
100 * sections like ``[slave:slavename]`` indicate a slave of name |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
101 ``slavename`` |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
102 * other sections are construction parameters for factories |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
103 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
104 What goes in each of these? |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
105 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
106 ``:master:`` The master section contains parameters appropriate to the |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
107 buildmaster: |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
108 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
109 * slaveport: which port to listen for the slaves on |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
110 * htmlport: which port to serve the waterfall display on |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
111 * pollInterval: how often to poll the repositories, in seconds [DEFAULT: 60] |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
112 * treeStable: how long to wait before the tree settles down before |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
113 building [DEFAULT: 60] |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
114 |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
115 The other defaults may be seen by running ``create-autobot-master --help``. |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
116 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
117 Other parameters given in the ``[:master:]`` section are used as the |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
118 baseline defaults for each slave. They may be overridden in each |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
119 ``slave:`` section |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
120 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
121 ---- |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
122 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
123 ``slave:`` parameters for each slave: |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
124 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
125 * password: each slave *must* have a password. Unless there's a |
| 206 | 126 reason to have a different password per slave, its probably better |
| 127 to put this in the ``:master:`` section | |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
128 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
129 * factories: space-separated list of factories to run on that slave. |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
130 If the special value ``*`` is used, all factories will be run. You |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
131 can view the factories available with |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
132 ``create-autobot --list-factories``. The factory name corresponds to |
| 206 | 133 the directory (or module) name in ``autobot.projects``. If |
| 134 every slave is meant to run the same factories, you can put this in | |
| 135 the ``:master:`` section | |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
136 |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
137 * os: the operating system of the slave. Should be one of ``linux``, |
| 206 | 138 ``win``, or ``mac`` (though see TODO about future use of MozInfo |
| 139 making this obselescent). You probably *shouldn't* have a default | |
| 140 key for this in the ``:master:`` section (though autobot won't try | |
| 141 to stop you!). | |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
142 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
143 ---- |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
144 |
| 206 | 145 factory sections: All other sections are used to build the factories. |
| 146 Each parameter in a factory section is used as a constructor | |
| 147 (``__init__``) keyword argument when they are created in the | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
148 ``master.cfg``. So a factory section like:: |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
149 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
150 [foo] |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
151 bar = fleem |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
152 baz = another parameter |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
153 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
154 will invoke the foo factory (lets say it maps to ``MyFooFactory``) on |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
155 each slave like:: |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
156 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
157 MyFooFactory(**dict(bar='fleem', baz='another parameter')) |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
158 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
159 In addition, if a factory has a special key, ``platform``, the slave |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
160 will pass its platform information when instantiating a factory. |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
161 Currently, this is a dict with a single key, ``os``, but more may be |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
162 added as needed. As noted in the TODO below, ideally this would be |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
163 deprecated entirely by MozInfo but such is the interim solution. |
| 184 | 164 |
| 165 | |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
166 Projects |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
167 -------- |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
168 |
| 184 | 169 What does autobot test? |
| 170 | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
171 * mozbase [WORKING] |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
172 * logparser [WORKING] |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
173 * profilemanager [IN FLIGHT] |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
174 * mozmill [IN FLIGHT] |
| 70 | 175 * devicemanager [IN FLIGHT] |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
176 * firebug [TODO] |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
177 * jetpack [TODO] |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
178 * talos [TODO] |
| 70 | 179 |
| 186 | 180 The projects are obtained from any factories in subdirectories of |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
181 ``autobot.projects`` (use |
|
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
182 ``python -c 'from autobot import projects; print projects.__file__'`` |
| 186 | 183 to see this location). |
| 184 | |
| 184 | 185 |
| 208 | 186 Sources |
| 187 ------- | |
| 188 | |
| 189 ``autobot.process.factory:SourceFactory`` is an abstract base class | |
| 190 for specifying sources. Sources live as a member on the instance, named | |
| 191 (oddly) ``sources``. This is a dict with the key being the source | |
| 192 type and the value being the source to use. Source types are | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
193 currently "git" and "hg", though this is easily extensible. |
| 208 | 194 |
| 195 Internally, an individual source is stored as a 2-tuple with the first | |
| 196 item being the URL of the source and the second item being the | |
| 197 branch. If the branch is ``None`` or not specified, this is assumed | |
| 198 to equate to ``master`` for git or ``default`` for hg. However, you | |
| 199 may also use a string in the form of ``URL#branch``. In addition, you | |
| 200 may specify a whitespace separated source, which ``SourceFactory`` | |
| 201 will split into a list of sources. So all of these translate into the | |
| 202 same source:: | |
| 203 | |
| 204 sources = {'hg': 'http://hg.mozilla.org/automation/logparser'} | |
| 205 | |
| 206 sources = {'hg': ['http://hg.mozilla.org/automation/logparser#default']} | |
| 207 | |
| 208 sources = {'hg': [('http://hg.mozilla.org/automation/logparser', None)] | |
| 209 | |
| 210 The generated ``master.cfg`` polls for changes on the given source, | |
| 211 then uses ``buildbot.schedulers.filter:Changefilter`` in conjunction | |
| 209 | 212 with a ``Scheduler`` to trigger the appropriate builds. This is done |
| 213 by ``GitPoller`` and ``HgPoller`` in | |
| 214 ``autobot.changes.poller``. Again, more pollers can be added as | |
| 215 needed or the provided buildbot change sources may be utilized. | |
| 208 | 216 |
| 217 While, in general, the canonical sources should be specified at the | |
| 218 class level, if an appropriate argument (e.g. *hg* or *git*) is passed | |
| 219 in to the ``SourceFactory`` constructor, this will override that | |
| 220 source type. This is useful for testing changes on non-canonical URLs | |
| 221 and/or branches. | |
| 222 | |
| 206 | 223 |
| 184 | 224 Adding a New Project |
| 225 -------------------- | |
| 226 | |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
227 Occassionally, you'll need to add a new project to test. You can add |
| 206 | 228 a ``__init__.py`` file to a directory under ``autobot.projects`` and, |
| 229 if you have a factory therein, autobot will find it and add it to the | |
| 230 list of factories it knows about. | |
| 231 | |
| 232 There is also a script, ``create-autobot-project``, that can create | |
| 233 this stub for you:: | |
| 234 | |
| 235 Usage: create-autobot-project [options] project <output> | |
| 236 | |
| 237 ``project`` is the name of the project and ``output``, if specified, | |
| 238 is the path of the file to create. If output is not specified, it | |
| 239 will find its way to a directory named for the project under | |
| 240 ``autobot.projects``. | |
| 186 | 241 |
|
210
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
242 Several factories (``BuildFactory`` descendents) are in |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
243 ``autobot.process`` to make building a new project easier: |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
244 |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
245 * ``SourceFactory``: as described above, this processes the sources |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
246 and gives a method (``checkout()``) for downloading them. |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
247 ``master.cfg`` looks in each factory and gleems its needed |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
248 repository from its ``sources`` attribute (if any) in the internal |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
249 storage mechanism of ``SourceFactory`` and sets up schedulers |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
250 accordingly. |
|
210
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
251 |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
252 * ``VirtualenvFactory``: inherits from ``SourceFactory``. creates a |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
253 python virtualenv and provides build properties ``%(virtualenv)s``, |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
254 ``%(python)s``, and ``%(scripts)s`` (location of ``bin`` on unix or |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
255 ``Scripts`` on windows with respect to the virtualenv. It also has |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
256 a ``findScript()`` method which accepts the unix-style script name |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
257 (no extension) and returns the path to the script on the platform. |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
258 |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
259 * ``PythonSourceFactory``: inherits from ``VirtualenvFactory``. It |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
260 treats *all* (for now, can be changed) sources as sources of python |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
261 packages. In addition to checking them out in |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
262 ``%(virtualenv)s/src``, it will also install them (using |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
263 ``%(python)s setup.py install``) |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
264 |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
265 * ``FirefoxDownloaderFactory``: downloads and unpacks a Firefox |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
266 bundle, using |
|
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
267 `GetLatestTinderbox <http://hg.mozilla.org/automation/getlatest-tinderbox>`_ . |
|
210
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
268 It also provides three properties: ``%(firefox_url)s``, |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
269 ``%(firefox_bundle)s``, and ``%(firefox)s`` -- the last, and most |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
270 interesting, being the absolute path to the Firefox executable. You |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
271 (currently) *must* pass a ``platform`` argument with |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
272 ``platform['os']`` set to "linux", "win", or "mac" in order for |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
273 ``FirefoxDownloaderFactory`` to figure out what to do with the thing |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
274 that it downloads (though see TODO, in the future it should/could |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
275 use MozInfo or guess the OS from the bundle extension...maybe). |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
276 Currently, mac is *NOT* supported, though its on my list of things |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
277 to do (I'd love advice/help!). |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
278 |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
279 See the ``autobot.process.factory`` file for more details. These |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
280 classes are intended as mix-ins (not my favorite pattern, but it seems |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
281 to basically be what buildbot wants you to do). More work needs to be |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
282 done to provide for the breadth of use-cases, but what exists now is |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
283 considered a "good start" versus a "final form". More factories may |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
284 be added per necessity. |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
285 |
| 186 | 286 |
| 287 Is your autobot being feisty? | |
| 288 ----------------------------- | |
| 289 | |
| 290 Let me know! I'd like to make autobot a solution that works for all | |
| 291 stake-holders, and if you're reading this, that means you! | |
| 292 | |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
293 jhammel@mozilla.com |
| 186 | 294 |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
295 |
| 186 | 296 TODO |
| 297 ---- | |
| 298 | |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
299 No software of any size is ever finished. Here are a few things I |
|
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
300 would like to add: |
| 186 | 301 |
|
266
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
302 * post to autolog: |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
303 http://hg.mozilla.org/users/jgriffin_mozilla.com/mozautolog/ |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
304 Example code can be found at |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
305 http://hg.mozilla.org/services/services-central/file/9f29daaecbcc/testing/tps/tps/testrunner.py#l491 |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
306 add a 'server' keyword to the AutologTestGroup constructor, set to |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
307 10.2.76.100:9200, that will cause the data to be written to |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
308 brasstacks test server. you can omit the restserver keyword, it will |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
309 use the correct default if you skip it |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
310 |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
311 * email notifications |
|
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
312 |
|
210
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
313 * get FirefoxDownloaderFactory working on Mac with .dmg files! |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
314 |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
315 * singular checkout of repos on slaves: the slaves should have a |
|
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
316 singular master repo that is checked out once for each repo URL, |
|
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
317 branch pair. It is then updated as the slaves need and (e.g.) |
|
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
318 cloned from there. This should effectively minimize fetch time. |
|
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
319 |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
320 * use MozInfo package for slave information |