Mercurial > hg > autobot
annotate README.txt @ 277:268349e6c941
remove devicemanager project; this now lives in mozbase
| author | Jeff Hammel <jhammel@mozilla.com> |
|---|---|
| date | Tue, 31 Jul 2012 14:45:04 -0700 |
| parents | 22bb1ac50eb7 |
| children | b6d0ce3cf430 |
| rev | line source |
|---|---|
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
1 autobot |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
2 ======= |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
3 |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
4 buildbot for the A*Team |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
5 |
| 184 | 6 |
| 7 What is autobot? | |
| 8 ---------------- | |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
9 |
| 184 | 10 autobot is a continuous integration solution for the Automation and |
| 11 Tools Team. We have a lot of software. We're really talented, so | |
| 12 usually it doesn't break. But we're not infalliable. Our robot ally, | |
| 206 | 13 autobot, is there to test things for us. Let's meet autobot! |
| 184 | 14 |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
15 |
| 184 | 16 Installing autobot |
| 17 ------------------ | |
| 18 | |
| 206 | 19 autobot may be installed using the install script:: |
| 184 | 20 |
| 21 curl http://k0s.org/mozilla/hg/autobot/raw-file/tip/INSTALL.sh | bash | |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
22 |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
23 This will create a virtualenv and install autobot for development |
| 206 | 24 ($VIRTUAL_ENV/src/autobot). You can also use the script as |
| 25 instructions and install it in the same manner as any other python package. | |
| 184 | 26 |
| 27 | |
| 28 Setting up a buildmaster and slave | |
| 29 ---------------------------------- | |
| 30 | |
| 31 Once you have autobot installed and the virtualenv activated, you'll | |
| 32 want to create a buildmaster and a buildslave. | |
| 33 | |
| 34 You can create a master-slave pair by running ``create-autobot`` after | |
| 35 activating the virtualenv. This is mostly useful for autobot | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
36 development. The scripts ``create-autobot-master`` and |
| 206 | 37 ``create-autobot-slave`` are also available and are more useful for |
| 38 production, as normally you will not want to keep the master and the | |
| 39 slave on the same machine except for development and testing. The | |
| 40 scripts will prompt you for a factory or you can specify one or more | |
| 41 from the commandline using ``--factory`` or ``-f``. The factories are | |
| 42 from ``autobot.projects`` and its subdirectories. You can list the | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
43 available factories with ``create-autobot --list-factories`` (or |
| 206 | 44 ``create-autobot-master --list-factories``). |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
45 |
| 186 | 46 Running ``create-autobot --help`` will give the variables you can set |
| 47 when it makes a new bot for you (``create-autobot-master`` and | |
| 48 ``create-autobot-slave`` also have a ``--help``, the variables in | |
| 49 ``create-autobot`` being a superset of these). | |
| 184 | 50 |
| 186 | 51 If you don't specify a variable, the default will be used to create |
| 206 | 52 your particular bot. You can change these in the resultant |
| 53 configuration later. | |
| 186 | 54 |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
55 |
| 184 | 56 Using autobot |
| 57 ------------- | |
| 58 | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
59 The buildmaster and buildslave are started with |
| 206 | 60 ``buildbot start master`` and ``buildslave stop slave`` from the bot's |
| 61 directories. Respective ``stop`` commands also exist. If you used the | |
| 62 ``create-autobot`` command the generated bot will have a | |
| 63 ``restart_buildbot.py`` script that will stop and start both the | |
| 64 master and slave and (if debug is set) remove the log as well. | |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
65 |
| 206 | 66 The generated ``master.cfg`` file reads values from a ``master.ini`` |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
67 file in the same directory. The master.ini contains a number of |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
68 different sections: |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
69 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
70 * ``[:master:]`` contains the top level configuration for the master |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
71 as well as default settings for slaves |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
72 * sections like ``[slave:slavename]`` indicate a slave of name |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
73 ``slavename`` |
| 206 | 74 * all other sections are construction parameters for factories |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
75 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
76 The format of the ``.ini`` file is detailed in the subsequent |
| 206 | 77 section. You may change which .ini file is used by editing the |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
78 ``master.cfg`` file, or, if for whatever reason you don't want to use |
| 206 | 79 an .ini file as a driver you can construct the appropriate |
| 80 configuration therein yourself. | |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
81 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
82 An overview of the build status is detailed at the waterfall display, |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
83 by default at http://localhost:8010/waterfall . To force a build, |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
84 click on the desired builder and there will be a force build button |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
85 towards the bottom. |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
86 |
| 184 | 87 It is important to remember that continuous integration is a safety |
| 206 | 88 net, not a first line of defense. In other words, continue to check |
| 89 your code *before* sending it to autobot. | |
| 90 | |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
91 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
92 .ini file format |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
93 ---------------- |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
94 |
| 206 | 95 As stated above, a ``master.ini`` file contains three different types |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
96 of sections: |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
97 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
98 * ``[:master:]`` contains the top level configuration for the master |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
99 as well as default settings for slaves |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
100 * sections like ``[slave:slavename]`` indicate a slave of name |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
101 ``slavename`` |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
102 * other sections are construction parameters for factories |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
103 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
104 What goes in each of these? |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
105 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
106 ``:master:`` The master section contains parameters appropriate to the |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
107 buildmaster: |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
108 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
109 * slaveport: which port to listen for the slaves on |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
110 * htmlport: which port to serve the waterfall display on |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
111 * pollInterval: how often to poll the repositories, in seconds [DEFAULT: 60] |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
112 * treeStable: how long to wait before the tree settles down before |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
113 building [DEFAULT: 60] |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
114 |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
115 The other defaults may be seen by running ``create-autobot-master --help``. |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
116 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
117 Other parameters given in the ``[:master:]`` section are used as the |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
118 baseline defaults for each slave. They may be overridden in each |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
119 ``slave:`` section |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
120 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
121 ---- |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
122 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
123 ``slave:`` parameters for each slave: |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
124 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
125 * password: each slave *must* have a password. Unless there's a |
| 206 | 126 reason to have a different password per slave, its probably better |
| 127 to put this in the ``:master:`` section | |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
128 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
129 * factories: space-separated list of factories to run on that slave. |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
130 If the special value ``*`` is used, all factories will be run. You |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
131 can view the factories available with |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
132 ``create-autobot --list-factories``. The factory name corresponds to |
| 206 | 133 the directory (or module) name in ``autobot.projects``. If |
| 134 every slave is meant to run the same factories, you can put this in | |
| 135 the ``:master:`` section | |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
136 |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
137 * os: the operating system of the slave. Should be one of ``linux``, |
| 206 | 138 ``win``, or ``mac`` (though see TODO about future use of MozInfo |
| 139 making this obselescent). You probably *shouldn't* have a default | |
| 140 key for this in the ``:master:`` section (though autobot won't try | |
| 141 to stop you!). | |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
142 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
143 ---- |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
144 |
| 206 | 145 factory sections: All other sections are used to build the factories. |
| 146 Each parameter in a factory section is used as a constructor | |
| 147 (``__init__``) keyword argument when they are created in the | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
148 ``master.cfg``. So a factory section like:: |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
149 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
150 [foo] |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
151 bar = fleem |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
152 baz = another parameter |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
153 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
154 will invoke the foo factory (lets say it maps to ``MyFooFactory``) on |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
155 each slave like:: |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
156 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
157 MyFooFactory(**dict(bar='fleem', baz='another parameter')) |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
158 |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
159 In addition, if a factory has a special key, ``platform``, the slave |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
160 will pass its platform information when instantiating a factory. |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
161 Currently, this is a dict with a single key, ``os``, but more may be |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
162 added as needed. As noted in the TODO below, ideally this would be |
|
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
163 deprecated entirely by MozInfo but such is the interim solution. |
| 184 | 164 |
| 165 | |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
166 Projects |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
167 -------- |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
168 |
| 184 | 169 What does autobot test? |
| 170 | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
171 * mozbase [WORKING] |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
172 * logparser [WORKING] |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
173 * profilemanager [IN FLIGHT] |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
174 * mozmill [IN FLIGHT] |
|
277
268349e6c941
remove devicemanager project; this now lives in mozbase
Jeff Hammel <jhammel@mozilla.com>
parents:
275
diff
changeset
|
175 * talos [IN FLIGHT] |
|
59
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
176 * firebug [TODO] |
|
e66165f2f31b
complete automagic imports
Jeff Hammel <jhammel@mozilla.com>
parents:
30
diff
changeset
|
177 * jetpack [TODO] |
| 70 | 178 |
| 186 | 179 The projects are obtained from any factories in subdirectories of |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
180 ``autobot.projects`` (use |
|
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
181 ``python -c 'from autobot import projects; print projects.__file__'`` |
| 186 | 182 to see this location). |
| 183 | |
| 184 | 184 |
| 208 | 185 Sources |
| 186 ------- | |
| 187 | |
| 188 ``autobot.process.factory:SourceFactory`` is an abstract base class | |
| 189 for specifying sources. Sources live as a member on the instance, named | |
| 190 (oddly) ``sources``. This is a dict with the key being the source | |
| 191 type and the value being the source to use. Source types are | |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
192 currently "git" and "hg", though this is easily extensible. |
| 208 | 193 |
| 194 Internally, an individual source is stored as a 2-tuple with the first | |
| 195 item being the URL of the source and the second item being the | |
| 196 branch. If the branch is ``None`` or not specified, this is assumed | |
| 197 to equate to ``master`` for git or ``default`` for hg. However, you | |
| 198 may also use a string in the form of ``URL#branch``. In addition, you | |
| 199 may specify a whitespace separated source, which ``SourceFactory`` | |
| 200 will split into a list of sources. So all of these translate into the | |
| 201 same source:: | |
| 202 | |
| 203 sources = {'hg': 'http://hg.mozilla.org/automation/logparser'} | |
| 204 | |
| 205 sources = {'hg': ['http://hg.mozilla.org/automation/logparser#default']} | |
| 206 | |
| 207 sources = {'hg': [('http://hg.mozilla.org/automation/logparser', None)] | |
| 208 | |
| 209 The generated ``master.cfg`` polls for changes on the given source, | |
| 210 then uses ``buildbot.schedulers.filter:Changefilter`` in conjunction | |
| 209 | 211 with a ``Scheduler`` to trigger the appropriate builds. This is done |
| 212 by ``GitPoller`` and ``HgPoller`` in | |
| 213 ``autobot.changes.poller``. Again, more pollers can be added as | |
| 214 needed or the provided buildbot change sources may be utilized. | |
| 208 | 215 |
| 216 While, in general, the canonical sources should be specified at the | |
| 217 class level, if an appropriate argument (e.g. *hg* or *git*) is passed | |
| 218 in to the ``SourceFactory`` constructor, this will override that | |
| 219 source type. This is useful for testing changes on non-canonical URLs | |
| 220 and/or branches. | |
| 221 | |
| 206 | 222 |
| 184 | 223 Adding a New Project |
| 224 -------------------- | |
| 225 | |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
226 Occassionally, you'll need to add a new project to test. You can add |
| 206 | 227 a ``__init__.py`` file to a directory under ``autobot.projects`` and, |
| 228 if you have a factory therein, autobot will find it and add it to the | |
| 229 list of factories it knows about. | |
| 230 | |
| 231 There is also a script, ``create-autobot-project``, that can create | |
| 232 this stub for you:: | |
| 233 | |
| 234 Usage: create-autobot-project [options] project <output> | |
| 235 | |
| 236 ``project`` is the name of the project and ``output``, if specified, | |
| 237 is the path of the file to create. If output is not specified, it | |
| 238 will find its way to a directory named for the project under | |
| 239 ``autobot.projects``. | |
| 186 | 240 |
|
210
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
241 Several factories (``BuildFactory`` descendents) are in |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
242 ``autobot.process`` to make building a new project easier: |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
243 |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
244 * ``SourceFactory``: as described above, this processes the sources |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
245 and gives a method (``checkout()``) for downloading them. |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
246 ``master.cfg`` looks in each factory and gleems its needed |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
247 repository from its ``sources`` attribute (if any) in the internal |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
248 storage mechanism of ``SourceFactory`` and sets up schedulers |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
249 accordingly. |
|
210
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
250 |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
251 * ``VirtualenvFactory``: inherits from ``SourceFactory``. creates a |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
252 python virtualenv and provides build properties ``%(virtualenv)s``, |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
253 ``%(python)s``, and ``%(scripts)s`` (location of ``bin`` on unix or |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
254 ``Scripts`` on windows with respect to the virtualenv. It also has |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
255 a ``findScript()`` method which accepts the unix-style script name |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
256 (no extension) and returns the path to the script on the platform. |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
257 |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
258 * ``PythonSourceFactory``: inherits from ``VirtualenvFactory``. It |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
259 treats *all* (for now, can be changed) sources as sources of python |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
260 packages. In addition to checking them out in |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
261 ``%(virtualenv)s/src``, it will also install them (using |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
262 ``%(python)s setup.py install``) |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
263 |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
264 * ``FirefoxDownloaderFactory``: downloads and unpacks a Firefox |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
265 bundle, using |
|
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
266 `GetLatestTinderbox <http://hg.mozilla.org/automation/getlatest-tinderbox>`_ . |
|
210
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
267 It also provides three properties: ``%(firefox_url)s``, |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
268 ``%(firefox_bundle)s``, and ``%(firefox)s`` -- the last, and most |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
269 interesting, being the absolute path to the Firefox executable. You |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
270 (currently) *must* pass a ``platform`` argument with |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
271 ``platform['os']`` set to "linux", "win", or "mac" in order for |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
272 ``FirefoxDownloaderFactory`` to figure out what to do with the thing |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
273 that it downloads (though see TODO, in the future it should/could |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
274 use MozInfo or guess the OS from the bundle extension...maybe). |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
275 Currently, mac is *NOT* supported, though its on my list of things |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
276 to do (I'd love advice/help!). |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
277 |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
278 See the ``autobot.process.factory`` file for more details. These |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
279 classes are intended as mix-ins (not my favorite pattern, but it seems |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
280 to basically be what buildbot wants you to do). More work needs to be |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
281 done to provide for the breadth of use-cases, but what exists now is |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
282 considered a "good start" versus a "final form". More factories may |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
283 be added per necessity. |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
284 |
| 186 | 285 |
| 286 Is your autobot being feisty? | |
| 287 ----------------------------- | |
| 288 | |
| 289 Let me know! I'd like to make autobot a solution that works for all | |
| 290 stake-holders, and if you're reading this, that means you! | |
| 291 | |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
292 jhammel@mozilla.com |
| 186 | 293 |
|
204
5f188d2a36aa
write how the .ini files work
Jeff Hammel <jhammel@mozilla.com>
parents:
187
diff
changeset
|
294 |
| 186 | 295 TODO |
| 296 ---- | |
| 297 | |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
298 No software of any size is ever finished. Here are a few things I |
|
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
299 would like to add: |
| 186 | 300 |
|
266
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
301 * post to autolog: |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
302 http://hg.mozilla.org/users/jgriffin_mozilla.com/mozautolog/ |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
303 Example code can be found at |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
304 http://hg.mozilla.org/services/services-central/file/9f29daaecbcc/testing/tps/tps/testrunner.py#l491 |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
305 add a 'server' keyword to the AutologTestGroup constructor, set to |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
306 10.2.76.100:9200, that will cause the data to be written to |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
307 brasstacks test server. you can omit the restserver keyword, it will |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
308 use the correct default if you skip it |
|
ef7d67448344
update readme to note autolog effort
Jeff Hammel <jhammel@mozilla.com>
parents:
210
diff
changeset
|
309 |
|
270
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
310 * email notifications |
|
c77b946e4496
minor fixes to the README
Jeff Hammel <jhammel@mozilla.com>
parents:
267
diff
changeset
|
311 |
|
210
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
312 * get FirefoxDownloaderFactory working on Mac with .dmg files! |
|
6606238e2b07
finish first-draft documentation, i think
Jeff Hammel <jhammel@mozilla.com>
parents:
209
diff
changeset
|
313 |
|
187
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
314 * singular checkout of repos on slaves: the slaves should have a |
|
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
315 singular master repo that is checked out once for each repo URL, |
|
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
316 branch pair. It is then updated as the slaves need and (e.g.) |
|
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
317 cloned from there. This should effectively minimize fetch time. |
|
bba97450cbc2
* update example and templates
Jeff Hammel <jhammel@mozilla.com>
parents:
186
diff
changeset
|
318 |