Mercurial > hg > simpypi
annotate README.txt @ 80:0f3603a3d37d
note need to accept setup.py upload
| author | Jeff Hammel <jhammel@mozilla.com> |
|---|---|
| date | Sun, 04 Mar 2012 10:45:33 -0800 |
| parents | f42df1456565 |
| children | c61d0fe16509 |
| rev | line source |
|---|---|
| 0 | 1 simpypi |
|
33
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
2 ======= |
| 0 | 3 |
| 75 | 4 Simple python package index |
| 5 | |
| 6 How simpypi works | |
| 7 ----------------- | |
| 8 | |
| 76 | 9 The heart of simpypi is ``simpypi.wsgi.SimPyPI``, a simple |
| 10 `WSGI <http://www.python.org/dev/peps/pep-0333/>`_ web app that | |
| 11 accepts a uploaded | |
| 12 `python package <http://guide.python-distribute.org/introduction.html>`_ | |
| 13 distribution and places it according to its name and version metadata | |
| 14 appropriate to the | |
| 15 `simple index protocol <http://guide.python-distribute.org/contributing.html#the-simple-index-protocol>`_ . | |
| 16 | |
| 17 For security, ``SimPyPI`` returns straight | |
| 18 `HTTP 400 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html>`_ s | |
| 77 | 19 for invalid requests. This could probably be improved. |
| 76 | 20 |
| 21 ``SimPyPI`` itself does not serve this directory. The directory | |
| 22 should be served by a fileserver that will generate directory indices | |
| 23 (as apprpropriate to | |
| 24 http://guide.python-distribute.org/contributing.html#the-simple-index-protocol | |
| 25 ) such as Apache or | |
| 26 `FileServer <http://pypi.python.org/pypi/FileServer>`_ . | |
| 77 | 27 ``simpypi.factory.factory`` does provide a factory to make a WSGI app |
| 28 that wraps ``FileServer`` in middleware and serves the simple index | |
| 29 under ``/index/`` and the ``SimPyPI`` app at ``/``. Additionally, a | |
| 30 ``simpypi`` command line program is provided that front-ends this. | |
| 76 | 31 |
| 75 | 32 Currently ``simpypi`` only works on source distributions (that is, |
| 33 packages made with ``python setup.py sdist``). | |
| 0 | 34 |
|
79
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
35 Example |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
36 ------- |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
37 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
38 I installed an instance of ``simpypi`` at http://k0s.org:8080 for |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
39 demonstration purposes with a package index at http://k0s.org:8080/index/ . |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
40 For testing purposes, I made a script, |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
41 `upload_mobase.py <http://k0s.org/mozilla/hg/simpypi/file/tip/tests/upload_mozbase.py>`_ |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
42 that uploads the |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
43 `mozbase <https://github.com/mozilla/mozbase>`_ packages to |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
44 ``simpypi``. So the |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
45 `index <http://k0s.org:8080/index/>`_ is now populated with them. |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
46 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
47 You can upload packages with |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
48 `curl <http://www.cs.sunysb.edu/documentation/curl/>`_ :: |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
49 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
50 > wget http://pypi.python.org/packages/source/P/PyYAML/PyYAML-3.10.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
51 > curl -F 'package=@PyYAML-3.10.tar.gz' http://k0s.org:8080/ |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
52 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
53 You can ``easy_install`` mozbase from http://k0s.org:8080/index/ . |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
54 If a package's dependencies can be found from the ``simpypi`` package |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
55 index, they will also be installed from the index:: |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
56 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
57 > virtualenv.py tmp |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
58 New python executable in tmp/bin/python |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
59 Installing setuptools............done. |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
60 Installing pip...............done. |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
61 > cd tmp/ |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
62 (tmp)│easy_install -i http://k0s.org:8080/index/ mozrunner |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
63 Searching for mozrunner |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
64 Reading http://k0s.org:8080/index/mozrunner/ |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
65 Best match: mozrunner 5.1 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
66 Downloading http://k0s.org:8080/index/mozrunner/mozrunner-5.1.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
67 Processing mozrunner-5.1.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
68 Running mozrunner-5.1/setup.py -q bdist_egg --dist-dir |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
69 /tmp/easy_install-gqerOV/mozrunner-5.1/egg-dist-tmp-Qyx3Cr |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
70 Adding mozrunner 5.1 to easy-install.pth file |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
71 Installing mozrunner script to /home/jhammel/tmp/bin |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
72 Installed |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
73 /home/jhammel/tmp/lib/python2.7/site-packages/mozrunner-5.1-py2.7.egg |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
74 Processing dependencies for mozrunner |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
75 Searching for mozprofile>=0.1 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
76 Reading http://k0s.org:8080/index/mozprofile/ |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
77 Best match: mozprofile 0.1 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
78 Downloading http://k0s.org:8080/index/mozprofile/mozprofile-0.1.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
79 Processing mozprofile-0.1.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
80 Running mozprofile-0.1/setup.py -q bdist_egg --dist-dir |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
81 /tmp/easy_install-4Im6x0/mozprofile-0.1/egg-dist-tmp-9Jp5TR |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
82 Adding mozprofile 0.1 to easy-install.pth file |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
83 Installing mozprofile script to /home/jhammel/tmp/bin |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
84 Installed |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
85 /home/jhammel/tmp/lib/python2.7/site-packages/mozprofile-0.1-py2.7.egg |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
86 Searching for mozprocess |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
87 Reading http://k0s.org:8080/index/mozprocess/ |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
88 Best match: mozprocess 0.1b2 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
89 Downloading |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
90 http://k0s.org:8080/index/mozprocess/mozprocess-0.1b2.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
91 Processing mozprocess-0.1b2.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
92 Running mozprocess-0.1b2/setup.py -q bdist_egg --dist-dir |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
93 /tmp/easy_install-KU6AiF/mozprocess-0.1b2/egg-dist-tmp-4j5CMP |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
94 Adding mozprocess 0.1b2 to easy-install.pth file |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
95 Installed |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
96 /home/jhammel/tmp/lib/python2.7/site-packages/mozprocess-0.1b2-py2.7.egg |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
97 Searching for mozinfo |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
98 Reading http://k0s.org:8080/index/mozinfo/ |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
99 Best match: mozinfo 0.3.3 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
100 Downloading http://k0s.org:8080/index/mozinfo/mozinfo-0.3.3.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
101 Processing mozinfo-0.3.3.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
102 Running mozinfo-0.3.3/setup.py -q bdist_egg --dist-dir |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
103 /tmp/easy_install-JaKeaz/mozinfo-0.3.3/egg-dist-tmp-xWojez |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
104 Adding mozinfo 0.3.3 to easy-install.pth file |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
105 Installing mozinfo script to /home/jhammel/tmp/bin |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
106 Installed |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
107 /home/jhammel/tmp/lib/python2.7/site-packages/mozinfo-0.3.3-py2.7.egg |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
108 Searching for ManifestDestiny>=0.5.4 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
109 Reading http://k0s.org:8080/index/ManifestDestiny/ |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
110 Best match: ManifestDestiny 0.5.4 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
111 Downloading |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
112 http://k0s.org:8080/index/ManifestDestiny/ManifestDestiny-0.5.4.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
113 Processing ManifestDestiny-0.5.4.tar.gz |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
114 Running ManifestDestiny-0.5.4/setup.py -q bdist_egg --dist-dir |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
115 /tmp/easy_install-2blF3S/ManifestDestiny-0.5.4/egg-dist-tmp-R3KZde |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
116 Adding ManifestDestiny 0.5.4 to easy-install.pth file |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
117 Installing manifestparser script to /home/jhammel/tmp/bin |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
118 Installed |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
119 /home/jhammel/tmp/lib/python2.7/site-packages/ManifestDestiny-0.5.4-py2.7.egg |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
120 Finished processing dependencies for mozrunner |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
121 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
122 Note that all of the packages come from the k0s.org installation and |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
123 not from http://pypi.python.org/ . |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
124 |
|
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
125 |
|
33
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
126 Running the Tests |
|
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
127 ----------------- |
|
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
128 |
| 78 | 129 The |
| 130 `tests directory <http://k0s.org/mozilla/hg/simpypi/file/tip/tests>`_ | |
| 131 contains | |
| 132 `doctests <http://docs.python.org/library/doctest.html>`_ and | |
| 133 the test-runner, | |
| 134 `test.py <http://k0s.org/mozilla/hg/simpypi/file/tip/tests/test.py>`_ . | |
| 135 These tests illustrate basic functionality and protect from | |
| 136 regressions if they are run before code is committed. | |
|
33
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
137 ``tests-require.txt`` contains dependencies that should be installed |
|
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
138 to run the tests. |
| 77 | 139 `Paste <http://pythonpaste.org/>`_ |
| 140 `TestApp <http://pythonpaste.org/testing-applications.html>`_ , | |
| 141 though this could be transitioned to | |
| 142 `WebTest <http://webtest.pythonpaste.org/en/latest/index.html>`_ . | |
| 143 `virtualenv <http://www.virtualenv.org/>`_ | |
| 144 is used for isolating python environments. | |
|
33
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
145 |
| 78 | 146 To run the tests, do:: |
| 147 | |
| 148 python test.py | |
| 149 | |
| 39 | 150 TODO |
| 151 ---- | |
| 152 | |
| 75 | 153 While simpypi is pretty simple, just because it is only 100 lines of |
| 154 code doesn't mean that it is the *right* 100 lines of code. The | |
| 155 following issues could be addressed: | |
| 156 | |
| 157 * the temporary package should be dealt with entirely in memory, | |
| 158 ideally. Currently we write to a file and move it. | |
| 159 | |
| 160 * ``simpypi`` use ``pkginfo.sdist`` to read the data from a source | |
| 161 distribution. Instead, the uploaded package should probably be | |
| 162 unpacked and ``python setup.py sdist`` run and the resulting | |
| 163 package put in the appropriate place. This will allow archives | |
| 164 without ``PKG-INFO`` | |
| 165 (e.g. http://hg.mozilla.org/build/talos/archive/tip.tar.gz ) | |
| 166 to be uploaded as well as at least partially correct for the fact | |
| 167 that currently ``simpypi`` only works for uploaded source | |
| 168 distributions. | |
| 39 | 169 |
|
80
0f3603a3d37d
note need to accept setup.py upload
Jeff Hammel <jhammel@mozilla.com>
parents:
79
diff
changeset
|
170 * ``simpypi`` does not accept packages upload in the usual python |
|
0f3603a3d37d
note need to accept setup.py upload
Jeff Hammel <jhammel@mozilla.com>
parents:
79
diff
changeset
|
171 way, e.g. ``python setup.py sdist upload -r http://url.of/simpypi`` . |
|
0f3603a3d37d
note need to accept setup.py upload
Jeff Hammel <jhammel@mozilla.com>
parents:
79
diff
changeset
|
172 It should. |
|
0f3603a3d37d
note need to accept setup.py upload
Jeff Hammel <jhammel@mozilla.com>
parents:
79
diff
changeset
|
173 |
| 0 | 174 -- |
| 175 | |
|
33
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
176 http://k0s.org/mozilla/hg/simpypi |