Mercurial > hg > simpypi
annotate README.txt @ 82:f974c6d79804 default tip
lost and found
author | Jeff Hammel <k0scist@gmail.com> |
---|---|
date | Tue, 19 May 2015 21:02:59 -0700 |
parents | c61d0fe16509 |
children |
rev | line source |
---|---|
0 | 1 simpypi |
33
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
2 ======= |
0 | 3 |
75 | 4 Simple python package index |
5 | |
6 How simpypi works | |
7 ----------------- | |
8 | |
76 | 9 The heart of simpypi is ``simpypi.wsgi.SimPyPI``, a simple |
10 `WSGI <http://www.python.org/dev/peps/pep-0333/>`_ web app that | |
11 accepts a uploaded | |
12 `python package <http://guide.python-distribute.org/introduction.html>`_ | |
13 distribution and places it according to its name and version metadata | |
14 appropriate to the | |
15 `simple index protocol <http://guide.python-distribute.org/contributing.html#the-simple-index-protocol>`_ . | |
16 | |
17 For security, ``SimPyPI`` returns straight | |
18 `HTTP 400 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html>`_ s | |
77 | 19 for invalid requests. This could probably be improved. |
76 | 20 |
21 ``SimPyPI`` itself does not serve this directory. The directory | |
22 should be served by a fileserver that will generate directory indices | |
23 (as apprpropriate to | |
24 http://guide.python-distribute.org/contributing.html#the-simple-index-protocol | |
25 ) such as Apache or | |
26 `FileServer <http://pypi.python.org/pypi/FileServer>`_ . | |
77 | 27 ``simpypi.factory.factory`` does provide a factory to make a WSGI app |
28 that wraps ``FileServer`` in middleware and serves the simple index | |
29 under ``/index/`` and the ``SimPyPI`` app at ``/``. Additionally, a | |
30 ``simpypi`` command line program is provided that front-ends this. | |
76 | 31 |
75 | 32 Currently ``simpypi`` only works on source distributions (that is, |
33 packages made with ``python setup.py sdist``). | |
0 | 34 |
79
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
35 Example |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
36 ------- |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
37 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
38 I installed an instance of ``simpypi`` at http://k0s.org:8080 for |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
39 demonstration purposes with a package index at http://k0s.org:8080/index/ . |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
40 For testing purposes, I made a script, |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
41 `upload_mobase.py <http://k0s.org/mozilla/hg/simpypi/file/tip/tests/upload_mozbase.py>`_ |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
42 that uploads the |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
43 `mozbase <https://github.com/mozilla/mozbase>`_ packages to |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
44 ``simpypi``. So the |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
45 `index <http://k0s.org:8080/index/>`_ is now populated with them. |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
46 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
47 You can upload packages with |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
48 `curl <http://www.cs.sunysb.edu/documentation/curl/>`_ :: |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
49 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
50 > wget http://pypi.python.org/packages/source/P/PyYAML/PyYAML-3.10.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
51 > curl -F 'package=@PyYAML-3.10.tar.gz' http://k0s.org:8080/ |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
52 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
53 You can ``easy_install`` mozbase from http://k0s.org:8080/index/ . |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
54 If a package's dependencies can be found from the ``simpypi`` package |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
55 index, they will also be installed from the index:: |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
56 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
57 > virtualenv.py tmp |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
58 New python executable in tmp/bin/python |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
59 Installing setuptools............done. |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
60 Installing pip...............done. |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
61 > cd tmp/ |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
62 (tmp)│easy_install -i http://k0s.org:8080/index/ mozrunner |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
63 Searching for mozrunner |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
64 Reading http://k0s.org:8080/index/mozrunner/ |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
65 Best match: mozrunner 5.1 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
66 Downloading http://k0s.org:8080/index/mozrunner/mozrunner-5.1.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
67 Processing mozrunner-5.1.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
68 Running mozrunner-5.1/setup.py -q bdist_egg --dist-dir |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
69 /tmp/easy_install-gqerOV/mozrunner-5.1/egg-dist-tmp-Qyx3Cr |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
70 Adding mozrunner 5.1 to easy-install.pth file |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
71 Installing mozrunner script to /home/jhammel/tmp/bin |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
72 Installed |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
73 /home/jhammel/tmp/lib/python2.7/site-packages/mozrunner-5.1-py2.7.egg |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
74 Processing dependencies for mozrunner |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
75 Searching for mozprofile>=0.1 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
76 Reading http://k0s.org:8080/index/mozprofile/ |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
77 Best match: mozprofile 0.1 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
78 Downloading http://k0s.org:8080/index/mozprofile/mozprofile-0.1.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
79 Processing mozprofile-0.1.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
80 Running mozprofile-0.1/setup.py -q bdist_egg --dist-dir |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
81 /tmp/easy_install-4Im6x0/mozprofile-0.1/egg-dist-tmp-9Jp5TR |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
82 Adding mozprofile 0.1 to easy-install.pth file |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
83 Installing mozprofile script to /home/jhammel/tmp/bin |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
84 Installed |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
85 /home/jhammel/tmp/lib/python2.7/site-packages/mozprofile-0.1-py2.7.egg |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
86 Searching for mozprocess |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
87 Reading http://k0s.org:8080/index/mozprocess/ |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
88 Best match: mozprocess 0.1b2 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
89 Downloading |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
90 http://k0s.org:8080/index/mozprocess/mozprocess-0.1b2.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
91 Processing mozprocess-0.1b2.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
92 Running mozprocess-0.1b2/setup.py -q bdist_egg --dist-dir |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
93 /tmp/easy_install-KU6AiF/mozprocess-0.1b2/egg-dist-tmp-4j5CMP |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
94 Adding mozprocess 0.1b2 to easy-install.pth file |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
95 Installed |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
96 /home/jhammel/tmp/lib/python2.7/site-packages/mozprocess-0.1b2-py2.7.egg |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
97 Searching for mozinfo |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
98 Reading http://k0s.org:8080/index/mozinfo/ |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
99 Best match: mozinfo 0.3.3 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
100 Downloading http://k0s.org:8080/index/mozinfo/mozinfo-0.3.3.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
101 Processing mozinfo-0.3.3.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
102 Running mozinfo-0.3.3/setup.py -q bdist_egg --dist-dir |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
103 /tmp/easy_install-JaKeaz/mozinfo-0.3.3/egg-dist-tmp-xWojez |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
104 Adding mozinfo 0.3.3 to easy-install.pth file |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
105 Installing mozinfo script to /home/jhammel/tmp/bin |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
106 Installed |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
107 /home/jhammel/tmp/lib/python2.7/site-packages/mozinfo-0.3.3-py2.7.egg |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
108 Searching for ManifestDestiny>=0.5.4 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
109 Reading http://k0s.org:8080/index/ManifestDestiny/ |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
110 Best match: ManifestDestiny 0.5.4 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
111 Downloading |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
112 http://k0s.org:8080/index/ManifestDestiny/ManifestDestiny-0.5.4.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
113 Processing ManifestDestiny-0.5.4.tar.gz |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
114 Running ManifestDestiny-0.5.4/setup.py -q bdist_egg --dist-dir |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
115 /tmp/easy_install-2blF3S/ManifestDestiny-0.5.4/egg-dist-tmp-R3KZde |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
116 Adding ManifestDestiny 0.5.4 to easy-install.pth file |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
117 Installing manifestparser script to /home/jhammel/tmp/bin |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
118 Installed |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
119 /home/jhammel/tmp/lib/python2.7/site-packages/ManifestDestiny-0.5.4-py2.7.egg |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
120 Finished processing dependencies for mozrunner |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
121 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
122 Note that all of the packages come from the k0s.org installation and |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
123 not from http://pypi.python.org/ . |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
124 |
f42df1456565
add a giant example section
Jeff Hammel <jhammel@mozilla.com>
parents:
78
diff
changeset
|
125 |
33
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
126 Running the Tests |
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
127 ----------------- |
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
128 |
78 | 129 The |
130 `tests directory <http://k0s.org/mozilla/hg/simpypi/file/tip/tests>`_ | |
131 contains | |
132 `doctests <http://docs.python.org/library/doctest.html>`_ and | |
133 the test-runner, | |
134 `test.py <http://k0s.org/mozilla/hg/simpypi/file/tip/tests/test.py>`_ . | |
135 These tests illustrate basic functionality and protect from | |
136 regressions if they are run before code is committed. | |
33
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
137 ``tests-require.txt`` contains dependencies that should be installed |
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
138 to run the tests. |
77 | 139 `Paste <http://pythonpaste.org/>`_ |
140 `TestApp <http://pythonpaste.org/testing-applications.html>`_ , | |
141 though this could be transitioned to | |
142 `WebTest <http://webtest.pythonpaste.org/en/latest/index.html>`_ . | |
143 `virtualenv <http://www.virtualenv.org/>`_ | |
144 is used for isolating python environments. | |
33
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
145 |
78 | 146 To run the tests, do:: |
147 | |
148 python test.py | |
149 | |
39 | 150 TODO |
151 ---- | |
152 | |
75 | 153 While simpypi is pretty simple, just because it is only 100 lines of |
154 code doesn't mean that it is the *right* 100 lines of code. The | |
155 following issues could be addressed: | |
156 | |
157 * the temporary package should be dealt with entirely in memory, | |
158 ideally. Currently we write to a file and move it. | |
159 | |
160 * ``simpypi`` use ``pkginfo.sdist`` to read the data from a source | |
161 distribution. Instead, the uploaded package should probably be | |
162 unpacked and ``python setup.py sdist`` run and the resulting | |
163 package put in the appropriate place. This will allow archives | |
164 without ``PKG-INFO`` | |
165 (e.g. http://hg.mozilla.org/build/talos/archive/tip.tar.gz ) | |
166 to be uploaded as well as at least partially correct for the fact | |
167 that currently ``simpypi`` only works for uploaded source | |
168 distributions. | |
39 | 169 |
80
0f3603a3d37d
note need to accept setup.py upload
Jeff Hammel <jhammel@mozilla.com>
parents:
79
diff
changeset
|
170 * ``simpypi`` does not accept packages upload in the usual python |
0f3603a3d37d
note need to accept setup.py upload
Jeff Hammel <jhammel@mozilla.com>
parents:
79
diff
changeset
|
171 way, e.g. ``python setup.py sdist upload -r http://url.of/simpypi`` . |
0f3603a3d37d
note need to accept setup.py upload
Jeff Hammel <jhammel@mozilla.com>
parents:
79
diff
changeset
|
172 It should. |
0f3603a3d37d
note need to accept setup.py upload
Jeff Hammel <jhammel@mozilla.com>
parents:
79
diff
changeset
|
173 |
81 | 174 Parallel Solutions |
175 ------------------ | |
176 | |
177 * https://github.com/SurveyMonkey/CheesePrism | |
178 | |
179 Reference | |
180 --------- | |
181 | |
182 * http://k0s.org/portfolio/pypi.html | |
183 * http://guide.python-distribute.org/contributing.html#the-simple-index-protocol | |
184 * http://www.python.org/~jeremy/weblog/030924.html | |
185 * https://bugzilla.mozilla.org/show_bug.cgi?id=701506 | |
186 | |
0 | 187 -- |
188 | |
33
058182b7b309
note test-requires.txt in the readme
Jeff Hammel <jhammel@mozilla.com>
parents:
0
diff
changeset
|
189 http://k0s.org/mozilla/hg/simpypi |