annotate carton.py @ 33:6d8d30e561b2

do something crazy
author Jeff Hammel <jhammel@mozilla.com>
date Mon, 11 Jul 2011 13:04:42 -0700
parents 8fef97a25925
children 122c56779f2b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
1 #!/usr/bin/env python
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
2
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
3 """
29
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
4 make a self-extracting virtualenv from directories or URLs of packages
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
5
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
6 To package up all files in a virtualenvs source directory (e.g.)::
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
7
29
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
8 python path/to/carton.py myproject project/src/*
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
9
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
10 This will create a self-extracting file, `myproject.py`, that will unfold
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
11 a virtualenv with the specified packages setup for development
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
12
29
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
13 The sources may be directories, local or HTTP-accessible tarballs, or ordinary
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
14 files. The `setup.py`s found in the `src` directory after extraction will be
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
15 run (via `python setup.py develop`) in the order they are provided. This makes
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
16 it possible to have completely local dependencies (without touching the net)
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
17 by correctly specifying the source order. If a `setup.py` is overwritten from
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
18 a later source, it will not be rerun (known limitation).
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
19
29
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
20 The extracted virtualenv will be created in the current directory and will have
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
21 the same name as provided initially (e.g. `myproject`) unless `--env` is
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
22 specified.
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
23
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
24 Normally, the entire contents of source directories are compressed and
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
25 packaged as-is. When running with the `--package` flag, a source tarball is
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
26 produced via `python setup.py sdist` if the directory contains a top-level
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
27 `setup.py`.
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
28
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
29 Since directories are compressed as-is, portable file-based VCS repositories,
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
30 such a mercurial and git, may be cartoned this way (though note that newer
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
31 repositories may not be backwards-compatible with older clients).
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
32 """
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
33
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
34 # imports
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
35 import os
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
36 import sys
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
37 import subprocess
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
38 import tarfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
39 import tempfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
40 import urllib2
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
41 from optparse import OptionParser
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
42 from StringIO import StringIO
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
43
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
44 # global variables
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
45 usage = "%prog [options] environment_name directory|url [...]"
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
46 virtualenv_url = 'http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.6.1.tar.gz'
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
47 template = """#!/usr/bin/env python
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
48
15
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
49 "create a virtualenv at %(ENV)s"
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
50
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
51 import os
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
52 import shutil
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
53 import subprocess
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
54 import sys
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
55 import tarfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
56 import tempfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
57 from optparse import OptionParser
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
58 from StringIO import StringIO
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
59
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
60 try:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
61 call = subprocess.check_call
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
62 except AttributeError:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
63 # old python; boo :(
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
64 call = subprocess.call
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
65
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
66 # virtualenv name
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
67 ENV='''%(ENV)s'''
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
68
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
69 # packed files
3
75919adb199a use compression, but it doesnt seem to help much
Jeff Hammel <jhammel@mozilla.com>
parents: 0
diff changeset
70 VIRTUAL_ENV='''%(VIRTUAL_ENV)s'''.decode('base64').decode('zlib')
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
71 PACKAGE_SOURCES=%(PACKAGE_SOURCES)s
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
72 CARTON=%(CARTON)s
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
73
15
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
74 # parse options
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
75 usage = os.path.basename(sys.argv[0]) + ' [options]'
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
76 parser = OptionParser(usage=usage, description=__doc__)
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
77 parser.add_option('--env', dest='env', help="environment name [DEFAULT: " + ENV + "]")
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
78 options, args = parser.parse_args()
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
79 if options.env:
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
80 ENV = options.env
f05e636b7444 now you can specify the name of the env in the created script
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
81
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
82 # unpack virtualenv
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
83 tempdir = tempfile.mkdtemp()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
84 buffer = StringIO()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
85 buffer.write(VIRTUAL_ENV)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
86 buffer.seek(0)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
87 tf = tarfile.open(mode='r', fileobj=buffer)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
88 tf.extractall(tempdir)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
89
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
90 # find the virtualenv
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
91 for root, dirs, files in os.walk(tempdir):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
92 if 'virtualenv.py' in files:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
93 virtualenv = os.path.join(root, 'virtualenv.py')
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
94 break
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
95 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
96 raise Exception("virtualenv.py not found in " + tempdir)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
97
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
98 # create the virtualenv
17
b05f5f1ec26e pop PYTHONHOME as this will fuxor things
Jeff Hammel <jhammel@mozilla.com>
parents: 16
diff changeset
99 os.environ.pop('PYTHONHOME', None)
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
100 call([sys.executable, virtualenv, ENV])
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
101
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
102 # find the bin/scripts directory
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
103 for i in ('bin', 'Scripts'):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
104 scripts_dir = os.path.abspath(os.path.join(ENV, i))
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
105 if os.path.exists(scripts_dir):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
106 break
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
107 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
108 raise Exception("Scripts directory not found in " + ENV)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
109
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
110 # find the virtualenv's python
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
111 for i in ('python', 'python.exe'):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
112 python = os.path.join(scripts_dir, i)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
113 if os.path.exists(python):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
114 break
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
115 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
116 raise Exception("python not found in " + scripts_dir)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
117
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
118 # unpack the sources and setup for development
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
119 srcdir = os.path.join(ENV, 'src')
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
120 os.mkdir(srcdir)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
121 setup_pys = set()
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
122 for source in PACKAGE_SOURCES:
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
123 source = source.decode('base64').decode('zlib')
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
124 buffer = StringIO()
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
125 buffer.write(source)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
126 buffer.seek(0)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
127 tf = tarfile.open(mode='r', fileobj=buffer)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
128 tf.extractall(srcdir)
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
129
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
130 # setup sources for development if there are any new setup.py files
16
c91abbdc871b * add TODO: dependencies
Jeff Hammel <jhammel@mozilla.com>
parents: 15
diff changeset
131 # TODO: ideally this would figure out dependency order for you
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
132 for i in os.listdir(srcdir):
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
133 if i in setup_pys:
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
134 continue
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
135 subdir = os.path.join(srcdir, i)
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
136 if os.path.exists(os.path.join(srcdir, i, 'setup.py')):
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
137 call([python, 'setup.py', 'develop'], cwd=subdir)
29
533a4f9718d9 improve documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
138 # TODO: try `setup.py install` if develop fails for distutils packages
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
139 setup_pys.add(i)
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
140
22
9c710f06e51d add virtualenv to the virtualenv(!)
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
141 # add virtualenv to the virtualenv (!)
9c710f06e51d add virtualenv to the virtualenv(!)
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
142 virtualenv_dir = os.path.dirname(virtualenv)
9c710f06e51d add virtualenv to the virtualenv(!)
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
143 if os.path.exists(os.path.join(virtualenv_dir, 'setup.py')):
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
144 call([python, 'setup.py', 'install'], cwd=virtualenv_dir, stdout=subprocess.PIPE)
22
9c710f06e51d add virtualenv to the virtualenv(!)
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
145
27
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
146 # add carton to the virtualenv (!)
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
147 if CARTON:
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
148 CARTON = CARTON.decode('base64').decode('zlib')
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
149 carton_filename = os.path.join(scripts_dir, 'carton.py')
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
150 f = file(carton_filename, 'w')
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
151 f.write(CARTON)
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
152 f.close()
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
153 try:
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
154 os.chmod(carton_filename, 0755)
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
155 except:
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
156 # you probably don't have os.chmod
dace84448c25 carry carton along with carton
Jeff Hammel <jhammel@mozilla.com>
parents: 26
diff changeset
157 pass
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
158
31
8fef97a25925 near version bump and documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
159 # cleanup virtualenv tempdir
8fef97a25925 near version bump and documentation
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
160 shutil.rmtree(tempdir)
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
161 """
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
162
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
163 def isURL(path):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
164 return path.startswith('http://') or path.startswith('https://')
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
165
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
166 try:
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
167 call = subprocess.check_call
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
168 except AttributeError:
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
169 # old python; boo :(
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
170 call = subprocess.call
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
171
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
172 def main(args=sys.argv[1:]):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
173
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
174 # parse CLI arguments
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
175 parser = OptionParser(usage=usage, description=__doc__)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
176 parser.add_option('-o', dest='outfile',
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
177 help="specify outfile; otherwise it will come from environment_name")
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
178 parser.add_option('-p', '--package', dest='package',
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
179 action='store_true', default=False,
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
180 help="create python packages from sources; do not take entire subdirectory")
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
181 parser.add_option('--virtualenv', dest='virtualenv',
12
542b46ac4e28 fix description more better
Jeff Hammel <jhammel@mozilla.com>
parents: 11
diff changeset
182 help="use this virtualenv URL or file tarball")
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
183 options, args = parser.parse_args(args)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
184 if len(args) < 2:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
185 parser.print_usage()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
186 parser.exit()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
187 environment = args[0]
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
188 if environment.endswith('.py'):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
189 # stop on .py; will add it in later
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
190 environment = environment[:-3]
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
191 sources = args[1:]
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
192
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
193 # tar up the sources
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
194 source_array = []
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
195 for source in sources:
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
196 buffer = None
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
197
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
198 if isURL(source):
14
c474362cf69a allow fetching of remote tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 13
diff changeset
199 # remote tarball or resource
c474362cf69a allow fetching of remote tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 13
diff changeset
200 buffer = urllib2.urlopen(source).read()
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
201 else:
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
202 # local directory or tarball
13
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
203 assert os.path.exists(source), "%s does not exist" % source
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
204
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
205 # package up the source if applicable
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
206 if options.package and os.path.exists(os.path.join(source, 'setup.py')):
24
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
207
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
208 # create a .tar.gz package
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
209 call([sys.executable, 'setup.py', 'sdist'], cwd=source, stdout=subprocess.PIPE)
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
210 dist_dir = os.path.join(source, 'dist')
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
211 assert os.path.isdir(dist_dir), "dist directory not created in %s" % source
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
212 tarfiles = [i for i in os.listdir(dist_dir)
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
213 if i.endswith('.tar.gz')]
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
214 assert tarfiles, "no .tar.gz files found in %s" % dist_dir
24
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
215
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
216 # use the last modified tarball
18
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
217 def last_modified(filename):
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
218 return os.path.getmtime(os.path.join(dist_dir, filename))
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
219 tarfiles.sort(key=last_modified)
c6a03199d4bf stub out package creation; next: to test this
Jeff Hammel <jhammel@mozilla.com>
parents: 17
diff changeset
220 source = os.path.join(dist_dir, tarfiles[-1])
13
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
221
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
222 if (not os.path.isdir(source)) and tarfile.is_tarfile(source):
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
223 # check for a tarball
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
224 buffer = file(source).read()
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
225 else:
24
bb19d3dac4c5 fix tests
Jeff Hammel <jhammel@mozilla.com>
parents: 23
diff changeset
226 # add other sources (files and directories) to the archive
13
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
227 source_buffer = StringIO()
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
228 source_tar = tarfile.open(mode="w:gz", fileobj=source_buffer)
26
e6ee0410ceef fix bug where os.path.sep is wrongly interpreted
Jeff Hammel <jhammel@mozilla.com>
parents: 24
diff changeset
229 source_tar.add(source, arcname=os.path.basename(source.rstrip(os.path.sep)))
13
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
230 source_tar.close()
f522620c6a78 now works properly with tarballs
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
231 buffer = source_buffer.getvalue()
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
232
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
233 # could use git, hg, etc repos. but probably shouldn't
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
234
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
235 source_array.append(buffer.encode('zlib').encode('base64'))
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
236
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
237 # tar up virtualenv if not available
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
238 if options.virtualenv:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
239 if isURL(options.virtualenv):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
240 globals()['VIRTUAL_ENV'] = urllib2.urlopen(options.virtualenv).read()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
241 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
242 assert os.path.exists(options.virtualenv)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
243 if os.path.isdir(options.virtualenv):
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
244 raise NotImplementedError("Hypothetically you should be able to use a local directory or tarball, but I haven't done this yet")
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
245 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
246 # assert a tarfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
247 assert tarfile.is_tarfile(options.virtualenv), "%s must be a tar file" % options.virtualenv
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
248 globals()['VIRTUAL_ENV'] = file(options.virtualenv).read()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
249 else:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
250 globals()['VIRTUAL_ENV'] = urllib2.urlopen(virtualenv_url).read()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
251 # TODO: used the below hashed value of VIRTUAL_ENV if set
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
252 # (set that with another file)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
253
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
254 # get the contents of this file
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
255 carton = None
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
256 try:
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
257 if __file__:
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
258 filename = __file__.rstrip('c') # avoid pyfiles
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
259 if os.path.exists(filename):
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
260 carton = file(filename).read().encode('zlib').encode('base64')
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
261 except NameError:
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
262 pass
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
263
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
264 # interpolate "template" -> output
30
0b9c124f128b note another TODO
Jeff Hammel <jhammel@mozilla.com>
parents: 29
diff changeset
265 # TODO: add the ability to include a post-deployment script
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
266 outfile = options.outfile
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
267 if outfile is None:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
268 outfile = environment + '.py'
3
75919adb199a use compression, but it doesnt seem to help much
Jeff Hammel <jhammel@mozilla.com>
parents: 0
diff changeset
269 variables = {'VIRTUAL_ENV': VIRTUAL_ENV.encode('zlib').encode('base64'),
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
270 'ENV': environment,
23
987086aad234 stub adding carton to to virtualenv
Jeff Hammel <jhammel@mozilla.com>
parents: 22
diff changeset
271 'CARTON': repr(carton),
11
e6a62ba0c24d now respect the order sources are installed in
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
272 'PACKAGE_SOURCES': repr(source_array)}
0
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
273 f = file(outfile, 'w')
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
274 f.write(template % variables)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
275 f.close()
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
276 try:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
277 os.chmod(outfile, 0755)
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
278 except:
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
279 # you probably don't have os.chmod
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
280 pass
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
281
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
282 VIRTUAL_ENV = """"""
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
283
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
284 if __name__ == '__main__':
1c7da6388dd9 initial commit of carton
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
285 main()