annotate paint/package.py @ 83:7f442f585580

paint/package.py
author Jeff Hammel <jhammel@mozilla.com>
date Tue, 17 Sep 2013 16:04:05 -0700
parents 4446cba4cee6
children 582ffe5e39a0
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4
e230ac76aaa4 stub package model
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
1 """
e230ac76aaa4 stub package model
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
2 package model for python PAckage INTrospection
e230ac76aaa4 stub package model
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
3 """
e230ac76aaa4 stub package model
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
4
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
5 # TODO: use pkginfo.sdist more
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
6
57
d5e5c7496784 stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 53
diff changeset
7 import info
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
8 import os
28
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
9 import pip
9
562230cc2e86 now the package class should at least extracts things. it doesnt actually do anything useful
Jeff Hammel <jhammel@mozilla.com>
parents: 8
diff changeset
10 import shutil
19
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
11 import subprocess
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
12 import sys
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
13 import tarfile
5
5b74792d46e4 stub package model;
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
14 import tempfile
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
15 import urllib2
21
4df3e715817d now have passing tests again
Jeff Hammel <jhammel@mozilla.com>
parents: 20
diff changeset
16 import urlparse
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
17 import utils
5
5b74792d46e4 stub package model;
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
18
62
726008cff4e3 remove code supporting old version
Jeff Hammel <jhammel@mozilla.com>
parents: 58
diff changeset
19 from subprocess import check_call as call
19
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
20
10
d63ae03cc300 * fix up __init__ imports
Jeff Hammel <jhammel@mozilla.com>
parents: 9
diff changeset
21 __all__ = ['Package']
d63ae03cc300 * fix up __init__ imports
Jeff Hammel <jhammel@mozilla.com>
parents: 9
diff changeset
22
4
e230ac76aaa4 stub package model
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
23 class Package(object):
33
5efb61fde8aa improved docstring
Jeff Hammel <jhammel@mozilla.com>
parents: 32
diff changeset
24 """
81
4446cba4cee6 update CommandParser and docstring for it
Jeff Hammel <jhammel@mozilla.com>
parents: 78
diff changeset
25 python package introspection.
4446cba4cee6 update CommandParser and docstring for it
Jeff Hammel <jhammel@mozilla.com>
parents: 78
diff changeset
26 invoked with the package source, 'src'
33
5efb61fde8aa improved docstring
Jeff Hammel <jhammel@mozilla.com>
parents: 32
diff changeset
27 """
4
e230ac76aaa4 stub package model
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
28
50
Jeff Hammel <jhammel@mozilla.com>
parents: 49
diff changeset
29 def __init__(self, src, verbose=True):
63
ad174d9f396d docstring
Jeff Hammel <jhammel@mozilla.com>
parents: 62
diff changeset
30 """
ad174d9f396d docstring
Jeff Hammel <jhammel@mozilla.com>
parents: 62
diff changeset
31 - src : URL or filesystem path to the package
81
4446cba4cee6 update CommandParser and docstring for it
Jeff Hammel <jhammel@mozilla.com>
parents: 78
diff changeset
32 - verbose : show verbose output
63
ad174d9f396d docstring
Jeff Hammel <jhammel@mozilla.com>
parents: 62
diff changeset
33 """
83
7f442f585580 paint/package.py
Jeff Hammel <jhammel@mozilla.com>
parents: 81
diff changeset
34 # TODO: directory of sources
7f442f585580 paint/package.py
Jeff Hammel <jhammel@mozilla.com>
parents: 81
diff changeset
35 # e.g. http://hg.mozilla.org/mozilla-central/file/tip/testing/mozbase
4
e230ac76aaa4 stub package model
Jeff Hammel <jhammel@mozilla.com>
parents:
diff changeset
36 self.src = src
50
Jeff Hammel <jhammel@mozilla.com>
parents: 49
diff changeset
37 self.verbose = verbose
13
0dd1f8f83be2 moving towards some sort of organization
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
38
0dd1f8f83be2 moving towards some sort of organization
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
39 # ephemeral data
5
5b74792d46e4 stub package model;
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
40 self._tmppath = None
28
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
41 self._egg_info_path = None
33
5efb61fde8aa improved docstring
Jeff Hammel <jhammel@mozilla.com>
parents: 32
diff changeset
42 self._build_path = None
40
06f21791eba1 add yet another indirection layer
Jeff Hammel <jhammel@mozilla.com>
parents: 39
diff changeset
43 self._pkg_info_path = None
5
5b74792d46e4 stub package model;
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
44
64
1a279bac0afa start moving this monster to a package information provider
Jeff Hammel <jhammel@mozilla.com>
parents: 63
diff changeset
45 # package metadata provider
1a279bac0afa start moving this monster to a package information provider
Jeff Hammel <jhammel@mozilla.com>
parents: 63
diff changeset
46 # TODO: this should be chooseable but currently the keys of the
1a279bac0afa start moving this monster to a package information provider
Jeff Hammel <jhammel@mozilla.com>
parents: 63
diff changeset
47 # interfaces are not the same
1a279bac0afa start moving this monster to a package information provider
Jeff Hammel <jhammel@mozilla.com>
parents: 63
diff changeset
48 self.package_info = info.EggInfo
1a279bac0afa start moving this monster to a package information provider
Jeff Hammel <jhammel@mozilla.com>
parents: 63
diff changeset
49
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
50 # TODO: list of temporary files/directories to be deleted
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
51
49
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
52 def _log(self, message):
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
53 if self.verbose:
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
54 print '>>> %s' % message
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
55
34
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
56 def _path(self):
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
57 """filesystem path to package directory"""
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
58
51
8b4243893474 more cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 50
diff changeset
59 self._log(">>> _path:Getting path to package")
48
abfb267e8332 debugging info; lord i hate this
Jeff Hammel <jhammel@mozilla.com>
parents: 47
diff changeset
60
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
61 # return cached copy if it exists
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
62 if self._tmppath:
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
63 return self._tmppath
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
64
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
65 # fetch from the web if a URL
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
66 tmpfile = None
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
67 src = self.src
5
5b74792d46e4 stub package model;
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
68 if utils.isURL(self.src):
8
c1181e9c9ca2 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 7
diff changeset
69 tmpfile = src = self.fetch()
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
70
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
71 # unpack if an archive
28
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
72 if self._is_archive(src):
8
c1181e9c9ca2 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 7
diff changeset
73 try:
37
36e70712fc9a things are just awful
Jeff Hammel <jhammel@mozilla.com>
parents: 36
diff changeset
74 self._unpack(src)
8
c1181e9c9ca2 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 7
diff changeset
75 finally:
c1181e9c9ca2 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 7
diff changeset
76 if tmpfile:
c1181e9c9ca2 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 7
diff changeset
77 os.remove(tmpfile)
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
78 return self._tmppath
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
79
5
5b74792d46e4 stub package model;
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
80 return self.src
5b74792d46e4 stub package model;
Jeff Hammel <jhammel@mozilla.com>
parents: 4
diff changeset
81
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
82 def fetch(self, filename=None):
8
c1181e9c9ca2 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 7
diff changeset
83 """fetch from remote source to a temporary file"""
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
84 if filename is None:
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
85 fd, filename = tempfile.mkstemp()
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
86 os.close(fd)
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
87 fp = file(filename, 'w')
15
8c8b7482772f fix a few failures but still failing
Jeff Hammel <jhammel@mozilla.com>
parents: 14
diff changeset
88 resource = urllib2.urlopen(self.src)
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
89 fp.write(resource.read())
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
90 fp.close()
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
91 return filename
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
92
37
36e70712fc9a things are just awful
Jeff Hammel <jhammel@mozilla.com>
parents: 36
diff changeset
93 def _unpack(self, archive):
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
94 """unpack the archive to a temporary destination"""
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
95 # TODO: should handle zipfile additionally at least
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
96 # Ideally, this would be pluggable, etc
8
c1181e9c9ca2 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 7
diff changeset
97 assert tarfile.is_tarfile(archive), "%s is not an archive" % self.src
9
562230cc2e86 now the package class should at least extracts things. it doesnt actually do anything useful
Jeff Hammel <jhammel@mozilla.com>
parents: 8
diff changeset
98 tf = tarfile.TarFile.open(archive)
562230cc2e86 now the package class should at least extracts things. it doesnt actually do anything useful
Jeff Hammel <jhammel@mozilla.com>
parents: 8
diff changeset
99 self._tmppath = tempfile.mkdtemp()
19
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
100 members = tf.getmembers()
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
101
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
102 # cut off the top level directory
37
36e70712fc9a things are just awful
Jeff Hammel <jhammel@mozilla.com>
parents: 36
diff changeset
103 members = [i for i in members if os.path.sep in i.name]
19
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
104 tld = set()
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
105 for member in members:
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
106 directory, member.name = member.name.split(os.path.sep, 1)
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
107 tld.add(directory)
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
108 assert len(tld) == 1
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
109
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
110 # extract
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
111 for member in members:
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
112 tf.extract(member, path=self._tmppath)
c54db80d6e7f fix test
Jeff Hammel <jhammel@mozilla.com>
parents: 18
diff changeset
113 tf.close()
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
114
28
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
115 def _is_archive(self, path):
7
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
116 """returns if the filesystem path is an archive"""
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
117 # TODO: should handle zipfile additionally at least
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
118 # Ideally, this would be pluggable, etc
a7bf894c96c7 yet more stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 5
diff changeset
119 return tarfile.is_tarfile(path)
9
562230cc2e86 now the package class should at least extracts things. it doesnt actually do anything useful
Jeff Hammel <jhammel@mozilla.com>
parents: 8
diff changeset
120
28
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
121 def _cleanup(self):
9
562230cc2e86 now the package class should at least extracts things. it doesnt actually do anything useful
Jeff Hammel <jhammel@mozilla.com>
parents: 8
diff changeset
122 if self._tmppath:
562230cc2e86 now the package class should at least extracts things. it doesnt actually do anything useful
Jeff Hammel <jhammel@mozilla.com>
parents: 8
diff changeset
123 shutil.rmtree(self._tmppath)
562230cc2e86 now the package class should at least extracts things. it doesnt actually do anything useful
Jeff Hammel <jhammel@mozilla.com>
parents: 8
diff changeset
124 self._tmppath = None
562230cc2e86 now the package class should at least extracts things. it doesnt actually do anything useful
Jeff Hammel <jhammel@mozilla.com>
parents: 8
diff changeset
125
65
6d6f5c5c26d4 add __del__ method
Jeff Hammel <jhammel@mozilla.com>
parents: 64
diff changeset
126 __del__ = _cleanup
11
c41a946d34af stub out info method
Jeff Hammel <jhammel@mozilla.com>
parents: 10
diff changeset
127
13
0dd1f8f83be2 moving towards some sort of organization
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
128 ### python-package-specific functionality
0dd1f8f83be2 moving towards some sort of organization
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
129
40
06f21791eba1 add yet another indirection layer
Jeff Hammel <jhammel@mozilla.com>
parents: 39
diff changeset
130 def info(self):
06f21791eba1 add yet another indirection layer
Jeff Hammel <jhammel@mozilla.com>
parents: 39
diff changeset
131 """return info dictionary for package"""
53
c588375a7ce4 starting a new strawman to work out kinks
Jeff Hammel <jhammel@mozilla.com>
parents: 52
diff changeset
132 self._log(">>> Getting the info")
66
af7609457dc6 introduce a failing test; yay!
Jeff Hammel <jhammel@mozilla.com>
parents: 65
diff changeset
133 return self.package_info(self._path())()
13
0dd1f8f83be2 moving towards some sort of organization
Jeff Hammel <jhammel@mozilla.com>
parents: 12
diff changeset
134
11
c41a946d34af stub out info method
Jeff Hammel <jhammel@mozilla.com>
parents: 10
diff changeset
135 def dependencies(self):
78
ab0620d3755b docstring
Jeff Hammel <jhammel@mozilla.com>
parents: 72
diff changeset
136 """list dependencies of a package"""
68
d001017d5870 migrate this code to the right place
Jeff Hammel <jhammel@mozilla.com>
parents: 66
diff changeset
137 return self.package_info(self._path()).dependencies()
24
54c1e7e43a54 stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
138
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
139 def extension(self):
35
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
140 """filename extension of the package"""
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
141
49
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
142 print ">>> extension:Getting package"
35
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
143 package = self.package()
49
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
144 print ">>> extension:package=%s" % package
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
145
35
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
146 # determine the extension (XXX hacky)
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
147 extensions = ('.tar.gz', '.zip', '.tar.bz2')
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
148 for ext in extensions:
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
149 if package.endswith(ext):
35
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
150 return ext
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
151
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
152 raise Exception("Extension %s not found: %s" % (extensions, package))
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
153
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
154 def package(self, destination=None):
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
155 """
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
156 repackage the package to ensure its actually in the right form
34
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
157 and return the path to the destination
49
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
158 - destination: if given, path to put the build in
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
159 """
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
160
49
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
161 self._log("package: Getting package directory, destination=%s" % repr(destination))
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
162
33
5efb61fde8aa improved docstring
Jeff Hammel <jhammel@mozilla.com>
parents: 32
diff changeset
163 if self._build_path:
49
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
164 self._log("package: build_path already set: %s" % self._build_path)
34
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
165 if destination:
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
166 shutil.copy(self._build_path, destination)
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
167 return os.path.abspath(destination)
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
168
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
169 # return cached copy
33
5efb61fde8aa improved docstring
Jeff Hammel <jhammel@mozilla.com>
parents: 32
diff changeset
170 return self._build_path
5efb61fde8aa improved docstring
Jeff Hammel <jhammel@mozilla.com>
parents: 32
diff changeset
171
34
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
172 path = self._path()
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
173 dist = os.path.join(path, 'dist')
49
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
174 self._log("package: dist directory: %s; (path=%s)" % (dist, path))
34
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
175 if os.path.exists(dist):
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
176 shutil.rmtree(dist)
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
177
49
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
178 command = [sys.executable, 'setup.py', 'sdist']
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
179 self._log("package: running: %s" % ' '.join(command))
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
180 call(command, cwd=path)
85374a69cf89 call in foreground to avoid hang :(
Jeff Hammel <jhammel@mozilla.com>
parents: 48
diff changeset
181 self._log("package: done running setup.py dist")
34
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
182
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
183 assert os.path.exists(dist)
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
184 contents = os.listdir(dist)
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
185 assert len(contents) == 1
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
186
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
187 self._build_path = os.path.join(dist, contents[0])
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
188
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
189 # destination
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
190 # use an evil recursive trick
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
191 if destination:
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
192 return self.package(destination=destination)
34
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
193
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
194 return self._build_path
acb2f4896291 working towards something
Jeff Hammel <jhammel@mozilla.com>
parents: 33
diff changeset
195
24
54c1e7e43a54 stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
196 def download(self, directory):
54c1e7e43a54 stubbing
Jeff Hammel <jhammel@mozilla.com>
parents: 21
diff changeset
197 """download a package and all its dependencies using pip"""
28
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
198 if not os.path.exists(directory):
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
199 os.makedirs(directory)
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
200 assert os.path.isdir(directory)
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
201 pip.main(['install', '--download', directory, self.src])
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
202
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
203 def pypi(self, directory):
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
204 """
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
205 download packages for a pypi directory structure
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
206 http://k0s.org/portfolio/pypi.html
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
207 """
29
59524b6d25c0 implementing making pypi directory strcuture from a package
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
208 if not os.path.exists(directory):
59524b6d25c0 implementing making pypi directory strcuture from a package
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
209 os.makedirs(directory)
59524b6d25c0 implementing making pypi directory strcuture from a package
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
210 assert os.path.isdir(directory)
28
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
211 tempdir = tempfile.mkdtemp()
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
212 try:
29
59524b6d25c0 implementing making pypi directory strcuture from a package
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
213 self.download(tempdir)
48
abfb267e8332 debugging info; lord i hate this
Jeff Hammel <jhammel@mozilla.com>
parents: 47
diff changeset
214 files = os.listdir(tempdir)
52
026d693cd561 cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 51
diff changeset
215 self._log(">>> Files: %s" % files)
48
abfb267e8332 debugging info; lord i hate this
Jeff Hammel <jhammel@mozilla.com>
parents: 47
diff changeset
216 for f in files:
29
59524b6d25c0 implementing making pypi directory strcuture from a package
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
217
59524b6d25c0 implementing making pypi directory strcuture from a package
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
218 # full path
47
213cf933366d stupid mistakes
Jeff Hammel <jhammel@mozilla.com>
parents: 45
diff changeset
219 src = os.path.join(tempdir, f)
29
59524b6d25c0 implementing making pypi directory strcuture from a package
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
220
35
e8d73f9e99fb extension checking
Jeff Hammel <jhammel@mozilla.com>
parents: 34
diff changeset
221 # make a package of the thing
48
abfb267e8332 debugging info; lord i hate this
Jeff Hammel <jhammel@mozilla.com>
parents: 47
diff changeset
222 print ">>> pypi:Packaging %s" % src
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
223 package = Package(src)
48
abfb267e8332 debugging info; lord i hate this
Jeff Hammel <jhammel@mozilla.com>
parents: 47
diff changeset
224 print ">>> pypi:DONE packaging %s" % src
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
225
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
226 # get destination dirname, filename
48
abfb267e8332 debugging info; lord i hate this
Jeff Hammel <jhammel@mozilla.com>
parents: 47
diff changeset
227 print ">>> pypi:Getting PyPI path"
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
228 dirname, filename = package.pypi_path()
48
abfb267e8332 debugging info; lord i hate this
Jeff Hammel <jhammel@mozilla.com>
parents: 47
diff changeset
229 print ">>> pypi:DONE PyPI path: %s/%s" % (dirname, filename)
29
59524b6d25c0 implementing making pypi directory strcuture from a package
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
230
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
231 # make the directory if it doesn't exist
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
232 subdir = os.path.join(directory, dirname)
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
233 if not os.path.exists(subdir):
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
234 os.makedirs(subdir)
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
235 assert os.path.isdir(subdir)
29
59524b6d25c0 implementing making pypi directory strcuture from a package
Jeff Hammel <jhammel@mozilla.com>
parents: 28
diff changeset
236
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
237 # move the file
48
abfb267e8332 debugging info; lord i hate this
Jeff Hammel <jhammel@mozilla.com>
parents: 47
diff changeset
238 print ">>> pypi:Moving to PyPI path %s/%s" % (subdir, filename)
47
213cf933366d stupid mistakes
Jeff Hammel <jhammel@mozilla.com>
parents: 45
diff changeset
239 package.package(destination=os.path.join(subdir, filename))
48
abfb267e8332 debugging info; lord i hate this
Jeff Hammel <jhammel@mozilla.com>
parents: 47
diff changeset
240 print ">>> Done with %s" % src
28
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
241 finally:
c44fb4b6918b CLI sorta works
Jeff Hammel <jhammel@mozilla.com>
parents: 25
diff changeset
242 shutil.rmtree(tempdir)
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
243
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
244 def pypi_path(self):
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
245 """
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
246 returns subpath 2-tuple appropriate for pypi path structure:
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
247 http://k0s.org/portfolio/pypi.html
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
248 """
71
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
249 self._log(">>> pypi_path:Getting info")
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
250 info = self.info()
71
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
251 self._log(">>> pypi_path:DONE getting info")
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
252
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
253 # determine the extension
71
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
254 self._log(">>> pypi_path:Getting extension")
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
255 extension = self.extension()
71
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
256 self._log(">>> pypi_path:DONE Getting extension: %s" % extension)
31
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
257
5fb1844db0b2 back to the drawring broad
Jeff Hammel <jhammel@mozilla.com>
parents: 30
diff changeset
258 # get the filename destination
36
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
259 name = info['Name']
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
260 version = info['Version']
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
261 filename = '%s-%s%s' % (name, version, extension)
f59da9e6be37 this almost works, dammit
Jeff Hammel <jhammel@mozilla.com>
parents: 35
diff changeset
262 return name, filename
71
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
263
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
264
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
265 class PackageSet(object):
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
266 """
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
267 a group of packages
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
268 """
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
269
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
270 def __init__(self, *packages):
72
017b75cd61d8 dont hardcode adding in __init__; thats just stupid
Jeff Hammel <jhammel@mozilla.com>
parents: 71
diff changeset
271 self.packages = []
71
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
272 for package in packages:
72
017b75cd61d8 dont hardcode adding in __init__; thats just stupid
Jeff Hammel <jhammel@mozilla.com>
parents: 71
diff changeset
273 self.add(package)
71
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
274 raise NotImplementedError
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
275
72
017b75cd61d8 dont hardcode adding in __init__; thats just stupid
Jeff Hammel <jhammel@mozilla.com>
parents: 71
diff changeset
276 def add(self, package):
017b75cd61d8 dont hardcode adding in __init__; thats just stupid
Jeff Hammel <jhammel@mozilla.com>
parents: 71
diff changeset
277 """
017b75cd61d8 dont hardcode adding in __init__; thats just stupid
Jeff Hammel <jhammel@mozilla.com>
parents: 71
diff changeset
278 add a package to the instance.
017b75cd61d8 dont hardcode adding in __init__; thats just stupid
Jeff Hammel <jhammel@mozilla.com>
parents: 71
diff changeset
279 If Package instance, add that, otherwise convert
017b75cd61d8 dont hardcode adding in __init__; thats just stupid
Jeff Hammel <jhammel@mozilla.com>
parents: 71
diff changeset
280 """
017b75cd61d8 dont hardcode adding in __init__; thats just stupid
Jeff Hammel <jhammel@mozilla.com>
parents: 71
diff changeset
281
017b75cd61d8 dont hardcode adding in __init__; thats just stupid
Jeff Hammel <jhammel@mozilla.com>
parents: 71
diff changeset
282 __iadd__ = add
017b75cd61d8 dont hardcode adding in __init__; thats just stupid
Jeff Hammel <jhammel@mozilla.com>
parents: 71
diff changeset
283
71
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
284 def dependencies(self):
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
285 """return dependecies for each package"""
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
286
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
287 def unroll_dependencies(self):
da69d58f960d stub method of conversion, PackageSet, and some cleanup
Jeff Hammel <jhammel@mozilla.com>
parents: 69
diff changeset
288 """unroll dependencies for a package set"""