Mercurial > hg > numerics

annotate numerics/convert.py @ 164:c16940bd2cee

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
this works
author Jeff Hammel <k0scist@gmail.com>
date Fri, 15 May 2015 16:59:09 -0700
parents 7578313b9fbf
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
1 #!/usr/bin/env python
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
2
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
3 """
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
4 convert between types
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
5 """
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
6
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
7 # imports
31
a655306ee78d stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 27
diff changeset
8 import argparse
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
9 import csv
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
10 import sys
52
37838ae694d2 import a thing
Jeff Hammel <k0scist@gmail.com>
parents: 47
diff changeset
11 from .data import transpose
47
6d34c02f7c9c inherit from the right thing
Jeff Hammel <k0scist@gmail.com>
parents: 46
diff changeset
12 from .read import read_csv, CSVParser
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
13
164
c16940bd2cee this works
Jeff Hammel <k0scist@gmail.com>
parents: 112
diff changeset
14 __all__ = ['numeric',
c16940bd2cee this works
Jeff Hammel <k0scist@gmail.com>
parents: 112
diff changeset
15 'default_cast',
107
19a5c2fb52bb add transpose functionality
Jeff Hammel <k0scist@gmail.com>
parents: 81
diff changeset
16 'cast',
81
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
17 'float_or_orig',
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
18 'main']
33
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
19
164
c16940bd2cee this works
Jeff Hammel <k0scist@gmail.com>
parents: 112
diff changeset
20 numeric = (int, float)
45
ef915968d104 put this in the parser so that i can use this in convert
Jeff Hammel <k0scist@gmail.com>
parents: 34
diff changeset
21 default_cast = (int, float, str)
6
Jeff Hammel <k0scist@gmail.com>
parents: 5
diff changeset
22
5
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
23 def cast(to_type, *values):
81
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
24 """
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
25 gently cast a thing to a thing;
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
26 if you can't, return the original value
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
27 """
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
28
33
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
29
5
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
30 retval = []
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
31 for value in values:
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
32 try:
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
33 retval.append(to_type(value))
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
34 except ValueError:
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
35 retval.append(value)
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
36 return retval
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
37
81
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
38 def cast_or_discard(to_type, *values):
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
39 """
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
40 cast to `to_type` if possible;
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
41 otherwise just throw away
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
42 """
111
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
43 retval = []
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
44 for value in values:
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
45 try:
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
46 retval.append(to_type(value))
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
47 except ValueError:
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
48 continue
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
49 return retval
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
50
33
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
51
5
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
52 def float_or_orig(*values):
81
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
53 return cast(float, *values)
111
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
54 # convenience function ; do we need this?
33
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
55
112
7578313b9fbf hook up basic plumbing
Jeff Hammel <k0scist@gmail.com>
parents: 111
diff changeset
56
34
Jeff Hammel <k0scist@gmail.com>
parents: 33
diff changeset
57 def column_type(values, types=default_cast):
Jeff Hammel <k0scist@gmail.com>
parents: 33
diff changeset
58 """determine the type of a column"""
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
59 for t in types:
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
60 for value in values:
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
61 try:
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
62 t(value)
56
06e487f90e05 this now works
Jeff Hammel <k0scist@gmail.com>
parents: 55
diff changeset
63 except ValueError, TypeError:
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
64 break
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
65 else:
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
66 return t
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
67
34
Jeff Hammel <k0scist@gmail.com>
parents: 33
diff changeset
68
59
3781174542bb note to self
Jeff Hammel <k0scist@gmail.com>
parents: 58
diff changeset
69 def cast_columns(columns, types=default_cast):
33
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
70 """
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
71 cast a column of data
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
72 """
109
fae24f57dcb1 streamlining
Jeff Hammel <k0scist@gmail.com>
parents: 107
diff changeset
73 column_types = [column_type(column, types=types)
fae24f57dcb1 streamlining
Jeff Hammel <k0scist@gmail.com>
parents: 107
diff changeset
74 for column in columns]
111
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
75 return [[_type(row) for row in column]
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
76 for _type, column in zip(column_types, columns)]
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
77
59
3781174542bb note to self
Jeff Hammel <k0scist@gmail.com>
parents: 58
diff changeset
78
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
79 def main(args=sys.argv[1:]):
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
80 """CLI"""
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
81
31
a655306ee78d stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 27
diff changeset
82 # parse command line
47
6d34c02f7c9c inherit from the right thing
Jeff Hammel <k0scist@gmail.com>
parents: 46
diff changeset
83 parser = CSVParser(description="interpolate types from file")
31
a655306ee78d stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 27
diff changeset
84 options = parser.parse_args(args)
a655306ee78d stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 27
diff changeset
85
109
fae24f57dcb1 streamlining
Jeff Hammel <k0scist@gmail.com>
parents: 107
diff changeset
86 # read CSV file columns
fae24f57dcb1 streamlining
Jeff Hammel <k0scist@gmail.com>
parents: 107
diff changeset
87 columns = parser.columns()
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
88
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
89 # get types
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
90 types = [column_type(column) for column in columns]
46
93850093eafd stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 45
diff changeset
91
93850093eafd stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 45
diff changeset
92 # print type information
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
93 writer = csv.writer(sys.stdout)
56
06e487f90e05 this now works
Jeff Hammel <k0scist@gmail.com>
parents: 55
diff changeset
94 writer.writerow([t.__name__ for t in types])
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
95
32
Jeff Hammel <k0scist@gmail.com>
parents: 31
diff changeset
96
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
97 if __name__ == '__main__':
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
98 main()