Mercurial > hg > numerics

annotate numerics/convert.py @ 150:8a1fe454c98a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
STUB
author Jeff Hammel <k0scist@gmail.com>
date Mon, 13 Apr 2015 10:17:47 -0700
parents 7578313b9fbf
children c16940bd2cee
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
1 #!/usr/bin/env python
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
2
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
3 """
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
4 convert between types
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
5 """
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
6
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
7 # imports
31
a655306ee78d stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 27
diff changeset
8 import argparse
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
9 import csv
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
10 import sys
52
37838ae694d2 import a thing
Jeff Hammel <k0scist@gmail.com>
parents: 47
diff changeset
11 from .data import transpose
47
6d34c02f7c9c inherit from the right thing
Jeff Hammel <k0scist@gmail.com>
parents: 46
diff changeset
12 from .read import read_csv, CSVParser
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
13
107
19a5c2fb52bb add transpose functionality
Jeff Hammel <k0scist@gmail.com>
parents: 81
diff changeset
14 __all__ = ['default_cast',
19a5c2fb52bb add transpose functionality
Jeff Hammel <k0scist@gmail.com>
parents: 81
diff changeset
15 'cast',
81
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
16 'float_or_orig',
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
17 'main']
33
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
18
45
ef915968d104 put this in the parser so that i can use this in convert
Jeff Hammel <k0scist@gmail.com>
parents: 34
diff changeset
19 default_cast = (int, float, str)
6
Jeff Hammel <k0scist@gmail.com>
parents: 5
diff changeset
20
5
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
21 def cast(to_type, *values):
81
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
22 """
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
23 gently cast a thing to a thing;
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
24 if you can't, return the original value
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
25 """
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
26
33
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
27
5
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
28 retval = []
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
29 for value in values:
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
30 try:
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
31 retval.append(to_type(value))
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
32 except ValueError:
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
33 retval.append(value)
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
34 return retval
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
35
81
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
36 def cast_or_discard(to_type, *values):
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
37 """
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
38 cast to `to_type` if possible;
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
39 otherwise just throw away
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
40 """
111
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
41 retval = []
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
42 for value in values:
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
43 try:
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
44 retval.append(to_type(value))
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
45 except ValueError:
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
46 continue
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
47 return retval
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
48
33
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
49
5
d5447d401c44 serializaion; pandas probably does this
Jeff Hammel <k0scist@gmail.com>
parents:
diff changeset
50 def float_or_orig(*values):
81
Jeff Hammel <k0scist@gmail.com>
parents: 60
diff changeset
51 return cast(float, *values)
111
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
52 # convenience function ; do we need this?
33
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
53
112
7578313b9fbf hook up basic plumbing
Jeff Hammel <k0scist@gmail.com>
parents: 111
diff changeset
54
34
Jeff Hammel <k0scist@gmail.com>
parents: 33
diff changeset
55 def column_type(values, types=default_cast):
Jeff Hammel <k0scist@gmail.com>
parents: 33
diff changeset
56 """determine the type of a column"""
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
57 for t in types:
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
58 for value in values:
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
59 try:
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
60 t(value)
56
06e487f90e05 this now works
Jeff Hammel <k0scist@gmail.com>
parents: 55
diff changeset
61 except ValueError, TypeError:
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
62 break
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
63 else:
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
64 return t
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
65
34
Jeff Hammel <k0scist@gmail.com>
parents: 33
diff changeset
66
59
3781174542bb note to self
Jeff Hammel <k0scist@gmail.com>
parents: 58
diff changeset
67 def cast_columns(columns, types=default_cast):
33
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
68 """
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
69 cast a column of data
f35b646eefa9 moar stubbin
Jeff Hammel <k0scist@gmail.com>
parents: 32
diff changeset
70 """
109
fae24f57dcb1 streamlining
Jeff Hammel <k0scist@gmail.com>
parents: 107
diff changeset
71 column_types = [column_type(column, types=types)
fae24f57dcb1 streamlining
Jeff Hammel <k0scist@gmail.com>
parents: 107
diff changeset
72 for column in columns]
111
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
73 return [[_type(row) for row in column]
c4d26ef63d8e fix error in convert and i think this works sorta well enouigh for now
Jeff Hammel <k0scist@gmail.com>
parents: 109
diff changeset
74 for _type, column in zip(column_types, columns)]
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
75
59
3781174542bb note to self
Jeff Hammel <k0scist@gmail.com>
parents: 58
diff changeset
76
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
77 def main(args=sys.argv[1:]):
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
78 """CLI"""
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
79
31
a655306ee78d stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 27
diff changeset
80 # parse command line
47
6d34c02f7c9c inherit from the right thing
Jeff Hammel <k0scist@gmail.com>
parents: 46
diff changeset
81 parser = CSVParser(description="interpolate types from file")
31
a655306ee78d stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 27
diff changeset
82 options = parser.parse_args(args)
a655306ee78d stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 27
diff changeset
83
109
fae24f57dcb1 streamlining
Jeff Hammel <k0scist@gmail.com>
parents: 107
diff changeset
84 # read CSV file columns
fae24f57dcb1 streamlining
Jeff Hammel <k0scist@gmail.com>
parents: 107
diff changeset
85 columns = parser.columns()
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
86
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
87 # get types
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
88 types = [column_type(column) for column in columns]
46
93850093eafd stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 45
diff changeset
89
93850093eafd stubbing
Jeff Hammel <k0scist@gmail.com>
parents: 45
diff changeset
90 # print type information
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
91 writer = csv.writer(sys.stdout)
56
06e487f90e05 this now works
Jeff Hammel <k0scist@gmail.com>
parents: 55
diff changeset
92 writer.writerow([t.__name__ for t in types])
55
ecaf1d4b1c2c this should work now; it doesnt, but it should
Jeff Hammel <k0scist@gmail.com>
parents: 52
diff changeset
93
32
Jeff Hammel <k0scist@gmail.com>
parents: 31
diff changeset
94
27
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
95 if __name__ == '__main__':
f865bc916593 stub for type inference
Jeff Hammel <k0scist@gmail.com>
parents: 6
diff changeset
96 main()