Mercurial > hg > Lemuriformes
changeset 6:244c29f46554
add module for chunking data
| author | Jeff Hammel <k0scist@gmail.com> |
|---|---|
| date | Sun, 10 Dec 2017 11:52:37 -0800 |
| parents | dbbf5344868c |
| children | 92ae11e33f85 |
| files | lemuriformes/chunk.py |
| diffstat | 1 files changed, 12 insertions(+), 0 deletions(-) [+] |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/lemuriformes/chunk.py Sun Dec 10 11:52:37 2017 -0800 @@ -0,0 +1,12 @@ +def chunk(data, chunksize=100000): + """ + chunk `data` into smaller sizes of `chunksize` + """ + + offset = 0 + chunk = data[0:chunksize] + while chunk: + yield chunk + offset += chunksize + chunk = data[offset:offset+chunksize] +