use Parquet.jl named tuples iterator: RecordCursor#25
use Parquet.jl named tuples iterator: RecordCursor#25tanmaykm wants to merge 4 commits intoqueryverse:masterfrom
Conversation
This [Parquet.jl update](JuliaIO/Parquet.jl#71) will add a named tuple iterator `RecordReader`. We will be able to use that here directly, instead of wrapping over the older `RecCursor`. The new `map_logical_types` option to `ParFile` automatically converts byte arrays to strings, so we do not need to handle that here now.
|
I think this broke live. julia> using Queryverse Package Planenvironment location: /home/blair/.julia/conda/3 added / updated specs: The following packages will be downloaded: The following NEW packages will be INSTALLED: xlrd pkgs/main/linux-64::xlrd-1.2.0-py37_0 Downloading and Extracting Packages |
|
Rec cursor isn't defined any more, but, it is being referred to. |
|
I think merging this would fix that issue. Parquet.jl has moved to I feel there should be an upperbound to the compat version range here so that we don't run into this issue again. I shall push a commit with that in a bit. |
|
@davidanthoff does this look good to be merged? |
|
I'd say merge this with master to fix the conflicts in Project.toml which has proper upper bounding. |
|
merging this will fix #26 and all the file read issues |
This Parquet.jl update will add a named tuple iterator
RecordReader. We will be able to use that here directly, instead of wrapping over the olderRecCursor.The new
map_logical_typesoption toParFileautomatically converts byte arrays to strings, so we do not need to handle that here now.This can be merged after JuliaIO/Parquet.jl#71 is released in Parquet.jl