[GitHub] [arrow-rs] alamb commented on issue #180: Parquet does not support wasm32-unknown-unknown target

[GitBox <gi...@apache.org>]

alamb commented on issue #180:
URL: https://github.com/apache/arrow-rs/issues/180#issuecomment-826807095


   Comment from Dominik Moritz(domoritz) @ 2021-02-12T21:59:13.949+0000:
   <pre>If lz4 is the issue, maybe we could switch to https://github.com/PSeitz/lz4_flex, which compiles to WASM. </pre>
   
   Comment from Andy Redhead(AndyRedhead1974) @ 2021-03-04T21:51:18.454+0000:
   <pre>A WebAssembly compatible Rust library that can read data from Parquet files would be very useful to anyone who would like to do "browser based" data processing/visualisation (better still if that library is in a family that includes efficient in-memory "data structures"). </pre>
   
   Comment from David Roher(droher) @ 2021-04-12T02:03:38.512+0000:
   <pre>I just got a version of DataFusion working on wasm32-unknown-unknown – it required disabling both the LZ4 and ZSTD features on Parquet and tweaking the hash function: [https://github.com/apache/arrow/compare/master...droher:master]
   
   To add to [~AndyRedhead1974]'s point above, it would also be useful in a serverless context – for instance, Cloudflare Workers Unbound is in beta now and will allow WASM functions to run at unlimited CPU usage. in this context, DataFusion could be a serverless data lake engine like AWS Athena. Maybe it could even be useful as a Ballista worker.</pre>
   
   Comment from Dominik Moritz(domoritz) @ 2021-04-12T04:38:44.388+0000:
   <pre>That's awesome. Do you want to add a note to https://issues.apache.org/jira/projects/ARROW/issues/ARROW-11615, which tracks DataFusion support for wasm?</pre>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org