[GitHub] [arrow] mhkeller opened a new issue, #35041: How to write an arrow file in Node.JS from an IPC stream?

["mhkeller (via GitHub)" <gi...@apache.org>]

mhkeller opened a new issue, #35041:
URL: https://github.com/apache/arrow/issues/35041

   ### Describe the usage question you have. Please include as many useful details as  possible.
   
   
   I'm trying to use the DuckDB method [`db.arrowIPCStream`](https://duckdb.org/docs/api/nodejs/reference#module_duckdb..Connection+arrowIPCStream) to read from a postgres database, return an IPC stream and then write to an arrow file. However, I'm having trouble serializing it to disk. While the [NodeJS Arrow docs](https://arrow.apache.org/docs/js/) show file reading, there aren't many examples showing file writing.
   
   Here are some example code snippets that could help but I haven't been able to stitch them together to get a working solution yet: 
   
   1. Writing JSON to arrow: https://github.com/apache/arrow/blob/main/js/bin/json-to-arrow.js
   2. Getting an array of objects from an IPC stream: https://github.com/duckdblabs/arrow/blob/main/test/nodejs/arrow_ipc.test.js
   3. Various streaming tests: https://github.com/duckdblabs/arrow/blob/main/test/nodejs/arrow_ipc.test.js
   
   I'd like to avoid converting the data into an array of objects, if possible. Thanks for any pointers or help in figuring this out.
   
   ### Component(s)
   
   Documentation, JavaScript


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] mhkeller commented on issue #35041: [JavaScript] How to write an arrow file in Node.JS from an IPC stream?

["mhkeller (via GitHub)" <gi...@apache.org>]

mhkeller commented on issue #35041:
URL: https://github.com/apache/arrow/issues/35041#issuecomment-1689171844

   Is there any update on this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [JavaScript] How to write an arrow file in Node.JS from an IPC stream? [arrow]

["domoritz (via GitHub)" <gi...@apache.org>]

domoritz closed issue #35041: [JavaScript] How to write an arrow file in Node.JS from an IPC stream?
URL: https://github.com/apache/arrow/issues/35041


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [JavaScript] How to write an arrow file in Node.JS from an IPC stream? [arrow]

["fawazahmed0 (via GitHub)" <gi...@apache.org>]

fawazahmed0 commented on issue #35041:
URL: https://github.com/apache/arrow/issues/35041#issuecomment-1835460455

   I am using these methods to read and write arrow file:
   ```js
   const arrow = require('apache-arrow')
   const fs = require('fs')
   
   // Read arrow file
   const table = arrow.tableFromIPC(fs.readFileSync('simple.arrow'))
   // Write arrow file
   fs.writeFileSync('simple.arrow', arrow.tableToIPC(table, 'file'))
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org