You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "xubaiwang (via GitHub)" <gi...@apache.org> on 2025/09/30 07:56:56 UTC

[PR] Implement Jupyter integration [arrow-js]

xubaiwang opened a new pull request, #296:
URL: https://github.com/apache/arrow-js/pull/296

   ## What's Changed
   
   This pull request implements Jupyter integration for arrow-js, including the following two mime type:
   
   - [x] `text/html`
   - [ ] `application/vnd.dataresource+json`
   
   This implementation basically follows `nodejs-polars`'s [implementation](https://github.com/pola-rs/nodejs-polars/blob/13181a13fc7f1b0515f7298b122b67dc1ea62847/polars/dataframe.ts#L2197-L2208).
   
   Closes #245.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Implement Jupyter integration [arrow-js]

Posted by "domoritz (via GitHub)" <gi...@apache.org>.
domoritz commented on PR #296:
URL: https://github.com/apache/arrow-js/pull/296#issuecomment-3352418380

   Maybe it's small enough to be okay. 
   
   Should columns also be supported?
   
   And for perf, does += perform similar to building an array and then joining it into a string? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Implement Jupyter integration [arrow-js]

Posted by "xubaiwang (via GitHub)" <gi...@apache.org>.
xubaiwang commented on PR #296:
URL: https://github.com/apache/arrow-js/pull/296#issuecomment-3350540202

   There is another point to discuss before I can implement it: how many rows and columns to display.
   
   - `pandas` use the [global options](https://pandas.pydata.org/docs/user_guide/options.html) `display.max_rows`, `display,max_columns`, `display,max_colwidth`
   - `polars` use the [environment variables](https://github.com/pola-rs/polars/blob/264cc0c5c284d49e5249425aab90e0a062c5fca5/polars/polars-core/src/config.rs#L2-L3) `POLARS_FMT_MAX_COLS`, `POLARS_FMT_MAX_ROWS`, `POLARS_FMT_STR_LEN` (and [`Config` contex](https://docs.pola.rs/api/python/stable/reference/api/polars.Config.set_tbl_cols.html#polars.Config.set_tbl_cols))
   
   I'm not sure which one is preferable.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Implement Jupyter integration [arrow-js]

Posted by "xubaiwang (via GitHub)" <gi...@apache.org>.
xubaiwang commented on PR #296:
URL: https://github.com/apache/arrow-js/pull/296#issuecomment-3350705444

   Here is an example of this integration (can preview on GitHub):
   
   https://gist.github.com/xubaiwang/d56db9d84c671dbaa75901813941bd3b


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Implement Jupyter integration [arrow-js]

Posted by "xubaiwang (via GitHub)" <gi...@apache.org>.
xubaiwang commented on PR #296:
URL: https://github.com/apache/arrow-js/pull/296#issuecomment-3352508702

   > And for perf, does += perform similar to building an array and then joining it into a string?
   
   This is why this is still a draft, there are many things left to implement and optimize.
   
   > Should columns also be supported?
   
   Do you mean something like `pandas.Series`?
   
   > Maybe it's small enough to be okay.
   
   However, the final implementation may be non trivial. There are other things like date formatting, json schema creation (for vnd.dataresource+json MIME type), formatting options, etc.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Implement Jupyter integration [arrow-js]

Posted by "domoritz (via GitHub)" <gi...@apache.org>.
domoritz commented on PR #296:
URL: https://github.com/apache/arrow-js/pull/296#issuecomment-3352948163

   I see. So maybe it does make sense to have a separate deno build then. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Implement Jupyter integration [arrow-js]

Posted by "xubaiwang (via GitHub)" <gi...@apache.org>.
xubaiwang commented on PR #296:
URL: https://github.com/apache/arrow-js/pull/296#issuecomment-3350569615

   I also failed to run `yarn build` with `ECONNREFUSED` error. How can I run the tests?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Implement Jupyter integration [arrow-js]

Posted by "domoritz (via GitHub)" <gi...@apache.org>.
domoritz commented on PR #296:
URL: https://github.com/apache/arrow-js/pull/296#issuecomment-3351839866

   Thanks for the pull request. I think we should not have this function live on the table (if possible) since it cannot be treeshaken. Is there a way to have it outside so we can tree shake the code when it's not needed? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Implement Jupyter integration [arrow-js]

Posted by "xubaiwang (via GitHub)" <gi...@apache.org>.
xubaiwang commented on PR #296:
URL: https://github.com/apache/arrow-js/pull/296#issuecomment-3352297101

   @domoritz I think it's currently impossible. 
   
   Although it can be moved to a separate function, this is far less useful in Jupyter context. People do not want an extra function to inspect a single value.
   
   Is it possible to create a Deno specific target/platform (`apache-arrow/deno` or `apache-arrow/jupyter`)? 
   As far as I know, all other JavaScript kernel has discontinued (`IJavascript`, `tslab`). 
   
   If we make this feature deno/jupyter platform specific (through prototype mutation and .d.ts), it will not affect other platforms.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org