You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "fangchenli (via GitHub)" <gi...@apache.org> on 2025/01/29 16:19:51 UTC
[I] [Python]Add support for pandas DataFrame.attrs [arrow]
fangchenli opened a new issue, #45382:
URL: https://github.com/apache/arrow/issues/45382
### Describe the enhancement requested
https://github.com/pandas-dev/pandas/issues/54321 added support for `DataFrame.attrs` during conversion between a DataFrame and a Parquet file. However, this should be done from the pyarrow side. Presumably in https://github.com/apache/arrow/blob/1a01b40c41ea849e348ef1e122587224346a772c/python/pyarrow/pandas_compat.py#L198
https://github.com/pandas-dev/pandas/issues/60675 is a related request for feather file.
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Add support for pandas DataFrame.attrs [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on issue #45382:
URL: https://github.com/apache/arrow/issues/45382#issuecomment-3170556562
@fangchenli now that this is merged, should we communicate it somewhere?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Add support for pandas DataFrame.attrs [arrow]
Posted by "fangchenli (via GitHub)" <gi...@apache.org>.
fangchenli commented on issue #45382:
URL: https://github.com/apache/arrow/issues/45382#issuecomment-3094712057
Thank you for picking up this issue. It looks good to me so far. The Arrow team will likely provide more insightful comments here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Add support for pandas DataFrame.attrs [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok closed issue #45382: [Python] Add support for pandas DataFrame.attrs
URL: https://github.com/apache/arrow/issues/45382
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Add support for pandas DataFrame.attrs [arrow]
Posted by "rmnskb (via GitHub)" <gi...@apache.org>.
rmnskb commented on issue #45382:
URL: https://github.com/apache/arrow/issues/45382#issuecomment-3288887382
Hey @wohenbushuang !
1. This change supports only the attributes of `pd.DataFrame`, as they were in scope for this issue. I'm not entirely sure if the `pd.Series` attributes can be handled on the Arrow side 🤔
2. Yes! The test for this change checks that this functionality is integrated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Add support for pandas DataFrame.attrs [arrow]
Posted by "rok (via GitHub)" <gi...@apache.org>.
rok commented on issue #45382:
URL: https://github.com/apache/arrow/issues/45382#issuecomment-3170551912
Issue resolved by pull request 47147
https://github.com/apache/arrow/pull/47147
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Add support for pandas DataFrame.attrs [arrow]
Posted by "wohenbushuang (via GitHub)" <gi...@apache.org>.
wohenbushuang commented on issue #45382:
URL: https://github.com/apache/arrow/issues/45382#issuecomment-3277345854
Thanks for your working! I have two questions:
1. Does this support attrs of columns (e.g., `df["col"].attrs["unit"]`)?
2. Can this work when using `pyarrow.Table.from_pandas(df)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Add support for pandas DataFrame.attrs [arrow]
Posted by "rmnskb (via GitHub)" <gi...@apache.org>.
rmnskb commented on issue #45382:
URL: https://github.com/apache/arrow/issues/45382#issuecomment-3094615492
Hey @fangchenli, my name's Bogdan, I'm part of EuroPython Sprints 2025, I was looking through the issues in this repo, and this one has piqued my interest, since I'm a daily user of pandas and pyarrow, and I've wanted to contribute for some time already.
Regarding the issue itself: I looked through the issues that you've attached, and noticed that right now you take existing metadata in Pandas during file reading or writing, and inject custom attributes to existing JSON.
As you've suggested, I decided to inject the attributes directly on pyarrow's end. The work is still in progress, but you can already see the underlying idea in the attached PR.
Please let me know if you have any additional comments or suggestions, I will be happy to accommodate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org