You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "ianmcook (via GitHub)" <gi...@apache.org> on 2024/05/21 13:36:32 UTC

[PR] [SPARK-48374][PYTHON] Support additional PyArrow Table column types [spark]

ianmcook opened a new pull request, #46688:
URL: https://github.com/apache/spark/pull/46688

   ### What changes were proposed in this pull request?
   This is a small follow-up to #46529. This adds support for some more Arrow data types:
   - fixed-size binary
   - fixed-size list
   - large list
   
   ### Why are the changes needed?
   Users who are creating Spark DataFrames from PyArrow Tables will expect it to work if their Tables contain these types of columns
   
   ### Does this PR introduce _any_ user-facing change?
   It will prevent an error in the case where the user has one of these types of columns in their PyArrow Table. There are no other user-facing changes.
   
   ### How was this patch tested?
   Tests are included.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   No


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-48374][PYTHON] Support additional PyArrow Table column types [spark]

Posted by "ianmcook (via GitHub)" <gi...@apache.org>.
ianmcook commented on PR #46688:
URL: https://github.com/apache/spark/pull/46688#issuecomment-2123227176

   The tests will fail until after #46529 is merged and I rebase.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-48374][PYTHON] Support additional PyArrow Table column types [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #46688:
URL: https://github.com/apache/spark/pull/46688#issuecomment-2144079911

   Merged to master.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-48374][PYTHON] Support additional PyArrow Table column types [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #46688:
URL: https://github.com/apache/spark/pull/46688#issuecomment-2144080154

   Thank you for fixing those.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-48374][PYTHON] Support additional PyArrow Table column types [spark]

Posted by "ianmcook (via GitHub)" <gi...@apache.org>.
ianmcook commented on PR #46688:
URL: https://github.com/apache/spark/pull/46688#issuecomment-2143883197

   cc @HyukjinKwon @xinrong-meng @zhengruifeng
   
   This is a small improvement that I split out from #46529. Thank you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-48374][PYTHON] Support additional PyArrow Table column types [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #46688: [SPARK-48374][PYTHON] Support additional PyArrow Table column types
URL: https://github.com/apache/spark/pull/46688


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org