You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Bryan Cutler (JIRA)" <ji...@apache.org> on 2017/09/19 23:38:02 UTC
[jira] [Updated] (SPARK-22067) ArrowWriter StringWriter not using
position of ByteBuffer holding data
[ https://issues.apache.org/jira/browse/SPARK-22067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Cutler updated SPARK-22067:
---------------------------------
Description: When ArrowWriter is copying a StringType column to ArrowData, then StringWriter gets data as a ByteBuffer and sets the ArrowBuf using a position of 0. This currently works because of a bug in Arrow ARROW-1443 that was not making use of the position. This has been fixed as of Arrow 0.7.0 and this surfaced when testing out this version. To fix this, the position of the ByteBuffer should be passed into {{setSafe}} instead of 0. (was: When ArrowWriter is copying a StringType column to ArrowData, then StringWriter gets data as a ByteBuffer and sets the ArrowBuf using a position of 0. This currently works because of a bug in Arrow ARROW-1447 that was not making use of the position. This has been fixed as of Arrow 0.7.0 and this surfaced when testing out this version. To fix this, the position of the ByteBuffer should be passed into {{setSafe}} instead of 0.)
> ArrowWriter StringWriter not using position of ByteBuffer holding data
> ----------------------------------------------------------------------
>
> Key: SPARK-22067
> URL: https://issues.apache.org/jira/browse/SPARK-22067
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.0
> Reporter: Bryan Cutler
>
> When ArrowWriter is copying a StringType column to ArrowData, then StringWriter gets data as a ByteBuffer and sets the ArrowBuf using a position of 0. This currently works because of a bug in Arrow ARROW-1443 that was not making use of the position. This has been fixed as of Arrow 0.7.0 and this surfaced when testing out this version. To fix this, the position of the ByteBuffer should be passed into {{setSafe}} instead of 0.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org