[jira] [Commented] (SPARK-21412) Reset BufferHolder while initialize an UnsafeRowWriter

["Apache Spark (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-21412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16086780#comment-16086780 ] 

Apache Spark commented on SPARK-21412:
--------------------------------------

User 'gczsjdy' has created a pull request for this issue:
https://github.com/apache/spark/pull/18632

> Reset BufferHolder while initialize an UnsafeRowWriter
> ------------------------------------------------------
>
>                 Key: SPARK-21412
>                 URL: https://issues.apache.org/jira/browse/SPARK-21412
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.1.1
>            Reporter: Chenzhao Guo
>
> UnsafeRowWriter's construtor should contain BufferHolder.reset to make the writer out of the box.
> While writing UnsafeRow using UnsafeRowWriter, developers should manually call BufferHolder.reset to make the BufferHolder's cursor(which indicates where to write variable length portion) right in order to write variable length fields like UTF8String, byte[], etc.
> If a developer doesn't reuse the BufferHolder so maybe he never noticed reset and the comments in code, the UnsafeRow won't be correct if he also writes variable length UTF8String. This API design doesn't make sense. We should reset the BufferHolder to make UnsafeRowWriter out of the box, but not let user read the code and manually call it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org