[GitHub] [arrow-datafusion] liukun4515 commented on a diff in pull request #2968: fix `RowWriter` index out of bounds error

[GitBox <gi...@apache.org>]

liukun4515 commented on code in PR #2968:
URL: https://github.com/apache/arrow-datafusion/pull/2968#discussion_r931725250


##########
datafusion/row/src/writer.rs:
##########
@@ -349,7 +349,7 @@ pub(crate) fn write_field_utf8(
     let from = from.as_any().downcast_ref::<StringArray>().unwrap();
     let s = from.value(row_idx);
     let new_width = to.current_width() + s.as_bytes().len();

Review Comment:
   @comphead you can go through the code of `set_utf8`.
   ```
       fn set_utf8(&mut self, idx: usize, value: &str) {
           self.assert_index_valid(idx);
           let bytes = value.as_bytes();
           let size = bytes.len();
           // this line: write the offset and size to the row_buffer
           self.set_offset_size(idx, size as u32);
           let varlena_offset = self.varlena_offset;
           self.data[varlena_offset..varlena_offset + size].copy_from_slice(bytes);
           self.varlena_offset += size;
           self.varlena_width += size;
       }
   ```
   Not per field, and variable length data type like utf8 and binary.
   From the @yjshen  codebase, I think the layout of utf8 and binary should be
   ```
   ─ ─ ─ ─ ─ ─  ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ 
   │ offset and size│     utf8 data or binary data │
   ─ ─ ─ ─ ─ ─  ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ 
   ```
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org