You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by "rymarm (via GitHub)" <gi...@apache.org> on 2024/05/15 11:46:01 UTC

[PR] DRILL-8495: Tried to remove unmanaged buffer (drill)

rymarm opened a new pull request, #2913:
URL: https://github.com/apache/drill/pull/2913

   # [DRILL-8495](https://issues.apache.org/jira/browse/DRILL-8495): Tried to remove unmanaged buffer
   
   The root cause of the issue is that multiple HiveWriters use the same `DrillBuf` and during execution they may reallocate the buffer if size of the buffer is not enough for a value (256 bytes+). Since `drillBuf.reallocIfNeeded(int size)` returns a new instance of `DrillBuf`, all other writers still have a reference for the old one buffer, which after `drillBuf.reallocIfNeeded(int size)` execution is unmanaged now.
   
   ## Description
   
   `HiveValueWriterFactory` now creates a unique `DrillBif` for each writer. 
   
   HiveWriters are actually used one-by-one and we could utilize a single buffer for all the writers. To do this, I could create a class holder for `DrillBuf`, so each writer has a reference for the same holder, where will be stored a new buffer from every `drillBuf.reallocIfNeeded(int size)` call. But I thought that such logic looked slightly confusing and I decided just to let each HiveWriter use its own buffer.
   
   ## Documentation
   \-
   
   ## Testing
   Add a new unit test to query a Hive table with variable-length values of Binary, VarChar, Char and String types.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] DRILL-8495: Tried to remove unmanaged buffer (drill)

Posted by "jnturton (via GitHub)" <gi...@apache.org>.
jnturton merged PR #2913:
URL: https://github.com/apache/drill/pull/2913


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] DRILL-8495: Tried to remove unmanaged buffer (drill)

Posted by "jnturton (via GitHub)" <gi...@apache.org>.
jnturton commented on PR #2913:
URL: https://github.com/apache/drill/pull/2913#issuecomment-2115109847

   P.S. I see that checkstyle is still upset.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] DRILL-8495: Tried to remove unmanaged buffer (drill)

Posted by "rymarm (via GitHub)" <gi...@apache.org>.
rymarm commented on PR #2913:
URL: https://github.com/apache/drill/pull/2913#issuecomment-2117347650

   @jnturton I addressed checkstyle issues and failed java tests. Should be fine now)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org