[GitHub] [incubator-doris] imay commented on issue #2075: Segment v2 stream load core dump(#2037)

[GitBox <gi...@apache.org>]

imay commented on issue #2075: Segment v2 stream load core dump(#2037)
URL: https://github.com/apache/incubator-doris/pull/2075#issuecomment-546959931
 
 
   > @imay
   > 
   > > I don't know why the second problem will be fixed by this PR.
   > 
   > For page produced by BitShuffleBuilder,ColumnWriter calls finish() to get Slice,that is calling BitShuffleBuilder.fastring->data(),but fastring may return a data array which not using new to create when not grow happens.
   > This pr using BitShuffleBuilder's finish_and_release method to produce page,that is calling BitShuffleBuilder's fastring.release,this page can be deleted by Page's Destructor;
   > 
   > > why create finish_and_release method?
   > 
   > The situation for PageBuilder in ColumnWriter is different from other situations,
   > Because all data produced by PageBuilder will be kept in ColumnWriter,ColumnWriter maintains the pages and delete at some time,it seems that calling PageBuilder.finish and then calling release() is ok;
   > But calling PageBuilder.finish means calling PageBuilder.faststring.finish,this comes back to the problem that when faststring not group,ColumnWriter will get a slice'data which not using new to create,so we must calling fastring.release to get data in ColumnWriter'context;
   > Calling PageBuilder.finish_and_release means calling fastring.release to get data
   
   @wangbo 
   I see and thanks for your reply.
   If then I think we have two choices to fix this problem.
   
   1. We make BitShuffleBuilder finish's return slice is same with what it will release. Then there is no need to add finish_and_release.
   2. We remove release interface and keep finish and finish_and_release. Because if we can't keep release and finish consistent, it will lead to other problems.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org