Re: Throw NullPointerException occasionally when query from stream table

[xm_zzc <44...@qq.com>]

Hi:
  The root cause is that when execute select sql, BlockDataMap will call
'SegmentPropertiesAndSchemaHolder.addSegmentProperties ' to add segment info
one by one, meanwhile if there are some segments updated, for example,
stream segment is handoff , handoff thread will call
'SegmentPropertiesAndSchemaHolder.invalidate' to delete segment info one by
one, if segmentIdAndSegmentPropertiesIndexWrapper.segmentIdSet.isEmpty() is
true, it will remove segmentPropertiesIndex, but select thread is still
using segmentPropertiesIndex to add/get segment info, and then NPE occur. 



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: Throw NullPointerException occasionally when query from stream table

[manishgupta88 <to...@gmail.com>]

Hi xm_zzc

As I can see from logs and the code flow, the hand-off code clears the cache
from executor code while the exception is thrown from driver code during
query.
You are getting the exception because you are using local mode whereIn
executor and driver are in the same JVM. We will check on this how can we
avoid the exception in local mode scenario. Meanwhile you can continue your
testing in cluster mode.

Regards
Manish Gupta



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: Throw NullPointerException occasionally when query from stream table

[xm_zzc <44...@qq.com>]

Hi David:
  please see the call stack: 
<http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/file/t133/QQ%E6%88%AA%E5%9B%BE20181106140118.png> 



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: Throw NullPointerException occasionally when query from stream table

[David CaiQiang <da...@gmail.com>]

Where do we call SegmentPropertiesAndSchemaHolder.invalidate in handoff
thread?



-----
Best Regards
David Cai
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/