[GitHub] [incubator-druid] leventov opened a new issue #3803: Concurrent modification of responseContext in QueryRunners

[GitBox <gi...@apache.org>]

leventov opened a new issue #3803: Concurrent modification of responseContext in QueryRunners
URL: https://github.com/apache/incubator-druid/issues/3803
 
 
   responseContext is a plain `HashMap`. It is accessed/modified concurrently without any synchronization in `AsyncQueryRunner`, `IntervalChunkingQueryRunner`, `ChainedExecutionQueryFactory`, etc.
   
   We didn't notice this problem before, because `responseContext` had very limited use, but as part of #3798 I started to use responseContexts more extensively and got `ConcurrentModificationException`.
   
   I quickly fixed this in this commit: https://github.com/metamx/druid/commit/54c2af24966e73d6f8c5951ec53e5c0407c653d2 but I don't like this solution, especially how it works in `AsyncQueryRunner` and `DirectDruidClient`, because it is super fragile and relies on what particular types of `QueryRunner`s call `AsyncQueryRunner` and called from `DirectDruidClient`. There are two main problems:
    - it is not specified how responseContext could and could not be used and when it is "finalized": after return from `QueryRunner.run()`, or after iteration over the returned `Sequence` is completed.
    - No difference between "sync" and "async" (lazy) QueryRunners/Sequences
   
   I don't have an idea yet what a better solution would be.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org