You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/05/06 07:48:03 UTC

[GitHub] [druid] asafhalili opened a new issue #11209: [Feature Proposal] Allow Query Only Historical Data (without realtime data)

asafhalili opened a new issue #11209:
URL: https://github.com/apache/druid/issues/11209


   ### Description
   
   I suggest to allow query only historical data, without realtime data.
   Meaning, the brokers will query only the historical nodes and not the realtime nodes.
   
   It can be added as a configuration variable to the brokers, something like:
   `druid.query.useRealtimeData=false` (the default value will be true) and as a query context parameter too.
   
   ### Motivation
   
   The main use-case is to improve performance, by not query the realtime nodes at all.
   For some use-cases the historical data is sufficient.
   
   Another benefit is reducing cost, since we need less resources for the realtime nodes.
   
   ### A use-case for example
   
   In one of our use-cases, we want to support data freshness of up to 1 hour.
   In this case, I'll configure the `taskDuration` or the `intermediateHandoffPeriod` of the ingestion tasks to 1 hour
   and the parameter I suggested above to false (`druid.query.useRealtimeData=false`)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

[GitHub] [druid] egor-ryashin commented on issue #11209: [Feature Proposal] Allow Query Only Historical Data (without realtime data)

Posted by GitBox <gi...@apache.org>.
egor-ryashin commented on issue #11209:
URL: https://github.com/apache/druid/issues/11209#issuecomment-846185731


   Just wondering if using batch ingestion can be a workaround?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

[GitHub] [druid] asafhalili commented on issue #11209: [Feature Proposal] Allow Query Only Historical Data (without realtime data)

Posted by GitBox <gi...@apache.org>.
asafhalili commented on issue #11209:
URL: https://github.com/apache/druid/issues/11209#issuecomment-846244058


   Hi @egor-ryashin,
   It can be a workaround, but for organizations that use realtime ingestion, it requires changing the ingestion architecture and it isn't a trivial change to do.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org