You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2018/11/13 22:33:29 UTC
[GitHub] fjy closed pull request #6591: Deprecate IntervalChunkingQueryRunner
fjy closed pull request #6591: Deprecate IntervalChunkingQueryRunner
URL: https://github.com/apache/incubator-druid/pull/6591
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/content/configuration/index.md b/docs/content/configuration/index.md
index f30c555eb44..c1cf723c1a7 100644
--- a/docs/content/configuration/index.md
+++ b/docs/content/configuration/index.md
@@ -1205,7 +1205,7 @@ Druid broker can optionally retry queries internally for transient errors.
#### Processing
-The broker uses processing configs for nested groupBy queries. And, optionally, Long-interval queries (of any type) can be broken into shorter interval queries and processed in parallel inside this thread pool. For more details, see "chunkPeriod" in [Query Context](../querying/query-context.html) doc.
+The broker uses processing configs for nested groupBy queries. And, if you use groupBy v1, long-interval queries (of any type) can be broken into shorter interval queries and processed in parallel inside this thread pool. For more details, see "chunkPeriod" in [Query Context](../querying/query-context.html) doc.
|Property|Description|Default|
|--------|-----------|-------|
diff --git a/docs/content/operations/metrics.md b/docs/content/operations/metrics.md
index 5144b0acb04..99c31583761 100644
--- a/docs/content/operations/metrics.md
+++ b/docs/content/operations/metrics.md
@@ -34,7 +34,7 @@ Available Metrics
|`query/node/bytes`|number of bytes returned from querying individual historical/realtime nodes.|id, status, server.| |
|`query/node/ttfb`|Time to first byte. Milliseconds elapsed until broker starts receiving the response from individual historical/realtime nodes.|id, status, server.|< 1s|
|`query/node/backpressure`|Milliseconds that the channel to this node has spent suspended due to backpressure.|id, status, server.| |
-|`query/intervalChunk/time`|Only emitted if interval chunking is enabled. Milliseconds required to query an interval chunk.|id, status, chunkInterval (if interval chunking is enabled).|< 1s|
+|`query/intervalChunk/time`|Only emitted if interval chunking is enabled. Milliseconds required to query an interval chunk. This metric is deprecated and will be removed in the future because interval chunking is deprecated. See [Query Context](../querying/query-context.html).|id, status, chunkInterval (if interval chunking is enabled).|< 1s|
|`query/count`|number of total queries|This metric is only available if the QueryCountStatsMonitor module is included.||
|`query/success/count`|number of queries successfully processed|This metric is only available if the QueryCountStatsMonitor module is included.||
|`query/failed/count`|number of failed queries|This metric is only available if the QueryCountStatsMonitor module is included.||
diff --git a/docs/content/querying/query-context.md b/docs/content/querying/query-context.md
index 3df148b20b5..189c35f8769 100644
--- a/docs/content/querying/query-context.md
+++ b/docs/content/querying/query-context.md
@@ -18,7 +18,7 @@ The query context is used for various query configuration parameters. The follow
|populateResultLevelCache | `false` | Flag indicating whether to save the results of the query to the result level cache. Primarily used for debugging. When set to false, it disables saving the results of this query to the query cache. When set to true, Druid uses druid.broker.cache.populateCache to determine whether or not to save the results of this query to the query cache |
|bySegment | `false` | Return "by segment" results. Primarily used for debugging, setting it to `true` returns results associated with the data segment they came from |
|finalize | `true` | Flag indicating whether to "finalize" aggregation results. Primarily used for debugging. For instance, the `hyperUnique` aggregator will return the full HyperLogLog sketch instead of the estimated cardinality when this flag is set to `false` |
-|chunkPeriod | `P0D` (off) | At the broker node level, long interval queries (of any type) may be broken into shorter interval queries to parallelize merging more than normal. Broken up queries will use a larger share of cluster resources, but may be able to complete faster as a result. Use ISO 8601 periods. For example, if this property is set to `P1M` (one month), then a query covering a year would be broken into 12 smaller queries. The broker uses its query processing executor service to initiate processing for query chunks, so make sure "druid.processing.numThreads" is configured appropriately on the broker. [groupBy queries](groupbyquery.html) do not support chunkPeriod by default, although they do if using the legacy "v1" engine. |
+|chunkPeriod | `P0D` (off) | At the broker node level, long interval queries (of any type) may be broken into shorter interval queries to parallelize merging more than normal. Broken up queries will use a larger share of cluster resources, but, if you use groupBy "v1, it may be able to complete faster as a result. Use ISO 8601 periods. For example, if this property is set to `P1M` (one month), then a query covering a year would be broken into 12 smaller queries. The broker uses its query processing executor service to initiate processing for query chunks, so make sure "druid.processing.numThreads" is configured appropriately on the broker. [groupBy queries](groupbyquery.html) do not support chunkPeriod by default, although they do if using the legacy "v1" engine. This context is deprecated since it's only useful for groupBy "v1", and will be removed in the future releases.|
|maxScatterGatherBytes| `druid.server.http.maxScatterGatherBytes` | Maximum number of bytes gathered from data nodes such as historicals and realtime processes to execute a query. This parameter can be used to further reduce `maxScatterGatherBytes` limit at query time. See [broker configuration](../configuration/index.html#broker) for more details.|
|maxQueuedBytes | `druid.broker.http.maxQueuedBytes` | Maximum number of bytes queued per query before exerting backpressure on the channel to the data server. Similar to `maxScatterGatherBytes`, except unlike that configuration, this one will trigger backpressure rather than query failure. Zero means disabled.|
|serializeDateTimeAsLong| `false` | If true, DateTime is serialized as long in the result returned by broker and the data transportation between broker and compute node|
diff --git a/processing/src/main/java/org/apache/druid/query/DefaultQueryMetrics.java b/processing/src/main/java/org/apache/druid/query/DefaultQueryMetrics.java
index b332fec98df..5531614b71d 100644
--- a/processing/src/main/java/org/apache/druid/query/DefaultQueryMetrics.java
+++ b/processing/src/main/java/org/apache/druid/query/DefaultQueryMetrics.java
@@ -225,6 +225,7 @@ public void identity(String identity)
return reportMillisTimeMetric("query/segmentAndCache/time", timeNs);
}
+ @Deprecated
@Override
public QueryMetrics<QueryType> reportIntervalChunkTime(long timeNs)
{
diff --git a/processing/src/main/java/org/apache/druid/query/IntervalChunkingQueryRunner.java b/processing/src/main/java/org/apache/druid/query/IntervalChunkingQueryRunner.java
index 4cf31a04b10..5bc76d03a45 100644
--- a/processing/src/main/java/org/apache/druid/query/IntervalChunkingQueryRunner.java
+++ b/processing/src/main/java/org/apache/druid/query/IntervalChunkingQueryRunner.java
@@ -40,7 +40,10 @@
import java.util.concurrent.ExecutorService;
/**
+ * This class is deprecated and will removed in the future.
+ * See https://github.com/apache/incubator-druid/pull/4004#issuecomment-284171911 for details about deprecation.
*/
+@Deprecated
public class IntervalChunkingQueryRunner<T> implements QueryRunner<T>
{
private final QueryRunner<T> baseRunner;
diff --git a/processing/src/main/java/org/apache/druid/query/IntervalChunkingQueryRunnerDecorator.java b/processing/src/main/java/org/apache/druid/query/IntervalChunkingQueryRunnerDecorator.java
index 075518ed0d6..97eb9e68af1 100644
--- a/processing/src/main/java/org/apache/druid/query/IntervalChunkingQueryRunnerDecorator.java
+++ b/processing/src/main/java/org/apache/druid/query/IntervalChunkingQueryRunnerDecorator.java
@@ -26,6 +26,11 @@
import java.util.concurrent.ExecutorService;
+/**
+ * This class is deprecated and will removed in the future.
+ * See https://github.com/apache/incubator-druid/pull/4004#issuecomment-284171911 for details about deprecation.
+ */
+@Deprecated
public class IntervalChunkingQueryRunnerDecorator
{
private final ExecutorService executor;
diff --git a/processing/src/main/java/org/apache/druid/query/QueryContexts.java b/processing/src/main/java/org/apache/druid/query/QueryContexts.java
index 3399aa9b1a2..a1922180709 100644
--- a/processing/src/main/java/org/apache/druid/query/QueryContexts.java
+++ b/processing/src/main/java/org/apache/druid/query/QueryContexts.java
@@ -35,6 +35,7 @@
public static final String MAX_SCATTER_GATHER_BYTES_KEY = "maxScatterGatherBytes";
public static final String MAX_QUEUED_BYTES_KEY = "maxQueuedBytes";
public static final String DEFAULT_TIMEOUT_KEY = "defaultTimeout";
+ @Deprecated
public static final String CHUNK_PERIOD_KEY = "chunkPeriod";
public static final boolean DEFAULT_BY_SEGMENT = false;
@@ -132,6 +133,7 @@
return parseInt(query, PRIORITY_KEY, defaultValue);
}
+ @Deprecated
public static <T> String getChunkPeriod(Query<T> query)
{
return query.getContextValue(CHUNK_PERIOD_KEY, "P0D");
diff --git a/processing/src/main/java/org/apache/druid/query/groupby/DefaultGroupByQueryMetricsFactory.java b/processing/src/main/java/org/apache/druid/query/groupby/DefaultGroupByQueryMetricsFactory.java
index f8de6c4a879..045f27ab7fd 100644
--- a/processing/src/main/java/org/apache/druid/query/groupby/DefaultGroupByQueryMetricsFactory.java
+++ b/processing/src/main/java/org/apache/druid/query/groupby/DefaultGroupByQueryMetricsFactory.java
@@ -34,8 +34,7 @@
/**
* Should be used only in tests, directly or indirectly (via {@link
- * GroupByQueryQueryToolChest#GroupByQueryQueryToolChest(org.apache.druid.query.groupby.strategy.GroupByStrategySelector,
- * org.apache.druid.query.IntervalChunkingQueryRunnerDecorator)}).
+ * GroupByQueryQueryToolChest#GroupByQueryQueryToolChest}).
*/
@VisibleForTesting
public static GroupByQueryMetricsFactory instance()
diff --git a/processing/src/main/java/org/apache/druid/query/groupby/GroupByQueryQueryToolChest.java b/processing/src/main/java/org/apache/druid/query/groupby/GroupByQueryQueryToolChest.java
index e47f64df6ef..a49b48ae216 100644
--- a/processing/src/main/java/org/apache/druid/query/groupby/GroupByQueryQueryToolChest.java
+++ b/processing/src/main/java/org/apache/druid/query/groupby/GroupByQueryQueryToolChest.java
@@ -86,6 +86,7 @@
public static final String GROUP_BY_MERGE_KEY = "groupByMerge";
private final GroupByStrategySelector strategySelector;
+ @Deprecated
private final IntervalChunkingQueryRunnerDecorator intervalChunkingQueryRunnerDecorator;
private final GroupByQueryMetricsFactory queryMetricsFactory;
diff --git a/processing/src/main/java/org/apache/druid/query/search/SearchQueryQueryToolChest.java b/processing/src/main/java/org/apache/druid/query/search/SearchQueryQueryToolChest.java
index 88291845565..61f77e35696 100644
--- a/processing/src/main/java/org/apache/druid/query/search/SearchQueryQueryToolChest.java
+++ b/processing/src/main/java/org/apache/druid/query/search/SearchQueryQueryToolChest.java
@@ -69,6 +69,7 @@
};
private final SearchQueryConfig config;
+ @Deprecated
private final IntervalChunkingQueryRunnerDecorator intervalChunkingQueryRunnerDecorator;
private final SearchQueryMetricsFactory queryMetricsFactory;
diff --git a/processing/src/main/java/org/apache/druid/query/select/SelectQueryQueryToolChest.java b/processing/src/main/java/org/apache/druid/query/select/SelectQueryQueryToolChest.java
index 3d342e3100e..aeaf7fb44be 100644
--- a/processing/src/main/java/org/apache/druid/query/select/SelectQueryQueryToolChest.java
+++ b/processing/src/main/java/org/apache/druid/query/select/SelectQueryQueryToolChest.java
@@ -78,6 +78,7 @@
};
private final ObjectMapper jsonMapper;
+ @Deprecated
private final IntervalChunkingQueryRunnerDecorator intervalChunkingQueryRunnerDecorator;
private final SelectQueryMetricsFactory queryMetricsFactory;
diff --git a/processing/src/main/java/org/apache/druid/query/timeseries/DefaultTimeseriesQueryMetricsFactory.java b/processing/src/main/java/org/apache/druid/query/timeseries/DefaultTimeseriesQueryMetricsFactory.java
index ed236a6ccbf..12124ebfae3 100644
--- a/processing/src/main/java/org/apache/druid/query/timeseries/DefaultTimeseriesQueryMetricsFactory.java
+++ b/processing/src/main/java/org/apache/druid/query/timeseries/DefaultTimeseriesQueryMetricsFactory.java
@@ -34,7 +34,7 @@
/**
* Should be used only in tests, directly or indirectly (via {@link
- * TimeseriesQueryQueryToolChest#TimeseriesQueryQueryToolChest(org.apache.druid.query.IntervalChunkingQueryRunnerDecorator)}).
+ * TimeseriesQueryQueryToolChest#TimeseriesQueryQueryToolChest}).
*/
@VisibleForTesting
public static TimeseriesQueryMetricsFactory instance()
diff --git a/processing/src/main/java/org/apache/druid/query/timeseries/TimeseriesQueryQueryToolChest.java b/processing/src/main/java/org/apache/druid/query/timeseries/TimeseriesQueryQueryToolChest.java
index 39b462c02fa..479fbf0c2d1 100644
--- a/processing/src/main/java/org/apache/druid/query/timeseries/TimeseriesQueryQueryToolChest.java
+++ b/processing/src/main/java/org/apache/druid/query/timeseries/TimeseriesQueryQueryToolChest.java
@@ -73,6 +73,7 @@
{
};
+ @Deprecated
private final IntervalChunkingQueryRunnerDecorator intervalChunkingQueryRunnerDecorator;
private final TimeseriesQueryMetricsFactory queryMetricsFactory;
diff --git a/processing/src/main/java/org/apache/druid/query/topn/TopNQueryQueryToolChest.java b/processing/src/main/java/org/apache/druid/query/topn/TopNQueryQueryToolChest.java
index a51521e1357..4774c5c9b1a 100644
--- a/processing/src/main/java/org/apache/druid/query/topn/TopNQueryQueryToolChest.java
+++ b/processing/src/main/java/org/apache/druid/query/topn/TopNQueryQueryToolChest.java
@@ -70,6 +70,7 @@
};
private final TopNQueryConfig config;
+ @Deprecated
private final IntervalChunkingQueryRunnerDecorator intervalChunkingQueryRunnerDecorator;
private final TopNQueryMetricsFactory queryMetricsFactory;
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org