You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "clintropolis (via GitHub)" <gi...@apache.org> on 2023/04/12 10:55:03 UTC
[GitHub] [druid] clintropolis commented on a diff in pull request #14065: docs: add docs for schema auto-discovery
clintropolis commented on code in PR #14065:
URL: https://github.com/apache/druid/pull/14065#discussion_r1163963390
##########
docs/ingestion/ingestion-spec.md:
##########
@@ -212,7 +241,7 @@ A `dimensionsSpec` can have the following components:
| `dimensions` | A list of [dimension names or objects](#dimension-objects). You cannot include the same column in both `dimensions` and `dimensionExclusions`.<br /><br />If `dimensions` and `spatialDimensions` are both null or empty arrays, Druid treats all columns other than timestamp or metrics that do not appear in `dimensionExclusions` as String-typed dimension columns. See [inclusions and exclusions](#inclusions-and-exclusions) for details.<br /><br />As a best practice, put the most frequently filtered dimensions at the beginning of the dimensions list. In this case, it would also be good to consider [`partitioning`](partitioning.md) by those same dimensions. | `[]` |
| `dimensionExclusions` | The names of dimensions to exclude from ingestion. Only names are supported here, not objects.<br /><br />This list is only used if the `dimensions` and `spatialDimensions` lists are both null or empty arrays; otherwise it is ignored. See [inclusions and exclusions](#inclusions-and-exclusions) below for details. | `[]` |
| `spatialDimensions` | An array of [spatial dimensions](../development/geo.md). | `[]` |
-| `includeAllDimensions` | You can set `includeAllDimensions` to true to ingest both explicit dimensions in the `dimensions` field and other dimensions that the ingestion task discovers from input data. In this case, the explicit dimensions will appear first in order that you specify them and the dimensions dynamically discovered will come after. This flag can be useful especially with auto schema discovery using [`flattenSpec`](./data-formats.html#flattenspec). If this is not set and the `dimensions` field is not empty, Druid will ingest only explicit dimensions. If this is not set and the `dimensions` field is empty, all discovered dimensions will be ingested. | false |
Review Comment:
this unfortunately hasn't actually been removed yet. I do want to consolidate it into `useSchemaDiscovery`, but haven't yet
##########
docs/ingestion/ingestion-spec.md:
##########
@@ -200,6 +212,23 @@ configuring [dimensions](./data-model.md#dimensions). An example `dimensionsSpec
}
```
+<!--Schema auto-discovery-->
+
+```json
+ "tuningConfig": {
+ ...
+ "appendableIndexSpec": {
+ ...,
+ "useSchemaDiscovery": true
+ },
+ },
Review Comment:
oops, the old PR descriptions are stale, `useSchemaDiscovery` is on `dimensionsSpec` these days
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org