You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/03/31 02:08:55 UTC
[GitHub] [druid] suneet-s opened a new issue #9591: TransformSpec for
metrics when re-indexing a datasource using ingestSegment appears to be
broken
suneet-s opened a new issue #9591: TransformSpec for metrics when re-indexing a datasource using ingestSegment appears to be broken
URL: https://github.com/apache/druid/issues/9591
### Affected Version
0.18
### Description
The following re-index spec spits out 0s for both "**double-deleted**" a transformation on a column "deleted" and "**one-plus-triple-deleted**" triple-deleted is a metric calculated from the original index job's transformSpec
```
{
"type": "index",
"spec": {
"dataSchema": {
"dataSource": "reindex-double-tfn-1",
"metricsSpec": [
{
"type": "doubleSum",
"name": "added",
"fieldName": "added"
},
{
"type": "doubleSum",
"name": "triple-added",
"fieldName": "triple-added"
},
{
"type": "doubleSum",
"name": "one-plus-triple-added",
"fieldName": "one-plus-triple-added"
},
{
"type": "doubleSum",
"name": "deleted",
"fieldName": "deleted"
},
{
"type": "doubleSum",
"name": "double-deleted",
"fieldName": "double-deleted"
},
{
"type": "doubleSum",
"name": "delta",
"fieldName": "delta"
}
],
"granularitySpec": {
"segmentGranularity": "DAY",
"queryGranularity": "second",
"intervals" : [ "2013-08-31/2013-09-01" ]
},
"parser": {
"parseSpec": {
"format" : "json",
"timestampSpec": {
"column": "timestamp",
"format": "iso"
},
"dimensionsSpec": {
"dimensions": [
"page",
{"type": "string", "name": "language", "createBitmapIndex": false},
"user",
"unpatrolled",
"newPage",
"anonymous",
"namespace",
"country",
"region",
"city"
]
},
"transformSpec": {
"transforms": [
{
"type": "expression",
"name": "newPage",
"expression": "page"
},
{
"type": "expression",
"name": "one-plus-triple-added",
"expression": "triple-added + 1"
},
{
"type": "expression",
"name": "double-deleted",
"expression": "deleted * 2"
}
]
}
}
}
},
"ioConfig": {
"type": "index",
"firehose": {
"type": "ingestSegment",
"dataSource": "wiki-tests-2",
"interval": "2013-08-31/2013-09-01"
}
},
"tuningConfig": {
"type": "index"
}
}
}
```
The ingest spec for wiki-tests-2 is the same as the one described in #9589
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] 2bethere commented on issue #9591: TransformSpec for
metrics when re-indexing a datasource using ingestSegment appears to be
broken
Posted by GitBox <gi...@apache.org>.
2bethere commented on issue #9591: TransformSpec for metrics when re-indexing a datasource using ingestSegment appears to be broken
URL: https://github.com/apache/druid/issues/9591#issuecomment-606942203
What's the expected output?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s commented on issue #9591: TransformSpec for
metrics when re-indexing a datasource using ingestSegment appears to be
broken
Posted by GitBox <gi...@apache.org>.
suneet-s commented on issue #9591: TransformSpec for metrics when re-indexing a datasource using ingestSegment appears to be broken
URL: https://github.com/apache/druid/issues/9591#issuecomment-606356941
This works if you use a DruidInputSource instead of the ingestSegmentFirehose
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org