[jira] [Updated] (GRIFFIN-313) No source information provided into metric output

["Tushar (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GRIFFIN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tushar updated GRIFFIN-313:
---------------------------
    Description: Users can provide multiple sources in Griffin config and apply a different constraint on each of the sources. But there are no details coming metric output that which constraint is applied on which source.  We should have source information in output to know about which constraint run on which source.  (was: Users can provide multiple sources in Griffin config and apply a different constraint on each of the sources. But there are no details coming metric output that which constraint is applied on which source. Please find below input config,

{{ "name": "accu_batch",
 "process.type": "batch",
 "data.sources": [\{ "name": "source", "baseline": true, "connectors": [{ "type": "avro", "version": "1.7", "config": { "file.name": "/griffin-master/measure/src/test/resources/users_info_src/users_info_src.avro" } }] }, \{ "name": "source_tgt", "baseline": true, "connectors": [{ "type": "avro", "version": "1.7", "config": { "file.name": "/griffin-master/measure/src/test/resources/users_info_src/users_info_src.avro" } }] }
 ],
 "evaluate.rule": \{ "rules": [{ "dsl.type": "griffin-dsl", "dq.type": "completeness", "out.dataframe.name": "comp", "rule": "email, post_code, first_name", "details": { "source": "source" }, "out": [\{ "type": "metric", "name": "completeness", "flatten": "map" }] }, \{ "dsl.type": "griffin-dsl", "dq.type": "completeness", "out.dataframe.name": "comp", "rule": "email, post_code, first_name", "details": { "source": "source_tgt" }, "out": [\{ "type": "metric", "name": "completeness_tgt", "flatten": "map" }] } ] }, "sinks": ["CONSOLE", "HDFS"]}

 

Output Json file :

{\{ "name": "accu_batch", "tmst": 1576833946344, "value": { "completeness": [{ "total": 50, "incomplete": 1, "complete": 49 }], "completeness_tgt": [\{ "total": 50, "incomplete": 1, "complete": 49 }] }, "metadata": \{ "applicationId": "local-1576833941125" }})

> No source information provided into metric output
> -------------------------------------------------
>
>                 Key: GRIFFIN-313
>                 URL: https://issues.apache.org/jira/browse/GRIFFIN-313
>             Project: Griffin
>          Issue Type: Improvement
>          Components: accuracy-batch, completeness-batch, timeliness-batch
>            Reporter: Tushar
>            Priority: Major
>
> Users can provide multiple sources in Griffin config and apply a different constraint on each of the sources. But there are no details coming metric output that which constraint is applied on which source.  We should have source information in output to know about which constraint run on which source.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)