You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@griffin.apache.org by Karan Gupta <ka...@tavant.com> on 2018/05/21 06:48:52 UTC

Apache Griffin Mappings For Profiling

Hi Lionel,

I created a job for profiling the data but I cannot see the job populating on the DQ Metrics. Seems like the same issue I ran into for accuracy. Could you send me the profiling mappings?
Also would the "api": "<E.S::PORT>/griffin/accuracy" be changed to "api": "<E.S::PORT>/griffin/profiling" in the env.json?

Thank you,
Karan Gupta
________________________________
Any comments or statements made in this email are not necessarily those of Tavant Technologies. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you have received this in error, please contact the sender and delete the material from any computer. All emails sent from or to Tavant Technologies may be subject to our monitoring procedures.

RE: Apache Griffin Mappings For Profiling

Posted by Karan Gupta <ka...@tavant.com>.
Hi Lionel,

Sorry for the inconvenience, I can now see the tabular representation of the profiling.

Thank you,
Karan Gupta

-----Original Message-----
From: Lionel, Liu <bh...@163.com> 
Sent: Monday, May 21, 2018 6:50 PM
To: dev@griffin.incubator.apache.org; Karan Gupta <ka...@tavant.com>
Subject: RE: Apache Griffin Mappings For Profiling

Hi Karan,

The profiling metrics seems to be more complex, which might be like this:
{
          "name" : "prof1",
          "tmst" : 1524812400000,
          "value" : {
            "count" : 50000,
            "null_count" : 500
          }
}
or like this:
{
          "name" : "prof2",
          "tmst" : 1524812400000,
          "value" : {
            "count" : 50000,
            "enum-count" : [
              {
                "enum" : "red",
                "count" : 28600
              },
              {
                "enum" : "yellow",
                "count" : 20400
              }
            ]
          }
}

I really don't know how to get the common mapping schema of "value" field, it depends on the rule you defined.

We don't need to define the mapping schema, it will create the mappings by the first value posted by default. But in your environment ES seems not working in such way, what's the ES version you are using?

Thanks
Lionel, Liu

From: Karan Gupta
Sent: 2018年5月21日 14:49
To: Lionel Liu; dev@griffin.incubator.apache.org
Subject: Apache Griffin Mappings For Profiling

Hi Lionel,

I created a job for profiling the data but I cannot see the job populating on the DQ Metrics. Seems like the same issue I ran into for accuracy. Could you send me the profiling mappings?
Also would the "api": "<E.S::PORT>/griffin/accuracy" be changed to "api": "<E.S::PORT>/griffin/profiling" in the env.json?

Thank you,
Karan Gupta
________________________________
Any comments or statements made in this email are not necessarily those of Tavant Technologies. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you have received this in error, please contact the sender and delete the material from any computer. All emails sent from or to Tavant Technologies may be subject to our monitoring procedures.

RE: Apache Griffin Mappings For Profiling

Posted by "Lionel, Liu" <bh...@163.com>.
Hi Karan,

The profiling metrics seems to be more complex, which might be like this:
{
          "name" : "prof1",
          "tmst" : 1524812400000,
          "value" : {
            "count" : 50000,
            "null_count" : 500
          }
}
or like this:
{
          "name" : "prof2",
          "tmst" : 1524812400000,
          "value" : {
            "count" : 50000,
            "enum-count" : [
              {
                "enum" : "red",
                "count" : 28600
              },
              {
                "enum" : "yellow",
                "count" : 20400
              }
            ]
          }
}

I really don't know how to get the common mapping schema of "value" field, it depends on the rule you defined.

We don't need to define the mapping schema, it will create the mappings by the first value posted by default. But in your environment ES seems not working in such way, what's the ES version you are using?

Thanks
Lionel, Liu

From: Karan Gupta
Sent: 2018年5月21日 14:49
To: Lionel Liu; dev@griffin.incubator.apache.org
Subject: Apache Griffin Mappings For Profiling

Hi Lionel,

I created a job for profiling the data but I cannot see the job populating on the DQ Metrics. Seems like the same issue I ran into for accuracy. Could you send me the profiling mappings?
Also would the "api": "<E.S::PORT>/griffin/accuracy" be changed to "api": "<E.S::PORT>/griffin/profiling" in the env.json?

Thank you,
Karan Gupta
________________________________
Any comments or statements made in this email are not necessarily those of Tavant Technologies. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you have received this in error, please contact the sender and delete the material from any computer. All emails sent from or to Tavant Technologies may be subject to our monitoring procedures.