You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@griffin.apache.org by Dmitry Ershov <de...@gmail.com> on 2019/02/19 16:45:27 UTC

ElasticSearchSink modification question

Hi all,

We are considering to use Griffin in our project and facing with a problem,
how to retrieve measure results for specific measure in ElasticSearch.
Actually ElasticSearchSink does not save the information about yarn
applicationId. We propose to add this applicationId into measure results.
The part of response will be looking like that:
"_source": {
                    "name": "de_demo_results_comparision",
                    "tmst": 1548937782703,
                    "value": {
                        "total": 74,
                        "miss": 74,
                        "matched": 0,
                        "matchedFraction": 0
                    },
                    "applicationId": "application_1548923518014_0001"
                }

After that we can find a specific measure information basing on this Id. In
this way we could include our griffin jobs in various ETL processes.

What does community think about such kind of improvement?

Thanks in advance!

RE: ElasticSearchSink modification question

Posted by "Lionel, Liu" <bh...@163.com>.
Yes, that’s a good feature. 
I think you can add it in all the sink types.

Thanks
Lionel, Liu

From: Nick Sokolov
Sent: 2019年2月20日 1:11
To: dev@griffin.apache.org
Subject: Re: ElasticSearchSink modification question

Sounds interesting!
Looks like it also allows to implement API to retrieve metric values for
particular job instance (by correlating results using this id).

On Tue, Feb 19, 2019 at 8:45 AM Dmitry Ershov <de...@gmail.com> wrote:

> Hi all,
>
> We are considering to use Griffin in our project and facing with a problem,
> how to retrieve measure results for specific measure in ElasticSearch.
> Actually ElasticSearchSink does not save the information about yarn
> applicationId. We propose to add this applicationId into measure results.
> The part of response will be looking like that:
> "_source": {
>                     "name": "de_demo_results_comparision",
>                     "tmst": 1548937782703,
>                     "value": {
>                         "total": 74,
>                         "miss": 74,
>                         "matched": 0,
>                         "matchedFraction": 0
>                     },
>                     "applicationId": "application_1548923518014_0001"
>                 }
>
> After that we can find a specific measure information basing on this Id. In
> this way we could include our griffin jobs in various ETL processes.
>
> What does community think about such kind of improvement?
>
> Thanks in advance!
>

Re: ElasticSearchSink modification question

Posted by Nick Sokolov <ch...@gmail.com>.
Sounds interesting!
Looks like it also allows to implement API to retrieve metric values for
particular job instance (by correlating results using this id).

On Tue, Feb 19, 2019 at 8:45 AM Dmitry Ershov <de...@gmail.com> wrote:

> Hi all,
>
> We are considering to use Griffin in our project and facing with a problem,
> how to retrieve measure results for specific measure in ElasticSearch.
> Actually ElasticSearchSink does not save the information about yarn
> applicationId. We propose to add this applicationId into measure results.
> The part of response will be looking like that:
> "_source": {
>                     "name": "de_demo_results_comparision",
>                     "tmst": 1548937782703,
>                     "value": {
>                         "total": 74,
>                         "miss": 74,
>                         "matched": 0,
>                         "matchedFraction": 0
>                     },
>                     "applicationId": "application_1548923518014_0001"
>                 }
>
> After that we can find a specific measure information basing on this Id. In
> this way we could include our griffin jobs in various ETL processes.
>
> What does community think about such kind of improvement?
>
> Thanks in advance!
>