Metrics of a streaming job

[Vikram Jain <vi...@enquero.com>]

Hello,
I’m trying my hands on Griffin and able to successfully deploy it on Docker.
The UI and REST APIs seems to work smoothly for the batch jobs but I’m facing issues running streaming jobs.
As per my understanding, metrics of streaming jobs are also stored in ElasticSearch. However, when I look at ES after streaming job execution starts, I don’t see any metrics being stored there.

Please help me understand where the metrics of streaming job are stored and how to retrive them. I am using the sample database and instructions as present in Apache Griffin Docker guide : https://github.com/apache/incubator-griffin/blob/master/griffin-doc/docker/griffin-docker-guide.md


Thanks & Regards,
Vikram Jain


Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

Re: Metrics of a streaming job

[Lionel Liu <li...@apache.org>]

Hi Vikram,

Seems you're following the guide for streaming mode:
https://github.com/apache/incubator-griffin/blob/master/griffin-doc/docker/griffin-docker-guide.md#how-to-use-griffin-docker-images-in-streaming-mode
At current, UI and service doesn't support streaming measure create and
submit. You can only submit the spark application manually via shell, as
the step in the previous guide.
By tracing the log, you can get the status of calculation, when the result
of each mini-batch printed in log, it should also be posted to
elasticsearch, after then you can be able to retrieve the metrics from ES.

Thanks,
Lionel


On Fri, Aug 10, 2018 at 5:52 PM Vikram Jain <vi...@enquero.com> wrote:

> Hello,
> I’m trying my hands on Griffin and able to successfully deploy it on
> Docker.
> The UI and REST APIs seems to work smoothly for the batch jobs but I’m
> facing issues running streaming jobs.
> As per my understanding, metrics of streaming jobs are also stored in
> ElasticSearch. However, when I look at ES after streaming job execution
> starts, I don’t see any metrics being stored there.
>
> Please help me understand where the metrics of streaming job are stored
> and how to retrive them. I am using the sample database and instructions as
> present in Apache Griffin Docker guide :
> https://github.com/apache/incubator-griffin/blob/master/griffin-doc/docker/griffin-docker-guide.md
>
>
> Thanks & Regards,
> Vikram Jain
>
>
> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>