You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Yi Pan <ni...@gmail.com> on 2018/02/02 01:20:30 UTC
Re: No Metrics Emitted in new High-Level API
Hi, Srujun,
Sorry to reply late. I am glad that you figured it out. The issue you
reported seems to be a bug that we set the default for metrics serde in one
case and not in the other one. I will open a ticket to track the
investigation.
Thanks a lot for reporting the issue!
-Yi
On Tue, Jan 30, 2018 at 2:38 PM, Srujun Thanmay Gupta <sgupta80@illinois.edu
> wrote:
> Hi,
>
> Thank you Yi, what you mentioned seemed to be the issue. Configuring the
> metrics serdes fixed the metrics output. The reason I had not added them
> initially was because the wikipedia-application.properties file in the
> hello-samza example here: (
> https://github.com/apache/samza-hello-samza/blob/master/
> src/main/config/wikipedia-application.properties
> )
> does not have them configured.
>
> Using that configuration for the wikipedia-application demo did not produce
> metrics. But, surprisingly, not configuring the serdes for the
> wikipedia-feed demo either (which uses the low-level API) worked fine,
> which is why I was wondering if metrics is configured differently when
> using the high-level API. I've linked the two demos below, with the
> .properties files used for each and the DEBUG logs generated.
>
> https://github.com/srujun/srujun.github.io/tree/master/
> files/samza/wikipedia-feed
> (metrics works as expected, emitting JSON-encoded messages to
> kafka.metrics)
> https://github.com/srujun/srujun.github.io/tree/master/
> files/samza/wikipedia-application
> (metrics not working unless serdes are also configured)
>
> From what I can see, there are metrics serialization errors in the
> wikipedia-application logs, as expected. But no such issue for
> wikipedia-feed, even though no serde was configured. Is there some other
> issue here?
>
> Thank you again.
>
> On Fri, Jan 26, 2018 at 11:00 PM, Yi Pan <ni...@gmail.com> wrote:
>
> > Hi, Srujun,
> >
> > I noticed that you did not configure the metrics serde as stated here:
> > http://samza.apache.org/learn/documentation/0.14/container/metrics.html.
> > Could that be the reason?
> >
> > -Yi
> >
> > On Fri, Jan 26, 2018 at 7:58 PM, Jagadish Venkatraman <
> > jagadish1989@gmail.com> wrote:
> >
> > > Can you turn on DEBUG logging for the MetricsSnapshotReporter class and
> > > re-attach the logs? I cannot observe any obvious red-flag from your
> logs.
> > >
> > > On Fri, Jan 26, 2018 at 7:13 PM, Srujun Thanmay Gupta <
> > > sgupta80@illinois.edu
> > > > wrote:
> > >
> > > > Hello Yi,
> > > >
> > > > The logs are less than 100KB each, I'm not sure if that's too large
> for
> > > the
> > > > mailing list system. I've uploaded the log files to the links below,
> in
> > > > case others would like to look at them too.
> > > >
> > > > http://srujun.com/files/samza/wikipedia-application_samza-
> > > container-0.log
> > > > http://srujun.com/files/samza/wikipedia-application_samza-
> > > > job-coordinator.log
> > > > http://srujun.com/files/samza/wikipedia-feed_samza-container-0.log
> > > > http://srujun.com/files/samza/wikipedia-feed_samza-job-
> coordinator.log
> > > >
> > > > Thank you for your help!
> > > >
> > > > --
> > > > Srujun Thanmay Gupta
> > > >
> > >
> > >
> > >
> > > --
> > > Jagadish V,
> > > Graduate Student,
> > > Department of Computer Science,
> > > Stanford University
> > >
> >
>
>
>
> --
> Srujun Thanmay Gupta
>
Re: No Metrics Emitted in new High-Level API
Posted by Yi Pan <ni...@gmail.com>.
SAMZA-1581 is the ticket
On Thu, Feb 1, 2018 at 5:20 PM, Yi Pan <ni...@gmail.com> wrote:
> Hi, Srujun,
>
> Sorry to reply late. I am glad that you figured it out. The issue you
> reported seems to be a bug that we set the default for metrics serde in one
> case and not in the other one. I will open a ticket to track the
> investigation.
>
> Thanks a lot for reporting the issue!
>
> -Yi
>
> On Tue, Jan 30, 2018 at 2:38 PM, Srujun Thanmay Gupta <
> sgupta80@illinois.edu> wrote:
>
>> Hi,
>>
>> Thank you Yi, what you mentioned seemed to be the issue. Configuring the
>> metrics serdes fixed the metrics output. The reason I had not added them
>> initially was because the wikipedia-application.properties file in the
>> hello-samza example here: (
>> https://github.com/apache/samza-hello-samza/blob/master/src/
>> main/config/wikipedia-application.properties
>> )
>> does not have them configured.
>>
>> Using that configuration for the wikipedia-application demo did not
>> produce
>> metrics. But, surprisingly, not configuring the serdes for the
>> wikipedia-feed demo either (which uses the low-level API) worked fine,
>> which is why I was wondering if metrics is configured differently when
>> using the high-level API. I've linked the two demos below, with the
>> .properties files used for each and the DEBUG logs generated.
>>
>> https://github.com/srujun/srujun.github.io/tree/master/files
>> /samza/wikipedia-feed
>> (metrics works as expected, emitting JSON-encoded messages to
>> kafka.metrics)
>> https://github.com/srujun/srujun.github.io/tree/master/files
>> /samza/wikipedia-application
>> (metrics not working unless serdes are also configured)
>>
>> From what I can see, there are metrics serialization errors in the
>> wikipedia-application logs, as expected. But no such issue for
>> wikipedia-feed, even though no serde was configured. Is there some other
>> issue here?
>>
>> Thank you again.
>>
>> On Fri, Jan 26, 2018 at 11:00 PM, Yi Pan <ni...@gmail.com> wrote:
>>
>> > Hi, Srujun,
>> >
>> > I noticed that you did not configure the metrics serde as stated here:
>> > http://samza.apache.org/learn/documentation/0.14/container/metrics.html
>> .
>> > Could that be the reason?
>> >
>> > -Yi
>> >
>> > On Fri, Jan 26, 2018 at 7:58 PM, Jagadish Venkatraman <
>> > jagadish1989@gmail.com> wrote:
>> >
>> > > Can you turn on DEBUG logging for the MetricsSnapshotReporter class
>> and
>> > > re-attach the logs? I cannot observe any obvious red-flag from your
>> logs.
>> > >
>> > > On Fri, Jan 26, 2018 at 7:13 PM, Srujun Thanmay Gupta <
>> > > sgupta80@illinois.edu
>> > > > wrote:
>> > >
>> > > > Hello Yi,
>> > > >
>> > > > The logs are less than 100KB each, I'm not sure if that's too large
>> for
>> > > the
>> > > > mailing list system. I've uploaded the log files to the links
>> below, in
>> > > > case others would like to look at them too.
>> > > >
>> > > > http://srujun.com/files/samza/wikipedia-application_samza-
>> > > container-0.log
>> > > > http://srujun.com/files/samza/wikipedia-application_samza-
>> > > > job-coordinator.log
>> > > > http://srujun.com/files/samza/wikipedia-feed_samza-container-0.log
>> > > > http://srujun.com/files/samza/wikipedia-feed_samza-job-coord
>> inator.log
>> > > >
>> > > > Thank you for your help!
>> > > >
>> > > > --
>> > > > Srujun Thanmay Gupta
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Jagadish V,
>> > > Graduate Student,
>> > > Department of Computer Science,
>> > > Stanford University
>> > >
>> >
>>
>>
>>
>> --
>> Srujun Thanmay Gupta
>>
>
>