You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Vaibhav Garg <va...@gmail.com> on 2020/02/26 11:14:33 UTC
Samza - New Relic integration
Hi,
I want to integrate New Relic in my Samza jobs. Can anyone help with this,
please?
Thanks,
Vaibhav Garg
+91-9505020924
vaibhavgarg90@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
Re: Samza - New Relic integration
Posted by Jagadish Venkatraman <ja...@gmail.com>.
That's great to hear Vaibhav. Feel free to reach out to the community if
you have further questions!
Best
Jagadish
On Sunday, March 1, 2020, Vaibhav Garg <va...@gmail.com> wrote:
> Ok. That worked for me.
>
> Thank you, Bharath, Yi Pan, and Jagadish for your help.
>
> Thanks,
> Vaibhav Garg
> +91-9505020924
> vaibhavgarg90@gmail.com
> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>
>
> On Sun, Mar 1, 2020 at 12:56 PM Vaibhav Garg <va...@gmail.com>
> wrote:
>
> > Ok. I will try that.
> >
> > Thanks,
> > Vaibhav Garg
> > +91-9505020924
> > vaibhavgarg90@gmail.com
> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >
> >
> > On Sun, Mar 1, 2020 at 12:31 PM Jagadish Venkatraman <
> > jagadish1989@gmail.com> wrote:
> >
> >> Hi Vaibhav,
> >>
> >> Can you try ./__package/newrelic/newrelic.jar?
> >>
> >> If that doesn't work, I'd recommend doing the following:
> >> 1. Print the current working directory of your Samza container JVM
> >> 2. Use (1) to determine a relative reference to the directory path
> within
> >> your package
> >>
> >> Cheers
> >> Jagadish
> >>
> >> On Saturday, February 29, 2020, Vaibhav Garg <va...@gmail.com>
> >> wrote:
> >>
> >> > I would be really grateful if someone could help me with this.
> >> >
> >> > Thanks,
> >> > Vaibhav Garg
> >> > +91-9505020924
> >> > vaibhavgarg90@gmail.com
> >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> >
> >> >
> >> > On Sat, Feb 29, 2020 at 9:02 PM Vaibhav Garg <vaibhavgarg90@gmail.com
> >
> >> > wrote:
> >> >
> >> > > Please do let me know if some other clarification is needed.
> >> > >
> >> > > Thanks,
> >> > > Vaibhav Garg
> >> > > +91-9505020924
> >> > > vaibhavgarg90@gmail.com
> >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> > >
> >> > >
> >> > > On Sat, Feb 29, 2020 at 10:31 AM Vaibhav Garg <
> >> vaibhavgarg90@gmail.com>
> >> > > wrote:
> >> > >
> >> > >> OK. Here is what I have inside my configuration in order to start
> >> > >> NewRelic at the time of the container start:
> >> > >>
> >> > >> task.opts=-Xmx2048m -XX:+UseConcMarkSweepGC
> >> > >> *-javaagent:/usr/home/vaibhav/newrelic/newrelic.jar*
> >> > >>
> >> > >> Now, I have specified the absolute path of the NewRelic jar. There
> >> are 2
> >> > >> challenges that I see with this approach:
> >> > >>
> >> > >> 1. The NewRelic jar and config would have to be present on all the
> >> Yarn
> >> > >> slave nodes since that container can start at any of the Yarn
> nodes.
> >> > >> 2. A single Yarn cluster can be used to run multiple Samza
> >> applications,
> >> > >> so I would have to keep configuration for each application at each
> of
> >> > the
> >> > >> Yarn slave nodes.
> >> > >>
> >> > >> In order to fix this, I am bundling NewRelic inside my tar file:
> >> > >>
> >> > >> // download new relic dependencies
> >> > >> task downloadNewrelic(type: Download) {
> >> > >> mkdir "newrelic"
> >> > >> src "
> >> > >> https://download.newrelic.com/newrelic/java-agent/newrelic-
> >> > agent/current/newrelic-java.zip
> >> > >> "
> >> > >> dest file("newrelic")
> >> > >> }
> >> > >>
> >> > >> task buildTar(dependsOn: [build, downloadNewrelic], type: Tar) {
> >> > >> // unzip new relic dependencies and add to the archive
> >> > >> into(".") {
> >> > >> from zipTree(file("newrelic/newrelic-java.zip"))
> >> > >> }
> >> > >> }
> >> > >>
> >> > >> With this build configuration, I have an additional *newrelic*
> >> directory
> >> > >> inside my archive. Here is the directory structure:
> >> > >> my-job-name.tar.gz
> >> > >> bin
> >> > >> config
> >> > >> lib
> >> > >> newrelic
> >> > >> newrelic.jar
> >> > >> newrelic.yml
> >> > >> ...
> >> > >>
> >> > >> What I would like to achieve is to:
> >> > >> - Remove hardcoding the *javaagent* inside *task.opts* in the
> >> container
> >> > >> config.
> >> > >> - Specify the path of NewRelic agent in *task.opts* relative to
> where
> >> > >> the container files are present on the Yarn slave node.
> >> > >>
> >> > >> I hope I was clearer this time. Please let me know if any other
> >> > >> clarification is needed.
> >> > >>
> >> > >> Thanks,
> >> > >> Vaibhav Garg
> >> > >> +91-9505020924
> >> > >> vaibhavgarg90@gmail.com
> >> > >> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> > >>
> >> > >>
> >> > >> On Sat, Feb 29, 2020 at 5:22 AM Yi Pan <ni...@gmail.com>
> wrote:
> >> > >>
> >> > >>> Hi, Vaibhav,
> >> > >>>
> >> > >>> Not quite sure whether I understand your ask. How do you package
> and
> >> > >>> distribute NewRelic jar in the YARN cluster? It would be much
> >> helpful
> >> > if
> >> > >>> you can share your job configure that worked and the places that
> you
> >> > >>> would
> >> > >>> like to change.
> >> > >>>
> >> > >>> Best,
> >> > >>>
> >> > >>> -Yi
> >> > >>>
> >> > >>> On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg <
> >> vaibhavgarg90@gmail.com
> >> > >
> >> > >>> wrote:
> >> > >>>
> >> > >>> > OK. I have been able to use NewRelic by specifying -javaagent
> >> option
> >> > in
> >> > >>> > task.opts.
> >> > >>> >
> >> > >>> > However, I had to give the absolute path to the NewRelic jar
> file
> >> in
> >> > >>> order
> >> > >>> > to make it work.
> >> > >>> >
> >> > >>> > How can I give a path that is relative to the container since I
> >> would
> >> > >>> like
> >> > >>> > to integrate NewRelic dependencies along with other container
> >> files?
> >> > >>> >
> >> > >>> > Thanks,
> >> > >>> > Vaibhav Garg
> >> > >>> > +91-9505020924
> >> > >>> > vaibhavgarg90@gmail.com
> >> > >>> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> > >>> >
> >> > >>> >
> >> > >>> > On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <
> >> > vaibhavgarg90@gmail.com
> >> > >>> >
> >> > >>> > wrote:
> >> > >>> >
> >> > >>> > > Yes. That's exactly what I am looking for.
> >> > >>> > >
> >> > >>> > > Let me try it out and I will get back to you if I face any
> >> issues.
> >> > >>> > >
> >> > >>> > > Thanks again
> >> > >>> > >
> >> > >>> > > Vaibhav Garg
> >> > >>> > > +91-9505020924
> >> > >>> > > vaibhavgarg90@gmail.com
> >> > >>> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> > >>> > >
> >> > >>> > >
> >> > >>> > > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <ni...@gmail.com>
> >> > wrote:
> >> > >>> > >
> >> > >>> > >> Hi, Vaibhav,
> >> > >>> > >>
> >> > >>> > >> Check the description of task.opts in the configuration doc
> >> here:
> >> > >>> > >>
> >> > >>> > >>
> >> > >>> >
> >> > >>> http://samza.apache.org/learn/documentation/latest/jobs/
> >> > samza-configurations.html
> >> > >>> > >>
> >> > >>> > >> Is this what you are looking for?
> >> > >>> > >>
> >> > >>> > >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <
> >> > >>> vaibhavgarg90@gmail.com>
> >> > >>> > >> wrote:
> >> > >>> > >>
> >> > >>> > >> > Hi,
> >> > >>> > >> >
> >> > >>> > >> > Any help here would be great.
> >> > >>> > >> >
> >> > >>> > >> > Thanks,
> >> > >>> > >> > Vaibhav Garg
> >> > >>> > >> > +91-9505020924
> >> > >>> > >> > vaibhavgarg90@gmail.com
> >> > >>> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> > >>> > >> >
> >> > >>> > >> >
> >> > >>> > >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <
> >> > >>> > vaibhavgarg90@gmail.com>
> >> > >>> > >> > wrote:
> >> > >>> > >> >
> >> > >>> > >> > > Dear Bharath,
> >> > >>> > >> > >
> >> > >>> > >> > > Thanks for your reply. I now realize that I put a very
> >> > ambiguous
> >> > >>> > >> question
> >> > >>> > >> > > to the community. Here is another attempt:
> >> > >>> > >> > >
> >> > >>> > >> > > I have set up a Yarn cluster that is configured to run
> >> > >>> containers of
> >> > >>> > >> > > multiple Samza jobs.
> >> > >>> > >> > >
> >> > >>> > >> > > I would like to treat all the containers of the job as a
> >> > single
> >> > >>> > >> > > application in New Relic (as expected).
> >> > >>> > >> > >
> >> > >>> > >> > > Since the containers of the application can be killed on
> >> one
> >> > >>> Yarn
> >> > >>> > node
> >> > >>> > >> > and
> >> > >>> > >> > > can start on another Yarn node, I would have to specify
> New
> >> > >>> Relic
> >> > >>> > >> > > environment settings at the container start.
> >> > >>> > >> > >
> >> > >>> > >> > > Now, I am not sure how to specify java agent (to include
> >> New
> >> > >>> Relic
> >> > >>> > >> java
> >> > >>> > >> > > agent) and pass additional arguments such as New Relic
> >> > >>> environment
> >> > >>> > >> every
> >> > >>> > >> > > time the container starts.
> >> > >>> > >> > >
> >> > >>> > >> > > Please let me know if my understanding is wrong or some
> >> other
> >> > >>> > >> > > clarification is needed.
> >> > >>> > >> > >
> >> > >>> > >> > > Thanks in advance,
> >> > >>> > >> > > Vaibhav Garg
> >> > >>> > >> > > +91-9505020924
> >> > >>> > >> > > vaibhavgarg90@gmail.com
> >> > >>> > >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> > >>> > >> > >
> >> > >>> > >> > >
> >> > >>> > >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara
> Subramanian
> >> <
> >> > >>> > >> > > codin.martial@gmail.com> wrote:
> >> > >>> > >> > >
> >> > >>> > >> > >> Hi,
> >> > >>> > >> > >>
> >> > >>> > >> > >> I am not sure I fully understand the ask.
> >> > >>> > >> > >>
> >> > >>> > >> > >> IIUC, Samza doesn't have native integration with New
> >> relic.
> >> > >>> > However,
> >> > >>> > >> you
> >> > >>> > >> > >> should still be able to integrate your application with
> >> New
> >> > >>> relic
> >> > >>> > on
> >> > >>> > >> > your
> >> > >>> > >> > >> end without native support.
> >> > >>> > >> > >> If you are particularly looking to integrate native
> Samza
> >> > >>> metrics
> >> > >>> > w/
> >> > >>> > >> New
> >> > >>> > >> > >> relic, you might need to implement your own custom
> metrics
> >> > >>> > reporter.
> >> > >>> > >> You
> >> > >>> > >> > >> can find more details here
> >> > >>> > >> > >> <
> >> > >>> > >> > >>
> >> > >>> > >> >
> >> > >>> > >>
> >> > >>> >
> >> > >>> https://samza.apache.org/learn/documentation/latest/
> >> > operations/monitoring.html#customreporter
> >> > >>> > >> > >> >
> >> > >>> > >> > >>
> >> > >>> > >> > >> Thanks,
> >> > >>> > >> > >> Bharath
> >> > >>> > >> > >>
> >> > >>> > >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <
> >> > >>> > >> vaibhavgarg90@gmail.com>
> >> > >>> > >> > >> wrote:
> >> > >>> > >> > >>
> >> > >>> > >> > >> > Hi,
> >> > >>> > >> > >> >
> >> > >>> > >> > >> > I want to integrate New Relic in my Samza jobs. Can
> >> anyone
> >> > >>> help
> >> > >>> > >> with
> >> > >>> > >> > >> this,
> >> > >>> > >> > >> > please?
> >> > >>> > >> > >> >
> >> > >>> > >> > >> > Thanks,
> >> > >>> > >> > >> > Vaibhav Garg
> >> > >>> > >> > >> > +91-9505020924
> >> > >>> > >> > >> > vaibhavgarg90@gmail.com
> >> > >>> > >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> > >>> > >> > >> >
> >> > >>> > >> > >>
> >> > >>> > >> > >
> >> > >>> > >> >
> >> > >>> > >>
> >> > >>> > >
> >> > >>> >
> >> > >>>
> >> > >>
> >> >
> >>
> >>
> >> --
> >> Jagadish
> >>
> >
>
--
Jagadish
Re: Samza - New Relic integration
Posted by Vaibhav Garg <va...@gmail.com>.
Ok. That worked for me.
Thank you, Bharath, Yi Pan, and Jagadish for your help.
Thanks,
Vaibhav Garg
+91-9505020924
vaibhavgarg90@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Sun, Mar 1, 2020 at 12:56 PM Vaibhav Garg <va...@gmail.com>
wrote:
> Ok. I will try that.
>
> Thanks,
> Vaibhav Garg
> +91-9505020924
> vaibhavgarg90@gmail.com
> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>
>
> On Sun, Mar 1, 2020 at 12:31 PM Jagadish Venkatraman <
> jagadish1989@gmail.com> wrote:
>
>> Hi Vaibhav,
>>
>> Can you try ./__package/newrelic/newrelic.jar?
>>
>> If that doesn't work, I'd recommend doing the following:
>> 1. Print the current working directory of your Samza container JVM
>> 2. Use (1) to determine a relative reference to the directory path within
>> your package
>>
>> Cheers
>> Jagadish
>>
>> On Saturday, February 29, 2020, Vaibhav Garg <va...@gmail.com>
>> wrote:
>>
>> > I would be really grateful if someone could help me with this.
>> >
>> > Thanks,
>> > Vaibhav Garg
>> > +91-9505020924
>> > vaibhavgarg90@gmail.com
>> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> >
>> >
>> > On Sat, Feb 29, 2020 at 9:02 PM Vaibhav Garg <va...@gmail.com>
>> > wrote:
>> >
>> > > Please do let me know if some other clarification is needed.
>> > >
>> > > Thanks,
>> > > Vaibhav Garg
>> > > +91-9505020924
>> > > vaibhavgarg90@gmail.com
>> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >
>> > >
>> > > On Sat, Feb 29, 2020 at 10:31 AM Vaibhav Garg <
>> vaibhavgarg90@gmail.com>
>> > > wrote:
>> > >
>> > >> OK. Here is what I have inside my configuration in order to start
>> > >> NewRelic at the time of the container start:
>> > >>
>> > >> task.opts=-Xmx2048m -XX:+UseConcMarkSweepGC
>> > >> *-javaagent:/usr/home/vaibhav/newrelic/newrelic.jar*
>> > >>
>> > >> Now, I have specified the absolute path of the NewRelic jar. There
>> are 2
>> > >> challenges that I see with this approach:
>> > >>
>> > >> 1. The NewRelic jar and config would have to be present on all the
>> Yarn
>> > >> slave nodes since that container can start at any of the Yarn nodes.
>> > >> 2. A single Yarn cluster can be used to run multiple Samza
>> applications,
>> > >> so I would have to keep configuration for each application at each of
>> > the
>> > >> Yarn slave nodes.
>> > >>
>> > >> In order to fix this, I am bundling NewRelic inside my tar file:
>> > >>
>> > >> // download new relic dependencies
>> > >> task downloadNewrelic(type: Download) {
>> > >> mkdir "newrelic"
>> > >> src "
>> > >> https://download.newrelic.com/newrelic/java-agent/newrelic-
>> > agent/current/newrelic-java.zip
>> > >> "
>> > >> dest file("newrelic")
>> > >> }
>> > >>
>> > >> task buildTar(dependsOn: [build, downloadNewrelic], type: Tar) {
>> > >> // unzip new relic dependencies and add to the archive
>> > >> into(".") {
>> > >> from zipTree(file("newrelic/newrelic-java.zip"))
>> > >> }
>> > >> }
>> > >>
>> > >> With this build configuration, I have an additional *newrelic*
>> directory
>> > >> inside my archive. Here is the directory structure:
>> > >> my-job-name.tar.gz
>> > >> bin
>> > >> config
>> > >> lib
>> > >> newrelic
>> > >> newrelic.jar
>> > >> newrelic.yml
>> > >> ...
>> > >>
>> > >> What I would like to achieve is to:
>> > >> - Remove hardcoding the *javaagent* inside *task.opts* in the
>> container
>> > >> config.
>> > >> - Specify the path of NewRelic agent in *task.opts* relative to where
>> > >> the container files are present on the Yarn slave node.
>> > >>
>> > >> I hope I was clearer this time. Please let me know if any other
>> > >> clarification is needed.
>> > >>
>> > >> Thanks,
>> > >> Vaibhav Garg
>> > >> +91-9505020924
>> > >> vaibhavgarg90@gmail.com
>> > >> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >>
>> > >>
>> > >> On Sat, Feb 29, 2020 at 5:22 AM Yi Pan <ni...@gmail.com> wrote:
>> > >>
>> > >>> Hi, Vaibhav,
>> > >>>
>> > >>> Not quite sure whether I understand your ask. How do you package and
>> > >>> distribute NewRelic jar in the YARN cluster? It would be much
>> helpful
>> > if
>> > >>> you can share your job configure that worked and the places that you
>> > >>> would
>> > >>> like to change.
>> > >>>
>> > >>> Best,
>> > >>>
>> > >>> -Yi
>> > >>>
>> > >>> On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg <
>> vaibhavgarg90@gmail.com
>> > >
>> > >>> wrote:
>> > >>>
>> > >>> > OK. I have been able to use NewRelic by specifying -javaagent
>> option
>> > in
>> > >>> > task.opts.
>> > >>> >
>> > >>> > However, I had to give the absolute path to the NewRelic jar file
>> in
>> > >>> order
>> > >>> > to make it work.
>> > >>> >
>> > >>> > How can I give a path that is relative to the container since I
>> would
>> > >>> like
>> > >>> > to integrate NewRelic dependencies along with other container
>> files?
>> > >>> >
>> > >>> > Thanks,
>> > >>> > Vaibhav Garg
>> > >>> > +91-9505020924
>> > >>> > vaibhavgarg90@gmail.com
>> > >>> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >>> >
>> > >>> >
>> > >>> > On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <
>> > vaibhavgarg90@gmail.com
>> > >>> >
>> > >>> > wrote:
>> > >>> >
>> > >>> > > Yes. That's exactly what I am looking for.
>> > >>> > >
>> > >>> > > Let me try it out and I will get back to you if I face any
>> issues.
>> > >>> > >
>> > >>> > > Thanks again
>> > >>> > >
>> > >>> > > Vaibhav Garg
>> > >>> > > +91-9505020924
>> > >>> > > vaibhavgarg90@gmail.com
>> > >>> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >>> > >
>> > >>> > >
>> > >>> > > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <ni...@gmail.com>
>> > wrote:
>> > >>> > >
>> > >>> > >> Hi, Vaibhav,
>> > >>> > >>
>> > >>> > >> Check the description of task.opts in the configuration doc
>> here:
>> > >>> > >>
>> > >>> > >>
>> > >>> >
>> > >>> http://samza.apache.org/learn/documentation/latest/jobs/
>> > samza-configurations.html
>> > >>> > >>
>> > >>> > >> Is this what you are looking for?
>> > >>> > >>
>> > >>> > >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <
>> > >>> vaibhavgarg90@gmail.com>
>> > >>> > >> wrote:
>> > >>> > >>
>> > >>> > >> > Hi,
>> > >>> > >> >
>> > >>> > >> > Any help here would be great.
>> > >>> > >> >
>> > >>> > >> > Thanks,
>> > >>> > >> > Vaibhav Garg
>> > >>> > >> > +91-9505020924
>> > >>> > >> > vaibhavgarg90@gmail.com
>> > >>> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >>> > >> >
>> > >>> > >> >
>> > >>> > >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <
>> > >>> > vaibhavgarg90@gmail.com>
>> > >>> > >> > wrote:
>> > >>> > >> >
>> > >>> > >> > > Dear Bharath,
>> > >>> > >> > >
>> > >>> > >> > > Thanks for your reply. I now realize that I put a very
>> > ambiguous
>> > >>> > >> question
>> > >>> > >> > > to the community. Here is another attempt:
>> > >>> > >> > >
>> > >>> > >> > > I have set up a Yarn cluster that is configured to run
>> > >>> containers of
>> > >>> > >> > > multiple Samza jobs.
>> > >>> > >> > >
>> > >>> > >> > > I would like to treat all the containers of the job as a
>> > single
>> > >>> > >> > > application in New Relic (as expected).
>> > >>> > >> > >
>> > >>> > >> > > Since the containers of the application can be killed on
>> one
>> > >>> Yarn
>> > >>> > node
>> > >>> > >> > and
>> > >>> > >> > > can start on another Yarn node, I would have to specify New
>> > >>> Relic
>> > >>> > >> > > environment settings at the container start.
>> > >>> > >> > >
>> > >>> > >> > > Now, I am not sure how to specify java agent (to include
>> New
>> > >>> Relic
>> > >>> > >> java
>> > >>> > >> > > agent) and pass additional arguments such as New Relic
>> > >>> environment
>> > >>> > >> every
>> > >>> > >> > > time the container starts.
>> > >>> > >> > >
>> > >>> > >> > > Please let me know if my understanding is wrong or some
>> other
>> > >>> > >> > > clarification is needed.
>> > >>> > >> > >
>> > >>> > >> > > Thanks in advance,
>> > >>> > >> > > Vaibhav Garg
>> > >>> > >> > > +91-9505020924
>> > >>> > >> > > vaibhavgarg90@gmail.com
>> > >>> > >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >>> > >> > >
>> > >>> > >> > >
>> > >>> > >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian
>> <
>> > >>> > >> > > codin.martial@gmail.com> wrote:
>> > >>> > >> > >
>> > >>> > >> > >> Hi,
>> > >>> > >> > >>
>> > >>> > >> > >> I am not sure I fully understand the ask.
>> > >>> > >> > >>
>> > >>> > >> > >> IIUC, Samza doesn't have native integration with New
>> relic.
>> > >>> > However,
>> > >>> > >> you
>> > >>> > >> > >> should still be able to integrate your application with
>> New
>> > >>> relic
>> > >>> > on
>> > >>> > >> > your
>> > >>> > >> > >> end without native support.
>> > >>> > >> > >> If you are particularly looking to integrate native Samza
>> > >>> metrics
>> > >>> > w/
>> > >>> > >> New
>> > >>> > >> > >> relic, you might need to implement your own custom metrics
>> > >>> > reporter.
>> > >>> > >> You
>> > >>> > >> > >> can find more details here
>> > >>> > >> > >> <
>> > >>> > >> > >>
>> > >>> > >> >
>> > >>> > >>
>> > >>> >
>> > >>> https://samza.apache.org/learn/documentation/latest/
>> > operations/monitoring.html#customreporter
>> > >>> > >> > >> >
>> > >>> > >> > >>
>> > >>> > >> > >> Thanks,
>> > >>> > >> > >> Bharath
>> > >>> > >> > >>
>> > >>> > >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <
>> > >>> > >> vaibhavgarg90@gmail.com>
>> > >>> > >> > >> wrote:
>> > >>> > >> > >>
>> > >>> > >> > >> > Hi,
>> > >>> > >> > >> >
>> > >>> > >> > >> > I want to integrate New Relic in my Samza jobs. Can
>> anyone
>> > >>> help
>> > >>> > >> with
>> > >>> > >> > >> this,
>> > >>> > >> > >> > please?
>> > >>> > >> > >> >
>> > >>> > >> > >> > Thanks,
>> > >>> > >> > >> > Vaibhav Garg
>> > >>> > >> > >> > +91-9505020924
>> > >>> > >> > >> > vaibhavgarg90@gmail.com
>> > >>> > >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >>> > >> > >> >
>> > >>> > >> > >>
>> > >>> > >> > >
>> > >>> > >> >
>> > >>> > >>
>> > >>> > >
>> > >>> >
>> > >>>
>> > >>
>> >
>>
>>
>> --
>> Jagadish
>>
>
Re: Samza - New Relic integration
Posted by Vaibhav Garg <va...@gmail.com>.
Ok. I will try that.
Thanks,
Vaibhav Garg
+91-9505020924
vaibhavgarg90@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Sun, Mar 1, 2020 at 12:31 PM Jagadish Venkatraman <ja...@gmail.com>
wrote:
> Hi Vaibhav,
>
> Can you try ./__package/newrelic/newrelic.jar?
>
> If that doesn't work, I'd recommend doing the following:
> 1. Print the current working directory of your Samza container JVM
> 2. Use (1) to determine a relative reference to the directory path within
> your package
>
> Cheers
> Jagadish
>
> On Saturday, February 29, 2020, Vaibhav Garg <va...@gmail.com>
> wrote:
>
> > I would be really grateful if someone could help me with this.
> >
> > Thanks,
> > Vaibhav Garg
> > +91-9505020924
> > vaibhavgarg90@gmail.com
> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >
> >
> > On Sat, Feb 29, 2020 at 9:02 PM Vaibhav Garg <va...@gmail.com>
> > wrote:
> >
> > > Please do let me know if some other clarification is needed.
> > >
> > > Thanks,
> > > Vaibhav Garg
> > > +91-9505020924
> > > vaibhavgarg90@gmail.com
> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >
> > >
> > > On Sat, Feb 29, 2020 at 10:31 AM Vaibhav Garg <vaibhavgarg90@gmail.com
> >
> > > wrote:
> > >
> > >> OK. Here is what I have inside my configuration in order to start
> > >> NewRelic at the time of the container start:
> > >>
> > >> task.opts=-Xmx2048m -XX:+UseConcMarkSweepGC
> > >> *-javaagent:/usr/home/vaibhav/newrelic/newrelic.jar*
> > >>
> > >> Now, I have specified the absolute path of the NewRelic jar. There
> are 2
> > >> challenges that I see with this approach:
> > >>
> > >> 1. The NewRelic jar and config would have to be present on all the
> Yarn
> > >> slave nodes since that container can start at any of the Yarn nodes.
> > >> 2. A single Yarn cluster can be used to run multiple Samza
> applications,
> > >> so I would have to keep configuration for each application at each of
> > the
> > >> Yarn slave nodes.
> > >>
> > >> In order to fix this, I am bundling NewRelic inside my tar file:
> > >>
> > >> // download new relic dependencies
> > >> task downloadNewrelic(type: Download) {
> > >> mkdir "newrelic"
> > >> src "
> > >> https://download.newrelic.com/newrelic/java-agent/newrelic-
> > agent/current/newrelic-java.zip
> > >> "
> > >> dest file("newrelic")
> > >> }
> > >>
> > >> task buildTar(dependsOn: [build, downloadNewrelic], type: Tar) {
> > >> // unzip new relic dependencies and add to the archive
> > >> into(".") {
> > >> from zipTree(file("newrelic/newrelic-java.zip"))
> > >> }
> > >> }
> > >>
> > >> With this build configuration, I have an additional *newrelic*
> directory
> > >> inside my archive. Here is the directory structure:
> > >> my-job-name.tar.gz
> > >> bin
> > >> config
> > >> lib
> > >> newrelic
> > >> newrelic.jar
> > >> newrelic.yml
> > >> ...
> > >>
> > >> What I would like to achieve is to:
> > >> - Remove hardcoding the *javaagent* inside *task.opts* in the
> container
> > >> config.
> > >> - Specify the path of NewRelic agent in *task.opts* relative to where
> > >> the container files are present on the Yarn slave node.
> > >>
> > >> I hope I was clearer this time. Please let me know if any other
> > >> clarification is needed.
> > >>
> > >> Thanks,
> > >> Vaibhav Garg
> > >> +91-9505020924
> > >> vaibhavgarg90@gmail.com
> > >> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >>
> > >>
> > >> On Sat, Feb 29, 2020 at 5:22 AM Yi Pan <ni...@gmail.com> wrote:
> > >>
> > >>> Hi, Vaibhav,
> > >>>
> > >>> Not quite sure whether I understand your ask. How do you package and
> > >>> distribute NewRelic jar in the YARN cluster? It would be much helpful
> > if
> > >>> you can share your job configure that worked and the places that you
> > >>> would
> > >>> like to change.
> > >>>
> > >>> Best,
> > >>>
> > >>> -Yi
> > >>>
> > >>> On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg <
> vaibhavgarg90@gmail.com
> > >
> > >>> wrote:
> > >>>
> > >>> > OK. I have been able to use NewRelic by specifying -javaagent
> option
> > in
> > >>> > task.opts.
> > >>> >
> > >>> > However, I had to give the absolute path to the NewRelic jar file
> in
> > >>> order
> > >>> > to make it work.
> > >>> >
> > >>> > How can I give a path that is relative to the container since I
> would
> > >>> like
> > >>> > to integrate NewRelic dependencies along with other container
> files?
> > >>> >
> > >>> > Thanks,
> > >>> > Vaibhav Garg
> > >>> > +91-9505020924
> > >>> > vaibhavgarg90@gmail.com
> > >>> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >>> >
> > >>> >
> > >>> > On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <
> > vaibhavgarg90@gmail.com
> > >>> >
> > >>> > wrote:
> > >>> >
> > >>> > > Yes. That's exactly what I am looking for.
> > >>> > >
> > >>> > > Let me try it out and I will get back to you if I face any
> issues.
> > >>> > >
> > >>> > > Thanks again
> > >>> > >
> > >>> > > Vaibhav Garg
> > >>> > > +91-9505020924
> > >>> > > vaibhavgarg90@gmail.com
> > >>> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >>> > >
> > >>> > >
> > >>> > > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <ni...@gmail.com>
> > wrote:
> > >>> > >
> > >>> > >> Hi, Vaibhav,
> > >>> > >>
> > >>> > >> Check the description of task.opts in the configuration doc
> here:
> > >>> > >>
> > >>> > >>
> > >>> >
> > >>> http://samza.apache.org/learn/documentation/latest/jobs/
> > samza-configurations.html
> > >>> > >>
> > >>> > >> Is this what you are looking for?
> > >>> > >>
> > >>> > >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <
> > >>> vaibhavgarg90@gmail.com>
> > >>> > >> wrote:
> > >>> > >>
> > >>> > >> > Hi,
> > >>> > >> >
> > >>> > >> > Any help here would be great.
> > >>> > >> >
> > >>> > >> > Thanks,
> > >>> > >> > Vaibhav Garg
> > >>> > >> > +91-9505020924
> > >>> > >> > vaibhavgarg90@gmail.com
> > >>> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >>> > >> >
> > >>> > >> >
> > >>> > >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <
> > >>> > vaibhavgarg90@gmail.com>
> > >>> > >> > wrote:
> > >>> > >> >
> > >>> > >> > > Dear Bharath,
> > >>> > >> > >
> > >>> > >> > > Thanks for your reply. I now realize that I put a very
> > ambiguous
> > >>> > >> question
> > >>> > >> > > to the community. Here is another attempt:
> > >>> > >> > >
> > >>> > >> > > I have set up a Yarn cluster that is configured to run
> > >>> containers of
> > >>> > >> > > multiple Samza jobs.
> > >>> > >> > >
> > >>> > >> > > I would like to treat all the containers of the job as a
> > single
> > >>> > >> > > application in New Relic (as expected).
> > >>> > >> > >
> > >>> > >> > > Since the containers of the application can be killed on one
> > >>> Yarn
> > >>> > node
> > >>> > >> > and
> > >>> > >> > > can start on another Yarn node, I would have to specify New
> > >>> Relic
> > >>> > >> > > environment settings at the container start.
> > >>> > >> > >
> > >>> > >> > > Now, I am not sure how to specify java agent (to include New
> > >>> Relic
> > >>> > >> java
> > >>> > >> > > agent) and pass additional arguments such as New Relic
> > >>> environment
> > >>> > >> every
> > >>> > >> > > time the container starts.
> > >>> > >> > >
> > >>> > >> > > Please let me know if my understanding is wrong or some
> other
> > >>> > >> > > clarification is needed.
> > >>> > >> > >
> > >>> > >> > > Thanks in advance,
> > >>> > >> > > Vaibhav Garg
> > >>> > >> > > +91-9505020924
> > >>> > >> > > vaibhavgarg90@gmail.com
> > >>> > >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >>> > >> > >
> > >>> > >> > >
> > >>> > >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
> > >>> > >> > > codin.martial@gmail.com> wrote:
> > >>> > >> > >
> > >>> > >> > >> Hi,
> > >>> > >> > >>
> > >>> > >> > >> I am not sure I fully understand the ask.
> > >>> > >> > >>
> > >>> > >> > >> IIUC, Samza doesn't have native integration with New relic.
> > >>> > However,
> > >>> > >> you
> > >>> > >> > >> should still be able to integrate your application with New
> > >>> relic
> > >>> > on
> > >>> > >> > your
> > >>> > >> > >> end without native support.
> > >>> > >> > >> If you are particularly looking to integrate native Samza
> > >>> metrics
> > >>> > w/
> > >>> > >> New
> > >>> > >> > >> relic, you might need to implement your own custom metrics
> > >>> > reporter.
> > >>> > >> You
> > >>> > >> > >> can find more details here
> > >>> > >> > >> <
> > >>> > >> > >>
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>> https://samza.apache.org/learn/documentation/latest/
> > operations/monitoring.html#customreporter
> > >>> > >> > >> >
> > >>> > >> > >>
> > >>> > >> > >> Thanks,
> > >>> > >> > >> Bharath
> > >>> > >> > >>
> > >>> > >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <
> > >>> > >> vaibhavgarg90@gmail.com>
> > >>> > >> > >> wrote:
> > >>> > >> > >>
> > >>> > >> > >> > Hi,
> > >>> > >> > >> >
> > >>> > >> > >> > I want to integrate New Relic in my Samza jobs. Can
> anyone
> > >>> help
> > >>> > >> with
> > >>> > >> > >> this,
> > >>> > >> > >> > please?
> > >>> > >> > >> >
> > >>> > >> > >> > Thanks,
> > >>> > >> > >> > Vaibhav Garg
> > >>> > >> > >> > +91-9505020924
> > >>> > >> > >> > vaibhavgarg90@gmail.com
> > >>> > >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >>> > >> > >> >
> > >>> > >> > >>
> > >>> > >> > >
> > >>> > >> >
> > >>> > >>
> > >>> > >
> > >>> >
> > >>>
> > >>
> >
>
>
> --
> Jagadish
>
Re: Samza - New Relic integration
Posted by Jagadish Venkatraman <ja...@gmail.com>.
Hi Vaibhav,
Can you try ./__package/newrelic/newrelic.jar?
If that doesn't work, I'd recommend doing the following:
1. Print the current working directory of your Samza container JVM
2. Use (1) to determine a relative reference to the directory path within
your package
Cheers
Jagadish
On Saturday, February 29, 2020, Vaibhav Garg <va...@gmail.com>
wrote:
> I would be really grateful if someone could help me with this.
>
> Thanks,
> Vaibhav Garg
> +91-9505020924
> vaibhavgarg90@gmail.com
> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>
>
> On Sat, Feb 29, 2020 at 9:02 PM Vaibhav Garg <va...@gmail.com>
> wrote:
>
> > Please do let me know if some other clarification is needed.
> >
> > Thanks,
> > Vaibhav Garg
> > +91-9505020924
> > vaibhavgarg90@gmail.com
> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >
> >
> > On Sat, Feb 29, 2020 at 10:31 AM Vaibhav Garg <va...@gmail.com>
> > wrote:
> >
> >> OK. Here is what I have inside my configuration in order to start
> >> NewRelic at the time of the container start:
> >>
> >> task.opts=-Xmx2048m -XX:+UseConcMarkSweepGC
> >> *-javaagent:/usr/home/vaibhav/newrelic/newrelic.jar*
> >>
> >> Now, I have specified the absolute path of the NewRelic jar. There are 2
> >> challenges that I see with this approach:
> >>
> >> 1. The NewRelic jar and config would have to be present on all the Yarn
> >> slave nodes since that container can start at any of the Yarn nodes.
> >> 2. A single Yarn cluster can be used to run multiple Samza applications,
> >> so I would have to keep configuration for each application at each of
> the
> >> Yarn slave nodes.
> >>
> >> In order to fix this, I am bundling NewRelic inside my tar file:
> >>
> >> // download new relic dependencies
> >> task downloadNewrelic(type: Download) {
> >> mkdir "newrelic"
> >> src "
> >> https://download.newrelic.com/newrelic/java-agent/newrelic-
> agent/current/newrelic-java.zip
> >> "
> >> dest file("newrelic")
> >> }
> >>
> >> task buildTar(dependsOn: [build, downloadNewrelic], type: Tar) {
> >> // unzip new relic dependencies and add to the archive
> >> into(".") {
> >> from zipTree(file("newrelic/newrelic-java.zip"))
> >> }
> >> }
> >>
> >> With this build configuration, I have an additional *newrelic* directory
> >> inside my archive. Here is the directory structure:
> >> my-job-name.tar.gz
> >> bin
> >> config
> >> lib
> >> newrelic
> >> newrelic.jar
> >> newrelic.yml
> >> ...
> >>
> >> What I would like to achieve is to:
> >> - Remove hardcoding the *javaagent* inside *task.opts* in the container
> >> config.
> >> - Specify the path of NewRelic agent in *task.opts* relative to where
> >> the container files are present on the Yarn slave node.
> >>
> >> I hope I was clearer this time. Please let me know if any other
> >> clarification is needed.
> >>
> >> Thanks,
> >> Vaibhav Garg
> >> +91-9505020924
> >> vaibhavgarg90@gmail.com
> >> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >>
> >>
> >> On Sat, Feb 29, 2020 at 5:22 AM Yi Pan <ni...@gmail.com> wrote:
> >>
> >>> Hi, Vaibhav,
> >>>
> >>> Not quite sure whether I understand your ask. How do you package and
> >>> distribute NewRelic jar in the YARN cluster? It would be much helpful
> if
> >>> you can share your job configure that worked and the places that you
> >>> would
> >>> like to change.
> >>>
> >>> Best,
> >>>
> >>> -Yi
> >>>
> >>> On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg <vaibhavgarg90@gmail.com
> >
> >>> wrote:
> >>>
> >>> > OK. I have been able to use NewRelic by specifying -javaagent option
> in
> >>> > task.opts.
> >>> >
> >>> > However, I had to give the absolute path to the NewRelic jar file in
> >>> order
> >>> > to make it work.
> >>> >
> >>> > How can I give a path that is relative to the container since I would
> >>> like
> >>> > to integrate NewRelic dependencies along with other container files?
> >>> >
> >>> > Thanks,
> >>> > Vaibhav Garg
> >>> > +91-9505020924
> >>> > vaibhavgarg90@gmail.com
> >>> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >>> >
> >>> >
> >>> > On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <
> vaibhavgarg90@gmail.com
> >>> >
> >>> > wrote:
> >>> >
> >>> > > Yes. That's exactly what I am looking for.
> >>> > >
> >>> > > Let me try it out and I will get back to you if I face any issues.
> >>> > >
> >>> > > Thanks again
> >>> > >
> >>> > > Vaibhav Garg
> >>> > > +91-9505020924
> >>> > > vaibhavgarg90@gmail.com
> >>> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >>> > >
> >>> > >
> >>> > > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <ni...@gmail.com>
> wrote:
> >>> > >
> >>> > >> Hi, Vaibhav,
> >>> > >>
> >>> > >> Check the description of task.opts in the configuration doc here:
> >>> > >>
> >>> > >>
> >>> >
> >>> http://samza.apache.org/learn/documentation/latest/jobs/
> samza-configurations.html
> >>> > >>
> >>> > >> Is this what you are looking for?
> >>> > >>
> >>> > >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <
> >>> vaibhavgarg90@gmail.com>
> >>> > >> wrote:
> >>> > >>
> >>> > >> > Hi,
> >>> > >> >
> >>> > >> > Any help here would be great.
> >>> > >> >
> >>> > >> > Thanks,
> >>> > >> > Vaibhav Garg
> >>> > >> > +91-9505020924
> >>> > >> > vaibhavgarg90@gmail.com
> >>> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >>> > >> >
> >>> > >> >
> >>> > >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <
> >>> > vaibhavgarg90@gmail.com>
> >>> > >> > wrote:
> >>> > >> >
> >>> > >> > > Dear Bharath,
> >>> > >> > >
> >>> > >> > > Thanks for your reply. I now realize that I put a very
> ambiguous
> >>> > >> question
> >>> > >> > > to the community. Here is another attempt:
> >>> > >> > >
> >>> > >> > > I have set up a Yarn cluster that is configured to run
> >>> containers of
> >>> > >> > > multiple Samza jobs.
> >>> > >> > >
> >>> > >> > > I would like to treat all the containers of the job as a
> single
> >>> > >> > > application in New Relic (as expected).
> >>> > >> > >
> >>> > >> > > Since the containers of the application can be killed on one
> >>> Yarn
> >>> > node
> >>> > >> > and
> >>> > >> > > can start on another Yarn node, I would have to specify New
> >>> Relic
> >>> > >> > > environment settings at the container start.
> >>> > >> > >
> >>> > >> > > Now, I am not sure how to specify java agent (to include New
> >>> Relic
> >>> > >> java
> >>> > >> > > agent) and pass additional arguments such as New Relic
> >>> environment
> >>> > >> every
> >>> > >> > > time the container starts.
> >>> > >> > >
> >>> > >> > > Please let me know if my understanding is wrong or some other
> >>> > >> > > clarification is needed.
> >>> > >> > >
> >>> > >> > > Thanks in advance,
> >>> > >> > > Vaibhav Garg
> >>> > >> > > +91-9505020924
> >>> > >> > > vaibhavgarg90@gmail.com
> >>> > >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >>> > >> > >
> >>> > >> > >
> >>> > >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
> >>> > >> > > codin.martial@gmail.com> wrote:
> >>> > >> > >
> >>> > >> > >> Hi,
> >>> > >> > >>
> >>> > >> > >> I am not sure I fully understand the ask.
> >>> > >> > >>
> >>> > >> > >> IIUC, Samza doesn't have native integration with New relic.
> >>> > However,
> >>> > >> you
> >>> > >> > >> should still be able to integrate your application with New
> >>> relic
> >>> > on
> >>> > >> > your
> >>> > >> > >> end without native support.
> >>> > >> > >> If you are particularly looking to integrate native Samza
> >>> metrics
> >>> > w/
> >>> > >> New
> >>> > >> > >> relic, you might need to implement your own custom metrics
> >>> > reporter.
> >>> > >> You
> >>> > >> > >> can find more details here
> >>> > >> > >> <
> >>> > >> > >>
> >>> > >> >
> >>> > >>
> >>> >
> >>> https://samza.apache.org/learn/documentation/latest/
> operations/monitoring.html#customreporter
> >>> > >> > >> >
> >>> > >> > >>
> >>> > >> > >> Thanks,
> >>> > >> > >> Bharath
> >>> > >> > >>
> >>> > >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <
> >>> > >> vaibhavgarg90@gmail.com>
> >>> > >> > >> wrote:
> >>> > >> > >>
> >>> > >> > >> > Hi,
> >>> > >> > >> >
> >>> > >> > >> > I want to integrate New Relic in my Samza jobs. Can anyone
> >>> help
> >>> > >> with
> >>> > >> > >> this,
> >>> > >> > >> > please?
> >>> > >> > >> >
> >>> > >> > >> > Thanks,
> >>> > >> > >> > Vaibhav Garg
> >>> > >> > >> > +91-9505020924
> >>> > >> > >> > vaibhavgarg90@gmail.com
> >>> > >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >>> > >> > >> >
> >>> > >> > >>
> >>> > >> > >
> >>> > >> >
> >>> > >>
> >>> > >
> >>> >
> >>>
> >>
>
--
Jagadish
Re: Samza - New Relic integration
Posted by Vaibhav Garg <va...@gmail.com>.
I would be really grateful if someone could help me with this.
Thanks,
Vaibhav Garg
+91-9505020924
vaibhavgarg90@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Sat, Feb 29, 2020 at 9:02 PM Vaibhav Garg <va...@gmail.com>
wrote:
> Please do let me know if some other clarification is needed.
>
> Thanks,
> Vaibhav Garg
> +91-9505020924
> vaibhavgarg90@gmail.com
> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>
>
> On Sat, Feb 29, 2020 at 10:31 AM Vaibhav Garg <va...@gmail.com>
> wrote:
>
>> OK. Here is what I have inside my configuration in order to start
>> NewRelic at the time of the container start:
>>
>> task.opts=-Xmx2048m -XX:+UseConcMarkSweepGC
>> *-javaagent:/usr/home/vaibhav/newrelic/newrelic.jar*
>>
>> Now, I have specified the absolute path of the NewRelic jar. There are 2
>> challenges that I see with this approach:
>>
>> 1. The NewRelic jar and config would have to be present on all the Yarn
>> slave nodes since that container can start at any of the Yarn nodes.
>> 2. A single Yarn cluster can be used to run multiple Samza applications,
>> so I would have to keep configuration for each application at each of the
>> Yarn slave nodes.
>>
>> In order to fix this, I am bundling NewRelic inside my tar file:
>>
>> // download new relic dependencies
>> task downloadNewrelic(type: Download) {
>> mkdir "newrelic"
>> src "
>> https://download.newrelic.com/newrelic/java-agent/newrelic-agent/current/newrelic-java.zip
>> "
>> dest file("newrelic")
>> }
>>
>> task buildTar(dependsOn: [build, downloadNewrelic], type: Tar) {
>> // unzip new relic dependencies and add to the archive
>> into(".") {
>> from zipTree(file("newrelic/newrelic-java.zip"))
>> }
>> }
>>
>> With this build configuration, I have an additional *newrelic* directory
>> inside my archive. Here is the directory structure:
>> my-job-name.tar.gz
>> bin
>> config
>> lib
>> newrelic
>> newrelic.jar
>> newrelic.yml
>> ...
>>
>> What I would like to achieve is to:
>> - Remove hardcoding the *javaagent* inside *task.opts* in the container
>> config.
>> - Specify the path of NewRelic agent in *task.opts* relative to where
>> the container files are present on the Yarn slave node.
>>
>> I hope I was clearer this time. Please let me know if any other
>> clarification is needed.
>>
>> Thanks,
>> Vaibhav Garg
>> +91-9505020924
>> vaibhavgarg90@gmail.com
>> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>>
>>
>> On Sat, Feb 29, 2020 at 5:22 AM Yi Pan <ni...@gmail.com> wrote:
>>
>>> Hi, Vaibhav,
>>>
>>> Not quite sure whether I understand your ask. How do you package and
>>> distribute NewRelic jar in the YARN cluster? It would be much helpful if
>>> you can share your job configure that worked and the places that you
>>> would
>>> like to change.
>>>
>>> Best,
>>>
>>> -Yi
>>>
>>> On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg <va...@gmail.com>
>>> wrote:
>>>
>>> > OK. I have been able to use NewRelic by specifying -javaagent option in
>>> > task.opts.
>>> >
>>> > However, I had to give the absolute path to the NewRelic jar file in
>>> order
>>> > to make it work.
>>> >
>>> > How can I give a path that is relative to the container since I would
>>> like
>>> > to integrate NewRelic dependencies along with other container files?
>>> >
>>> > Thanks,
>>> > Vaibhav Garg
>>> > +91-9505020924
>>> > vaibhavgarg90@gmail.com
>>> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>>> >
>>> >
>>> > On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <vaibhavgarg90@gmail.com
>>> >
>>> > wrote:
>>> >
>>> > > Yes. That's exactly what I am looking for.
>>> > >
>>> > > Let me try it out and I will get back to you if I face any issues.
>>> > >
>>> > > Thanks again
>>> > >
>>> > > Vaibhav Garg
>>> > > +91-9505020924
>>> > > vaibhavgarg90@gmail.com
>>> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>>> > >
>>> > >
>>> > > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <ni...@gmail.com> wrote:
>>> > >
>>> > >> Hi, Vaibhav,
>>> > >>
>>> > >> Check the description of task.opts in the configuration doc here:
>>> > >>
>>> > >>
>>> >
>>> http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html
>>> > >>
>>> > >> Is this what you are looking for?
>>> > >>
>>> > >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <
>>> vaibhavgarg90@gmail.com>
>>> > >> wrote:
>>> > >>
>>> > >> > Hi,
>>> > >> >
>>> > >> > Any help here would be great.
>>> > >> >
>>> > >> > Thanks,
>>> > >> > Vaibhav Garg
>>> > >> > +91-9505020924
>>> > >> > vaibhavgarg90@gmail.com
>>> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>>> > >> >
>>> > >> >
>>> > >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <
>>> > vaibhavgarg90@gmail.com>
>>> > >> > wrote:
>>> > >> >
>>> > >> > > Dear Bharath,
>>> > >> > >
>>> > >> > > Thanks for your reply. I now realize that I put a very ambiguous
>>> > >> question
>>> > >> > > to the community. Here is another attempt:
>>> > >> > >
>>> > >> > > I have set up a Yarn cluster that is configured to run
>>> containers of
>>> > >> > > multiple Samza jobs.
>>> > >> > >
>>> > >> > > I would like to treat all the containers of the job as a single
>>> > >> > > application in New Relic (as expected).
>>> > >> > >
>>> > >> > > Since the containers of the application can be killed on one
>>> Yarn
>>> > node
>>> > >> > and
>>> > >> > > can start on another Yarn node, I would have to specify New
>>> Relic
>>> > >> > > environment settings at the container start.
>>> > >> > >
>>> > >> > > Now, I am not sure how to specify java agent (to include New
>>> Relic
>>> > >> java
>>> > >> > > agent) and pass additional arguments such as New Relic
>>> environment
>>> > >> every
>>> > >> > > time the container starts.
>>> > >> > >
>>> > >> > > Please let me know if my understanding is wrong or some other
>>> > >> > > clarification is needed.
>>> > >> > >
>>> > >> > > Thanks in advance,
>>> > >> > > Vaibhav Garg
>>> > >> > > +91-9505020924
>>> > >> > > vaibhavgarg90@gmail.com
>>> > >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>>> > >> > >
>>> > >> > >
>>> > >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
>>> > >> > > codin.martial@gmail.com> wrote:
>>> > >> > >
>>> > >> > >> Hi,
>>> > >> > >>
>>> > >> > >> I am not sure I fully understand the ask.
>>> > >> > >>
>>> > >> > >> IIUC, Samza doesn't have native integration with New relic.
>>> > However,
>>> > >> you
>>> > >> > >> should still be able to integrate your application with New
>>> relic
>>> > on
>>> > >> > your
>>> > >> > >> end without native support.
>>> > >> > >> If you are particularly looking to integrate native Samza
>>> metrics
>>> > w/
>>> > >> New
>>> > >> > >> relic, you might need to implement your own custom metrics
>>> > reporter.
>>> > >> You
>>> > >> > >> can find more details here
>>> > >> > >> <
>>> > >> > >>
>>> > >> >
>>> > >>
>>> >
>>> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter
>>> > >> > >> >
>>> > >> > >>
>>> > >> > >> Thanks,
>>> > >> > >> Bharath
>>> > >> > >>
>>> > >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <
>>> > >> vaibhavgarg90@gmail.com>
>>> > >> > >> wrote:
>>> > >> > >>
>>> > >> > >> > Hi,
>>> > >> > >> >
>>> > >> > >> > I want to integrate New Relic in my Samza jobs. Can anyone
>>> help
>>> > >> with
>>> > >> > >> this,
>>> > >> > >> > please?
>>> > >> > >> >
>>> > >> > >> > Thanks,
>>> > >> > >> > Vaibhav Garg
>>> > >> > >> > +91-9505020924
>>> > >> > >> > vaibhavgarg90@gmail.com
>>> > >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>>> > >> > >> >
>>> > >> > >>
>>> > >> > >
>>> > >> >
>>> > >>
>>> > >
>>> >
>>>
>>
Re: Samza - New Relic integration
Posted by Vaibhav Garg <va...@gmail.com>.
Please do let me know if some other clarification is needed.
Thanks,
Vaibhav Garg
+91-9505020924
vaibhavgarg90@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Sat, Feb 29, 2020 at 10:31 AM Vaibhav Garg <va...@gmail.com>
wrote:
> OK. Here is what I have inside my configuration in order to start NewRelic
> at the time of the container start:
>
> task.opts=-Xmx2048m -XX:+UseConcMarkSweepGC
> *-javaagent:/usr/home/vaibhav/newrelic/newrelic.jar*
>
> Now, I have specified the absolute path of the NewRelic jar. There are 2
> challenges that I see with this approach:
>
> 1. The NewRelic jar and config would have to be present on all the Yarn
> slave nodes since that container can start at any of the Yarn nodes.
> 2. A single Yarn cluster can be used to run multiple Samza applications,
> so I would have to keep configuration for each application at each of the
> Yarn slave nodes.
>
> In order to fix this, I am bundling NewRelic inside my tar file:
>
> // download new relic dependencies
> task downloadNewrelic(type: Download) {
> mkdir "newrelic"
> src "
> https://download.newrelic.com/newrelic/java-agent/newrelic-agent/current/newrelic-java.zip
> "
> dest file("newrelic")
> }
>
> task buildTar(dependsOn: [build, downloadNewrelic], type: Tar) {
> // unzip new relic dependencies and add to the archive
> into(".") {
> from zipTree(file("newrelic/newrelic-java.zip"))
> }
> }
>
> With this build configuration, I have an additional *newrelic* directory
> inside my archive. Here is the directory structure:
> my-job-name.tar.gz
> bin
> config
> lib
> newrelic
> newrelic.jar
> newrelic.yml
> ...
>
> What I would like to achieve is to:
> - Remove hardcoding the *javaagent* inside *task.opts* in the container
> config.
> - Specify the path of NewRelic agent in *task.opts* relative to where the
> container files are present on the Yarn slave node.
>
> I hope I was clearer this time. Please let me know if any other
> clarification is needed.
>
> Thanks,
> Vaibhav Garg
> +91-9505020924
> vaibhavgarg90@gmail.com
> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>
>
> On Sat, Feb 29, 2020 at 5:22 AM Yi Pan <ni...@gmail.com> wrote:
>
>> Hi, Vaibhav,
>>
>> Not quite sure whether I understand your ask. How do you package and
>> distribute NewRelic jar in the YARN cluster? It would be much helpful if
>> you can share your job configure that worked and the places that you would
>> like to change.
>>
>> Best,
>>
>> -Yi
>>
>> On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg <va...@gmail.com>
>> wrote:
>>
>> > OK. I have been able to use NewRelic by specifying -javaagent option in
>> > task.opts.
>> >
>> > However, I had to give the absolute path to the NewRelic jar file in
>> order
>> > to make it work.
>> >
>> > How can I give a path that is relative to the container since I would
>> like
>> > to integrate NewRelic dependencies along with other container files?
>> >
>> > Thanks,
>> > Vaibhav Garg
>> > +91-9505020924
>> > vaibhavgarg90@gmail.com
>> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> >
>> >
>> > On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <va...@gmail.com>
>> > wrote:
>> >
>> > > Yes. That's exactly what I am looking for.
>> > >
>> > > Let me try it out and I will get back to you if I face any issues.
>> > >
>> > > Thanks again
>> > >
>> > > Vaibhav Garg
>> > > +91-9505020924
>> > > vaibhavgarg90@gmail.com
>> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >
>> > >
>> > > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <ni...@gmail.com> wrote:
>> > >
>> > >> Hi, Vaibhav,
>> > >>
>> > >> Check the description of task.opts in the configuration doc here:
>> > >>
>> > >>
>> >
>> http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html
>> > >>
>> > >> Is this what you are looking for?
>> > >>
>> > >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <
>> vaibhavgarg90@gmail.com>
>> > >> wrote:
>> > >>
>> > >> > Hi,
>> > >> >
>> > >> > Any help here would be great.
>> > >> >
>> > >> > Thanks,
>> > >> > Vaibhav Garg
>> > >> > +91-9505020924
>> > >> > vaibhavgarg90@gmail.com
>> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >> >
>> > >> >
>> > >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <
>> > vaibhavgarg90@gmail.com>
>> > >> > wrote:
>> > >> >
>> > >> > > Dear Bharath,
>> > >> > >
>> > >> > > Thanks for your reply. I now realize that I put a very ambiguous
>> > >> question
>> > >> > > to the community. Here is another attempt:
>> > >> > >
>> > >> > > I have set up a Yarn cluster that is configured to run
>> containers of
>> > >> > > multiple Samza jobs.
>> > >> > >
>> > >> > > I would like to treat all the containers of the job as a single
>> > >> > > application in New Relic (as expected).
>> > >> > >
>> > >> > > Since the containers of the application can be killed on one Yarn
>> > node
>> > >> > and
>> > >> > > can start on another Yarn node, I would have to specify New Relic
>> > >> > > environment settings at the container start.
>> > >> > >
>> > >> > > Now, I am not sure how to specify java agent (to include New
>> Relic
>> > >> java
>> > >> > > agent) and pass additional arguments such as New Relic
>> environment
>> > >> every
>> > >> > > time the container starts.
>> > >> > >
>> > >> > > Please let me know if my understanding is wrong or some other
>> > >> > > clarification is needed.
>> > >> > >
>> > >> > > Thanks in advance,
>> > >> > > Vaibhav Garg
>> > >> > > +91-9505020924
>> > >> > > vaibhavgarg90@gmail.com
>> > >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >> > >
>> > >> > >
>> > >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
>> > >> > > codin.martial@gmail.com> wrote:
>> > >> > >
>> > >> > >> Hi,
>> > >> > >>
>> > >> > >> I am not sure I fully understand the ask.
>> > >> > >>
>> > >> > >> IIUC, Samza doesn't have native integration with New relic.
>> > However,
>> > >> you
>> > >> > >> should still be able to integrate your application with New
>> relic
>> > on
>> > >> > your
>> > >> > >> end without native support.
>> > >> > >> If you are particularly looking to integrate native Samza
>> metrics
>> > w/
>> > >> New
>> > >> > >> relic, you might need to implement your own custom metrics
>> > reporter.
>> > >> You
>> > >> > >> can find more details here
>> > >> > >> <
>> > >> > >>
>> > >> >
>> > >>
>> >
>> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter
>> > >> > >> >
>> > >> > >>
>> > >> > >> Thanks,
>> > >> > >> Bharath
>> > >> > >>
>> > >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <
>> > >> vaibhavgarg90@gmail.com>
>> > >> > >> wrote:
>> > >> > >>
>> > >> > >> > Hi,
>> > >> > >> >
>> > >> > >> > I want to integrate New Relic in my Samza jobs. Can anyone
>> help
>> > >> with
>> > >> > >> this,
>> > >> > >> > please?
>> > >> > >> >
>> > >> > >> > Thanks,
>> > >> > >> > Vaibhav Garg
>> > >> > >> > +91-9505020924
>> > >> > >> > vaibhavgarg90@gmail.com
>> > >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >> > >> >
>> > >> > >>
>> > >> > >
>> > >> >
>> > >>
>> > >
>> >
>>
>
Re: Samza - New Relic integration
Posted by Vaibhav Garg <va...@gmail.com>.
OK. Here is what I have inside my configuration in order to start NewRelic
at the time of the container start:
task.opts=-Xmx2048m -XX:+UseConcMarkSweepGC
*-javaagent:/usr/home/vaibhav/newrelic/newrelic.jar*
Now, I have specified the absolute path of the NewRelic jar. There are 2
challenges that I see with this approach:
1. The NewRelic jar and config would have to be present on all the Yarn
slave nodes since that container can start at any of the Yarn nodes.
2. A single Yarn cluster can be used to run multiple Samza applications, so
I would have to keep configuration for each application at each of the Yarn
slave nodes.
In order to fix this, I am bundling NewRelic inside my tar file:
// download new relic dependencies
task downloadNewrelic(type: Download) {
mkdir "newrelic"
src "
https://download.newrelic.com/newrelic/java-agent/newrelic-agent/current/newrelic-java.zip
"
dest file("newrelic")
}
task buildTar(dependsOn: [build, downloadNewrelic], type: Tar) {
// unzip new relic dependencies and add to the archive
into(".") {
from zipTree(file("newrelic/newrelic-java.zip"))
}
}
With this build configuration, I have an additional *newrelic* directory
inside my archive. Here is the directory structure:
my-job-name.tar.gz
bin
config
lib
newrelic
newrelic.jar
newrelic.yml
...
What I would like to achieve is to:
- Remove hardcoding the *javaagent* inside *task.opts* in the container
config.
- Specify the path of NewRelic agent in *task.opts* relative to where the
container files are present on the Yarn slave node.
I hope I was clearer this time. Please let me know if any other
clarification is needed.
Thanks,
Vaibhav Garg
+91-9505020924
vaibhavgarg90@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Sat, Feb 29, 2020 at 5:22 AM Yi Pan <ni...@gmail.com> wrote:
> Hi, Vaibhav,
>
> Not quite sure whether I understand your ask. How do you package and
> distribute NewRelic jar in the YARN cluster? It would be much helpful if
> you can share your job configure that worked and the places that you would
> like to change.
>
> Best,
>
> -Yi
>
> On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg <va...@gmail.com>
> wrote:
>
> > OK. I have been able to use NewRelic by specifying -javaagent option in
> > task.opts.
> >
> > However, I had to give the absolute path to the NewRelic jar file in
> order
> > to make it work.
> >
> > How can I give a path that is relative to the container since I would
> like
> > to integrate NewRelic dependencies along with other container files?
> >
> > Thanks,
> > Vaibhav Garg
> > +91-9505020924
> > vaibhavgarg90@gmail.com
> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >
> >
> > On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <va...@gmail.com>
> > wrote:
> >
> > > Yes. That's exactly what I am looking for.
> > >
> > > Let me try it out and I will get back to you if I face any issues.
> > >
> > > Thanks again
> > >
> > > Vaibhav Garg
> > > +91-9505020924
> > > vaibhavgarg90@gmail.com
> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >
> > >
> > > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <ni...@gmail.com> wrote:
> > >
> > >> Hi, Vaibhav,
> > >>
> > >> Check the description of task.opts in the configuration doc here:
> > >>
> > >>
> >
> http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html
> > >>
> > >> Is this what you are looking for?
> > >>
> > >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <vaibhavgarg90@gmail.com
> >
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > Any help here would be great.
> > >> >
> > >> > Thanks,
> > >> > Vaibhav Garg
> > >> > +91-9505020924
> > >> > vaibhavgarg90@gmail.com
> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >> >
> > >> >
> > >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <
> > vaibhavgarg90@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Dear Bharath,
> > >> > >
> > >> > > Thanks for your reply. I now realize that I put a very ambiguous
> > >> question
> > >> > > to the community. Here is another attempt:
> > >> > >
> > >> > > I have set up a Yarn cluster that is configured to run containers
> of
> > >> > > multiple Samza jobs.
> > >> > >
> > >> > > I would like to treat all the containers of the job as a single
> > >> > > application in New Relic (as expected).
> > >> > >
> > >> > > Since the containers of the application can be killed on one Yarn
> > node
> > >> > and
> > >> > > can start on another Yarn node, I would have to specify New Relic
> > >> > > environment settings at the container start.
> > >> > >
> > >> > > Now, I am not sure how to specify java agent (to include New Relic
> > >> java
> > >> > > agent) and pass additional arguments such as New Relic environment
> > >> every
> > >> > > time the container starts.
> > >> > >
> > >> > > Please let me know if my understanding is wrong or some other
> > >> > > clarification is needed.
> > >> > >
> > >> > > Thanks in advance,
> > >> > > Vaibhav Garg
> > >> > > +91-9505020924
> > >> > > vaibhavgarg90@gmail.com
> > >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >> > >
> > >> > >
> > >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
> > >> > > codin.martial@gmail.com> wrote:
> > >> > >
> > >> > >> Hi,
> > >> > >>
> > >> > >> I am not sure I fully understand the ask.
> > >> > >>
> > >> > >> IIUC, Samza doesn't have native integration with New relic.
> > However,
> > >> you
> > >> > >> should still be able to integrate your application with New relic
> > on
> > >> > your
> > >> > >> end without native support.
> > >> > >> If you are particularly looking to integrate native Samza metrics
> > w/
> > >> New
> > >> > >> relic, you might need to implement your own custom metrics
> > reporter.
> > >> You
> > >> > >> can find more details here
> > >> > >> <
> > >> > >>
> > >> >
> > >>
> >
> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter
> > >> > >> >
> > >> > >>
> > >> > >> Thanks,
> > >> > >> Bharath
> > >> > >>
> > >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <
> > >> vaibhavgarg90@gmail.com>
> > >> > >> wrote:
> > >> > >>
> > >> > >> > Hi,
> > >> > >> >
> > >> > >> > I want to integrate New Relic in my Samza jobs. Can anyone help
> > >> with
> > >> > >> this,
> > >> > >> > please?
> > >> > >> >
> > >> > >> > Thanks,
> > >> > >> > Vaibhav Garg
> > >> > >> > +91-9505020924
> > >> > >> > vaibhavgarg90@gmail.com
> > >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >> > >> >
> > >> > >>
> > >> > >
> > >> >
> > >>
> > >
> >
>
Re: Samza - New Relic integration
Posted by Yi Pan <ni...@gmail.com>.
Hi, Vaibhav,
Not quite sure whether I understand your ask. How do you package and
distribute NewRelic jar in the YARN cluster? It would be much helpful if
you can share your job configure that worked and the places that you would
like to change.
Best,
-Yi
On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg <va...@gmail.com>
wrote:
> OK. I have been able to use NewRelic by specifying -javaagent option in
> task.opts.
>
> However, I had to give the absolute path to the NewRelic jar file in order
> to make it work.
>
> How can I give a path that is relative to the container since I would like
> to integrate NewRelic dependencies along with other container files?
>
> Thanks,
> Vaibhav Garg
> +91-9505020924
> vaibhavgarg90@gmail.com
> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>
>
> On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <va...@gmail.com>
> wrote:
>
> > Yes. That's exactly what I am looking for.
> >
> > Let me try it out and I will get back to you if I face any issues.
> >
> > Thanks again
> >
> > Vaibhav Garg
> > +91-9505020924
> > vaibhavgarg90@gmail.com
> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >
> >
> > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <ni...@gmail.com> wrote:
> >
> >> Hi, Vaibhav,
> >>
> >> Check the description of task.opts in the configuration doc here:
> >>
> >>
> http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html
> >>
> >> Is this what you are looking for?
> >>
> >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <va...@gmail.com>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > Any help here would be great.
> >> >
> >> > Thanks,
> >> > Vaibhav Garg
> >> > +91-9505020924
> >> > vaibhavgarg90@gmail.com
> >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> >
> >> >
> >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <
> vaibhavgarg90@gmail.com>
> >> > wrote:
> >> >
> >> > > Dear Bharath,
> >> > >
> >> > > Thanks for your reply. I now realize that I put a very ambiguous
> >> question
> >> > > to the community. Here is another attempt:
> >> > >
> >> > > I have set up a Yarn cluster that is configured to run containers of
> >> > > multiple Samza jobs.
> >> > >
> >> > > I would like to treat all the containers of the job as a single
> >> > > application in New Relic (as expected).
> >> > >
> >> > > Since the containers of the application can be killed on one Yarn
> node
> >> > and
> >> > > can start on another Yarn node, I would have to specify New Relic
> >> > > environment settings at the container start.
> >> > >
> >> > > Now, I am not sure how to specify java agent (to include New Relic
> >> java
> >> > > agent) and pass additional arguments such as New Relic environment
> >> every
> >> > > time the container starts.
> >> > >
> >> > > Please let me know if my understanding is wrong or some other
> >> > > clarification is needed.
> >> > >
> >> > > Thanks in advance,
> >> > > Vaibhav Garg
> >> > > +91-9505020924
> >> > > vaibhavgarg90@gmail.com
> >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> > >
> >> > >
> >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
> >> > > codin.martial@gmail.com> wrote:
> >> > >
> >> > >> Hi,
> >> > >>
> >> > >> I am not sure I fully understand the ask.
> >> > >>
> >> > >> IIUC, Samza doesn't have native integration with New relic.
> However,
> >> you
> >> > >> should still be able to integrate your application with New relic
> on
> >> > your
> >> > >> end without native support.
> >> > >> If you are particularly looking to integrate native Samza metrics
> w/
> >> New
> >> > >> relic, you might need to implement your own custom metrics
> reporter.
> >> You
> >> > >> can find more details here
> >> > >> <
> >> > >>
> >> >
> >>
> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter
> >> > >> >
> >> > >>
> >> > >> Thanks,
> >> > >> Bharath
> >> > >>
> >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <
> >> vaibhavgarg90@gmail.com>
> >> > >> wrote:
> >> > >>
> >> > >> > Hi,
> >> > >> >
> >> > >> > I want to integrate New Relic in my Samza jobs. Can anyone help
> >> with
> >> > >> this,
> >> > >> > please?
> >> > >> >
> >> > >> > Thanks,
> >> > >> > Vaibhav Garg
> >> > >> > +91-9505020924
> >> > >> > vaibhavgarg90@gmail.com
> >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> > >> >
> >> > >>
> >> > >
> >> >
> >>
> >
>
Re: Samza - New Relic integration
Posted by Vaibhav Garg <va...@gmail.com>.
OK. I have been able to use NewRelic by specifying -javaagent option in
task.opts.
However, I had to give the absolute path to the NewRelic jar file in order
to make it work.
How can I give a path that is relative to the container since I would like
to integrate NewRelic dependencies along with other container files?
Thanks,
Vaibhav Garg
+91-9505020924
vaibhavgarg90@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <va...@gmail.com>
wrote:
> Yes. That's exactly what I am looking for.
>
> Let me try it out and I will get back to you if I face any issues.
>
> Thanks again
>
> Vaibhav Garg
> +91-9505020924
> vaibhavgarg90@gmail.com
> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>
>
> On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <ni...@gmail.com> wrote:
>
>> Hi, Vaibhav,
>>
>> Check the description of task.opts in the configuration doc here:
>>
>> http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html
>>
>> Is this what you are looking for?
>>
>> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <va...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > Any help here would be great.
>> >
>> > Thanks,
>> > Vaibhav Garg
>> > +91-9505020924
>> > vaibhavgarg90@gmail.com
>> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> >
>> >
>> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <va...@gmail.com>
>> > wrote:
>> >
>> > > Dear Bharath,
>> > >
>> > > Thanks for your reply. I now realize that I put a very ambiguous
>> question
>> > > to the community. Here is another attempt:
>> > >
>> > > I have set up a Yarn cluster that is configured to run containers of
>> > > multiple Samza jobs.
>> > >
>> > > I would like to treat all the containers of the job as a single
>> > > application in New Relic (as expected).
>> > >
>> > > Since the containers of the application can be killed on one Yarn node
>> > and
>> > > can start on another Yarn node, I would have to specify New Relic
>> > > environment settings at the container start.
>> > >
>> > > Now, I am not sure how to specify java agent (to include New Relic
>> java
>> > > agent) and pass additional arguments such as New Relic environment
>> every
>> > > time the container starts.
>> > >
>> > > Please let me know if my understanding is wrong or some other
>> > > clarification is needed.
>> > >
>> > > Thanks in advance,
>> > > Vaibhav Garg
>> > > +91-9505020924
>> > > vaibhavgarg90@gmail.com
>> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >
>> > >
>> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
>> > > codin.martial@gmail.com> wrote:
>> > >
>> > >> Hi,
>> > >>
>> > >> I am not sure I fully understand the ask.
>> > >>
>> > >> IIUC, Samza doesn't have native integration with New relic. However,
>> you
>> > >> should still be able to integrate your application with New relic on
>> > your
>> > >> end without native support.
>> > >> If you are particularly looking to integrate native Samza metrics w/
>> New
>> > >> relic, you might need to implement your own custom metrics reporter.
>> You
>> > >> can find more details here
>> > >> <
>> > >>
>> >
>> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter
>> > >> >
>> > >>
>> > >> Thanks,
>> > >> Bharath
>> > >>
>> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <
>> vaibhavgarg90@gmail.com>
>> > >> wrote:
>> > >>
>> > >> > Hi,
>> > >> >
>> > >> > I want to integrate New Relic in my Samza jobs. Can anyone help
>> with
>> > >> this,
>> > >> > please?
>> > >> >
>> > >> > Thanks,
>> > >> > Vaibhav Garg
>> > >> > +91-9505020924
>> > >> > vaibhavgarg90@gmail.com
>> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> > >> >
>> > >>
>> > >
>> >
>>
>
Re: Samza - New Relic integration
Posted by Vaibhav Garg <va...@gmail.com>.
Yes. That's exactly what I am looking for.
Let me try it out and I will get back to you if I face any issues.
Thanks again
Vaibhav Garg
+91-9505020924
vaibhavgarg90@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <ni...@gmail.com> wrote:
> Hi, Vaibhav,
>
> Check the description of task.opts in the configuration doc here:
>
> http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html
>
> Is this what you are looking for?
>
> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <va...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Any help here would be great.
> >
> > Thanks,
> > Vaibhav Garg
> > +91-9505020924
> > vaibhavgarg90@gmail.com
> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >
> >
> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <va...@gmail.com>
> > wrote:
> >
> > > Dear Bharath,
> > >
> > > Thanks for your reply. I now realize that I put a very ambiguous
> question
> > > to the community. Here is another attempt:
> > >
> > > I have set up a Yarn cluster that is configured to run containers of
> > > multiple Samza jobs.
> > >
> > > I would like to treat all the containers of the job as a single
> > > application in New Relic (as expected).
> > >
> > > Since the containers of the application can be killed on one Yarn node
> > and
> > > can start on another Yarn node, I would have to specify New Relic
> > > environment settings at the container start.
> > >
> > > Now, I am not sure how to specify java agent (to include New Relic java
> > > agent) and pass additional arguments such as New Relic environment
> every
> > > time the container starts.
> > >
> > > Please let me know if my understanding is wrong or some other
> > > clarification is needed.
> > >
> > > Thanks in advance,
> > > Vaibhav Garg
> > > +91-9505020924
> > > vaibhavgarg90@gmail.com
> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >
> > >
> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
> > > codin.martial@gmail.com> wrote:
> > >
> > >> Hi,
> > >>
> > >> I am not sure I fully understand the ask.
> > >>
> > >> IIUC, Samza doesn't have native integration with New relic. However,
> you
> > >> should still be able to integrate your application with New relic on
> > your
> > >> end without native support.
> > >> If you are particularly looking to integrate native Samza metrics w/
> New
> > >> relic, you might need to implement your own custom metrics reporter.
> You
> > >> can find more details here
> > >> <
> > >>
> >
> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter
> > >> >
> > >>
> > >> Thanks,
> > >> Bharath
> > >>
> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <vaibhavgarg90@gmail.com
> >
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > I want to integrate New Relic in my Samza jobs. Can anyone help with
> > >> this,
> > >> > please?
> > >> >
> > >> > Thanks,
> > >> > Vaibhav Garg
> > >> > +91-9505020924
> > >> > vaibhavgarg90@gmail.com
> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >> >
> > >>
> > >
> >
>
Re: Samza - New Relic integration
Posted by Yi Pan <ni...@gmail.com>.
Hi, Vaibhav,
Check the description of task.opts in the configuration doc here:
http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html
Is this what you are looking for?
On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <va...@gmail.com>
wrote:
> Hi,
>
> Any help here would be great.
>
> Thanks,
> Vaibhav Garg
> +91-9505020924
> vaibhavgarg90@gmail.com
> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>
>
> On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <va...@gmail.com>
> wrote:
>
> > Dear Bharath,
> >
> > Thanks for your reply. I now realize that I put a very ambiguous question
> > to the community. Here is another attempt:
> >
> > I have set up a Yarn cluster that is configured to run containers of
> > multiple Samza jobs.
> >
> > I would like to treat all the containers of the job as a single
> > application in New Relic (as expected).
> >
> > Since the containers of the application can be killed on one Yarn node
> and
> > can start on another Yarn node, I would have to specify New Relic
> > environment settings at the container start.
> >
> > Now, I am not sure how to specify java agent (to include New Relic java
> > agent) and pass additional arguments such as New Relic environment every
> > time the container starts.
> >
> > Please let me know if my understanding is wrong or some other
> > clarification is needed.
> >
> > Thanks in advance,
> > Vaibhav Garg
> > +91-9505020924
> > vaibhavgarg90@gmail.com
> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >
> >
> > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
> > codin.martial@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> I am not sure I fully understand the ask.
> >>
> >> IIUC, Samza doesn't have native integration with New relic. However, you
> >> should still be able to integrate your application with New relic on
> your
> >> end without native support.
> >> If you are particularly looking to integrate native Samza metrics w/ New
> >> relic, you might need to implement your own custom metrics reporter. You
> >> can find more details here
> >> <
> >>
> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter
> >> >
> >>
> >> Thanks,
> >> Bharath
> >>
> >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <va...@gmail.com>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I want to integrate New Relic in my Samza jobs. Can anyone help with
> >> this,
> >> > please?
> >> >
> >> > Thanks,
> >> > Vaibhav Garg
> >> > +91-9505020924
> >> > vaibhavgarg90@gmail.com
> >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >> >
> >>
> >
>
Re: Samza - New Relic integration
Posted by Vaibhav Garg <va...@gmail.com>.
Hi,
Any help here would be great.
Thanks,
Vaibhav Garg
+91-9505020924
vaibhavgarg90@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <va...@gmail.com>
wrote:
> Dear Bharath,
>
> Thanks for your reply. I now realize that I put a very ambiguous question
> to the community. Here is another attempt:
>
> I have set up a Yarn cluster that is configured to run containers of
> multiple Samza jobs.
>
> I would like to treat all the containers of the job as a single
> application in New Relic (as expected).
>
> Since the containers of the application can be killed on one Yarn node and
> can start on another Yarn node, I would have to specify New Relic
> environment settings at the container start.
>
> Now, I am not sure how to specify java agent (to include New Relic java
> agent) and pass additional arguments such as New Relic environment every
> time the container starts.
>
> Please let me know if my understanding is wrong or some other
> clarification is needed.
>
> Thanks in advance,
> Vaibhav Garg
> +91-9505020924
> vaibhavgarg90@gmail.com
> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>
>
> On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
> codin.martial@gmail.com> wrote:
>
>> Hi,
>>
>> I am not sure I fully understand the ask.
>>
>> IIUC, Samza doesn't have native integration with New relic. However, you
>> should still be able to integrate your application with New relic on your
>> end without native support.
>> If you are particularly looking to integrate native Samza metrics w/ New
>> relic, you might need to implement your own custom metrics reporter. You
>> can find more details here
>> <
>> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter
>> >
>>
>> Thanks,
>> Bharath
>>
>> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <va...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I want to integrate New Relic in my Samza jobs. Can anyone help with
>> this,
>> > please?
>> >
>> > Thanks,
>> > Vaibhav Garg
>> > +91-9505020924
>> > vaibhavgarg90@gmail.com
>> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>> >
>>
>
Re: Samza - New Relic integration
Posted by Vaibhav Garg <va...@gmail.com>.
Dear Bharath,
Thanks for your reply. I now realize that I put a very ambiguous question
to the community. Here is another attempt:
I have set up a Yarn cluster that is configured to run containers of
multiple Samza jobs.
I would like to treat all the containers of the job as a single application
in New Relic (as expected).
Since the containers of the application can be killed on one Yarn node and
can start on another Yarn node, I would have to specify New Relic
environment settings at the container start.
Now, I am not sure how to specify java agent (to include New Relic java
agent) and pass additional arguments such as New Relic environment every
time the container starts.
Please let me know if my understanding is wrong or some other clarification
is needed.
Thanks in advance,
Vaibhav Garg
+91-9505020924
vaibhavgarg90@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
codin.martial@gmail.com> wrote:
> Hi,
>
> I am not sure I fully understand the ask.
>
> IIUC, Samza doesn't have native integration with New relic. However, you
> should still be able to integrate your application with New relic on your
> end without native support.
> If you are particularly looking to integrate native Samza metrics w/ New
> relic, you might need to implement your own custom metrics reporter. You
> can find more details here
> <
> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter
> >
>
> Thanks,
> Bharath
>
> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <va...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I want to integrate New Relic in my Samza jobs. Can anyone help with
> this,
> > please?
> >
> > Thanks,
> > Vaibhav Garg
> > +91-9505020924
> > vaibhavgarg90@gmail.com
> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >
>
Re: Samza - New Relic integration
Posted by Bharath Kumara Subramanian <co...@gmail.com>.
Hi,
I am not sure I fully understand the ask.
IIUC, Samza doesn't have native integration with New relic. However, you
should still be able to integrate your application with New relic on your
end without native support.
If you are particularly looking to integrate native Samza metrics w/ New
relic, you might need to implement your own custom metrics reporter. You
can find more details here
<https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter>
Thanks,
Bharath
On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <va...@gmail.com>
wrote:
> Hi,
>
> I want to integrate New Relic in my Samza jobs. Can anyone help with this,
> please?
>
> Thanks,
> Vaibhav Garg
> +91-9505020924
> vaibhavgarg90@gmail.com
> LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
>