You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sirona.apache.org by Otis Gospodnetic <ot...@gmail.com> on 2014/06/07 15:42:45 UTC
Sirona roadmap: automatic bytecode instrumentation?
Hi,
Quick intro for context. I'm an Apache committer/member of many years,
used to be more actively involved with Lucene and Solr. Now working on SPM
<http://sematext.com/spm/>, a performance monitoring tool.
I've been aware of Sirona for the last few months, but I'm still not 100%
sure what it does or, rather, what the plan/roadmap/direction is. More
specifically, what I am wondering is if the plan is to have Sirona be
capable of *automatic* JVM instrumentation in order to do bytecode
injection around specific key methods in an array of popular frameworks and
libraries, for purposes of being able to trace call execution, timing
execution of each step, and thus being able to provide information about
which component/class/method is slow?
Is that Sirona's direction?
I see there is ability to do some instrumentation of Spring, and the
ability for one to monitor Webapps by specifying Sirona's Servlet Filter in
web.xml, as well as monitoring JDBC by replacing the JDBC driver with
Sirona's.....
... but this all requires one to change the config and explicitly refer to
Sirona's classes, jars, etc.
What I'm wondering is whether the plan is for Sirona to be run as
"javaagent" and *automatically* inject its bytecode for
service/class/method call counting and timing purposes without requiring
code or config modifications?
Thanks,
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
Re: Sirona roadmap: automatic bytecode instrumentation?
Posted by Olivier Lamy <ol...@apache.org>.
Hi,
Apologize for such delayed response. My really bad!!!
On 10 June 2014 23:24, Otis Gospodnetic <ot...@gmail.com> wrote:
> Hi,
>
> On Mon, Jun 9, 2014 at 8:34 PM, Olivier Lamy <ol...@apache.org> wrote:
>
>> On 10 June 2014 07:53, Otis Gospodnetic <ot...@gmail.com>
>> wrote:
>> > Hi Romain,
>> >
>> > Path tracking - that's great to hear. It looks like this is limited to
>> > Java webapps, right? Do you plan on working on RUM and capturing things
>> > starting from that point and tracking the transaction all the way down?
>> Or
>> > is the plan to always capture things on the server side?
>> >
>>
>> It's not limited to webapp as it's a javaagent.
>>
>
> Right, sorry, I think I meant to say "Java webapp", not just "webapp". For
> example, if my webapp is written in PHP, can Sirona help me today? I think
> it can't, but I could be missing what it can do today or what is planned
> for the (near) future.
No can't really help today except boomerang plugin
>
>> This doesn't seem to be in 0.1, but is in 0.2, which should be released
>> > soon?
>>
>> I hope released soon but it's always a pain to get binding votes here
>> so I have no idea when it will be released.
>> Currently the ui for path tracking has not been done.
>>
>> >
>> > Do you know if anyone is using this in production yet?
>>
>> To be honest no :-) This need some performance improvment.
>>
>
> Right, I thoughts so, but wanted to check anyway. :) It's young, so this
> is completely expected.
>
>>
>> > Do you plan on instrumenting other methods to track, say, JDBC calls
>> > without requiring that one replaces their driver with Sirona's?
>>
>> Can be done using the path tracking with black/white list.
>>
>
> Are there any examples of that in the code, or tests or javadocs or ..... ?
> I couldn't find much about this.
> Any pointers would be great to have!
>
>> I looked at the code relatively briefly..... where does the data currently
>> > get stored? Cassandra? Is the data storage pluggable by any chance?
>> We'd
>> > be interested in writing data to HBase and/or Elasticsearch, for example.
>>
>> Yup that is definitely pluggable. I had some plan to do an
>> ElasticSearch impl but didn't have time for that.
>>
>
> Would one have to implement all classes (13, I think) under
> https://github.com/apache/sirona/tree/c915578939fd32b20b1878d7d5f64335b8458750/server/store/cassandra/src/main/java/org/apache/sirona/cassandra
> ?
Not really just check o.a.s.c.collector package. And it depends what
sort of collector you want to implement. If all yes will be all
subpackages
If only the path tracking part just check the pathtracking sub package.
>
> Thanks,
> Otis
>
>
>
>>
>> > On Sat, Jun 7, 2014 at 1:19 PM, Romain Manni-Bucau <
>> rmannibucau@gmail.com>
>> > wrote:
>> >
>> >> Hi
>> >>
>> >> in one word: yes.
>> >>
>> >> Actually sirona already has a javaagent and can even instrument
>> >> HttpUrlConnection of the JVM. This will be useful for Path Tracking we
>> are
>> >> working on.
>> >>
>> >>
>> >>
>> >>
>> >> Romain Manni-Bucau
>> >> Twitter: @rmannibucau
>> >> Blog: http://rmannibucau.wordpress.com/
>> >> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>> >> Github: https://github.com/rmannibucau
>> >>
>> >>
>> >> 2014-06-07 15:42 GMT+02:00 Otis Gospodnetic <otis.gospodnetic@gmail.com
>> >:
>> >>
>> >> > Hi,
>> >> >
>> >> > Quick intro for context. I'm an Apache committer/member of many
>> years,
>> >> > used to be more actively involved with Lucene and Solr. Now working
>> on
>> >> SPM
>> >> > <http://sematext.com/spm/>, a performance monitoring tool.
>> >> >
>> >> > I've been aware of Sirona for the last few months, but I'm still not
>> 100%
>> >> > sure what it does or, rather, what the plan/roadmap/direction is.
>> More
>> >> > specifically, what I am wondering is if the plan is to have Sirona be
>> >> > capable of *automatic* JVM instrumentation in order to do bytecode
>> >> > injection around specific key methods in an array of popular
>> frameworks
>> >> and
>> >> > libraries, for purposes of being able to trace call execution, timing
>> >> > execution of each step, and thus being able to provide information
>> about
>> >> > which component/class/method is slow?
>> >> >
>> >> > Is that Sirona's direction?
>> >> >
>> >> > I see there is ability to do some instrumentation of Spring, and the
>> >> > ability for one to monitor Webapps by specifying Sirona's Servlet
>> Filter
>> >> in
>> >> > web.xml, as well as monitoring JDBC by replacing the JDBC driver with
>> >> > Sirona's.....
>> >> >
>> >> > ... but this all requires one to change the config and explicitly
>> refer
>> >> to
>> >> > Sirona's classes, jars, etc.
>> >> >
>> >> > What I'm wondering is whether the plan is for Sirona to be run as
>> >> > "javaagent" and *automatically* inject its bytecode for
>> >> > service/class/method call counting and timing purposes without
>> requiring
>> >> > code or config modifications?
>> >> >
>> >> > Thanks,
>> >> > Otis
>> >> > --
>> >> > Performance Monitoring * Log Analytics * Search Analytics
>> >> > Solr & Elasticsearch Support * http://sematext.com/
>> >> >
>> >>
>>
>>
>>
>> --
>> Olivier Lamy
>> Ecetera: http://ecetera.com.au
>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>
--
Olivier Lamy
Ecetera: http://ecetera.com.au
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Sirona roadmap: automatic bytecode instrumentation?
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
On Mon, Jun 9, 2014 at 8:34 PM, Olivier Lamy <ol...@apache.org> wrote:
> On 10 June 2014 07:53, Otis Gospodnetic <ot...@gmail.com>
> wrote:
> > Hi Romain,
> >
> > Path tracking - that's great to hear. It looks like this is limited to
> > Java webapps, right? Do you plan on working on RUM and capturing things
> > starting from that point and tracking the transaction all the way down?
> Or
> > is the plan to always capture things on the server side?
> >
>
> It's not limited to webapp as it's a javaagent.
>
Right, sorry, I think I meant to say "Java webapp", not just "webapp". For
example, if my webapp is written in PHP, can Sirona help me today? I think
it can't, but I could be missing what it can do today or what is planned
for the (near) future.
> This doesn't seem to be in 0.1, but is in 0.2, which should be released
> > soon?
>
> I hope released soon but it's always a pain to get binding votes here
> so I have no idea when it will be released.
> Currently the ui for path tracking has not been done.
>
> >
> > Do you know if anyone is using this in production yet?
>
> To be honest no :-) This need some performance improvment.
>
Right, I thoughts so, but wanted to check anyway. :) It's young, so this
is completely expected.
>
> > Do you plan on instrumenting other methods to track, say, JDBC calls
> > without requiring that one replaces their driver with Sirona's?
>
> Can be done using the path tracking with black/white list.
>
Are there any examples of that in the code, or tests or javadocs or ..... ?
I couldn't find much about this.
Any pointers would be great to have!
> I looked at the code relatively briefly..... where does the data currently
> > get stored? Cassandra? Is the data storage pluggable by any chance?
> We'd
> > be interested in writing data to HBase and/or Elasticsearch, for example.
>
> Yup that is definitely pluggable. I had some plan to do an
> ElasticSearch impl but didn't have time for that.
>
Would one have to implement all classes (13, I think) under
https://github.com/apache/sirona/tree/c915578939fd32b20b1878d7d5f64335b8458750/server/store/cassandra/src/main/java/org/apache/sirona/cassandra
?
Thanks,
Otis
>
> > On Sat, Jun 7, 2014 at 1:19 PM, Romain Manni-Bucau <
> rmannibucau@gmail.com>
> > wrote:
> >
> >> Hi
> >>
> >> in one word: yes.
> >>
> >> Actually sirona already has a javaagent and can even instrument
> >> HttpUrlConnection of the JVM. This will be useful for Path Tracking we
> are
> >> working on.
> >>
> >>
> >>
> >>
> >> Romain Manni-Bucau
> >> Twitter: @rmannibucau
> >> Blog: http://rmannibucau.wordpress.com/
> >> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> >> Github: https://github.com/rmannibucau
> >>
> >>
> >> 2014-06-07 15:42 GMT+02:00 Otis Gospodnetic <otis.gospodnetic@gmail.com
> >:
> >>
> >> > Hi,
> >> >
> >> > Quick intro for context. I'm an Apache committer/member of many
> years,
> >> > used to be more actively involved with Lucene and Solr. Now working
> on
> >> SPM
> >> > <http://sematext.com/spm/>, a performance monitoring tool.
> >> >
> >> > I've been aware of Sirona for the last few months, but I'm still not
> 100%
> >> > sure what it does or, rather, what the plan/roadmap/direction is.
> More
> >> > specifically, what I am wondering is if the plan is to have Sirona be
> >> > capable of *automatic* JVM instrumentation in order to do bytecode
> >> > injection around specific key methods in an array of popular
> frameworks
> >> and
> >> > libraries, for purposes of being able to trace call execution, timing
> >> > execution of each step, and thus being able to provide information
> about
> >> > which component/class/method is slow?
> >> >
> >> > Is that Sirona's direction?
> >> >
> >> > I see there is ability to do some instrumentation of Spring, and the
> >> > ability for one to monitor Webapps by specifying Sirona's Servlet
> Filter
> >> in
> >> > web.xml, as well as monitoring JDBC by replacing the JDBC driver with
> >> > Sirona's.....
> >> >
> >> > ... but this all requires one to change the config and explicitly
> refer
> >> to
> >> > Sirona's classes, jars, etc.
> >> >
> >> > What I'm wondering is whether the plan is for Sirona to be run as
> >> > "javaagent" and *automatically* inject its bytecode for
> >> > service/class/method call counting and timing purposes without
> requiring
> >> > code or config modifications?
> >> >
> >> > Thanks,
> >> > Otis
> >> > --
> >> > Performance Monitoring * Log Analytics * Search Analytics
> >> > Solr & Elasticsearch Support * http://sematext.com/
> >> >
> >>
>
>
>
> --
> Olivier Lamy
> Ecetera: http://ecetera.com.au
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
Re: Sirona roadmap: automatic bytecode instrumentation?
Posted by Olivier Lamy <ol...@apache.org>.
On 10 June 2014 07:53, Otis Gospodnetic <ot...@gmail.com> wrote:
> Hi Romain,
>
> Path tracking - that's great to hear. It looks like this is limited to
> Java webapps, right? Do you plan on working on RUM and capturing things
> starting from that point and tracking the transaction all the way down? Or
> is the plan to always capture things on the server side?
>
It's not limited to webapp as it's a javaagent.
> This doesn't seem to be in 0.1, but is in 0.2, which should be released
> soon?
I hope released soon but it's always a pain to get binding votes here
so I have no idea when it will be released.
Currently the ui for path tracking has not been done.
>
> Do you know if anyone is using this in production yet?
To be honest no :-) This need some performance improvment.
>
> Do you plan on instrumenting other methods to track, say, JDBC calls
> without requiring that one replaces their driver with Sirona's?
Can be done using the path tracking with black/white list.
>
>
> I looked at the code relatively briefly..... where does the data currently
> get stored? Cassandra? Is the data storage pluggable by any chance? We'd
> be interested in writing data to HBase and/or Elasticsearch, for example.
Yup that is definitely pluggable. I had some plan to do an
ElasticSearch impl but didn't have time for that.
>
> Thanks,
> Otis
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>
>
> On Sat, Jun 7, 2014 at 1:19 PM, Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> Hi
>>
>> in one word: yes.
>>
>> Actually sirona already has a javaagent and can even instrument
>> HttpUrlConnection of the JVM. This will be useful for Path Tracking we are
>> working on.
>>
>>
>>
>>
>> Romain Manni-Bucau
>> Twitter: @rmannibucau
>> Blog: http://rmannibucau.wordpress.com/
>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>> Github: https://github.com/rmannibucau
>>
>>
>> 2014-06-07 15:42 GMT+02:00 Otis Gospodnetic <ot...@gmail.com>:
>>
>> > Hi,
>> >
>> > Quick intro for context. I'm an Apache committer/member of many years,
>> > used to be more actively involved with Lucene and Solr. Now working on
>> SPM
>> > <http://sematext.com/spm/>, a performance monitoring tool.
>> >
>> > I've been aware of Sirona for the last few months, but I'm still not 100%
>> > sure what it does or, rather, what the plan/roadmap/direction is. More
>> > specifically, what I am wondering is if the plan is to have Sirona be
>> > capable of *automatic* JVM instrumentation in order to do bytecode
>> > injection around specific key methods in an array of popular frameworks
>> and
>> > libraries, for purposes of being able to trace call execution, timing
>> > execution of each step, and thus being able to provide information about
>> > which component/class/method is slow?
>> >
>> > Is that Sirona's direction?
>> >
>> > I see there is ability to do some instrumentation of Spring, and the
>> > ability for one to monitor Webapps by specifying Sirona's Servlet Filter
>> in
>> > web.xml, as well as monitoring JDBC by replacing the JDBC driver with
>> > Sirona's.....
>> >
>> > ... but this all requires one to change the config and explicitly refer
>> to
>> > Sirona's classes, jars, etc.
>> >
>> > What I'm wondering is whether the plan is for Sirona to be run as
>> > "javaagent" and *automatically* inject its bytecode for
>> > service/class/method call counting and timing purposes without requiring
>> > code or config modifications?
>> >
>> > Thanks,
>> > Otis
>> > --
>> > Performance Monitoring * Log Analytics * Search Analytics
>> > Solr & Elasticsearch Support * http://sematext.com/
>> >
>>
--
Olivier Lamy
Ecetera: http://ecetera.com.au
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Sirona roadmap: automatic bytecode instrumentation?
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi Romain,
Path tracking - that's great to hear. It looks like this is limited to
Java webapps, right? Do you plan on working on RUM and capturing things
starting from that point and tracking the transaction all the way down? Or
is the plan to always capture things on the server side?
This doesn't seem to be in 0.1, but is in 0.2, which should be released
soon?
Do you know if anyone is using this in production yet?
Do you plan on instrumenting other methods to track, say, JDBC calls
without requiring that one replaces their driver with Sirona's?
I looked at the code relatively briefly..... where does the data currently
get stored? Cassandra? Is the data storage pluggable by any chance? We'd
be interested in writing data to HBase and/or Elasticsearch, for example.
Thanks,
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
On Sat, Jun 7, 2014 at 1:19 PM, Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Hi
>
> in one word: yes.
>
> Actually sirona already has a javaagent and can even instrument
> HttpUrlConnection of the JVM. This will be useful for Path Tracking we are
> working on.
>
>
>
>
> Romain Manni-Bucau
> Twitter: @rmannibucau
> Blog: http://rmannibucau.wordpress.com/
> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> Github: https://github.com/rmannibucau
>
>
> 2014-06-07 15:42 GMT+02:00 Otis Gospodnetic <ot...@gmail.com>:
>
> > Hi,
> >
> > Quick intro for context. I'm an Apache committer/member of many years,
> > used to be more actively involved with Lucene and Solr. Now working on
> SPM
> > <http://sematext.com/spm/>, a performance monitoring tool.
> >
> > I've been aware of Sirona for the last few months, but I'm still not 100%
> > sure what it does or, rather, what the plan/roadmap/direction is. More
> > specifically, what I am wondering is if the plan is to have Sirona be
> > capable of *automatic* JVM instrumentation in order to do bytecode
> > injection around specific key methods in an array of popular frameworks
> and
> > libraries, for purposes of being able to trace call execution, timing
> > execution of each step, and thus being able to provide information about
> > which component/class/method is slow?
> >
> > Is that Sirona's direction?
> >
> > I see there is ability to do some instrumentation of Spring, and the
> > ability for one to monitor Webapps by specifying Sirona's Servlet Filter
> in
> > web.xml, as well as monitoring JDBC by replacing the JDBC driver with
> > Sirona's.....
> >
> > ... but this all requires one to change the config and explicitly refer
> to
> > Sirona's classes, jars, etc.
> >
> > What I'm wondering is whether the plan is for Sirona to be run as
> > "javaagent" and *automatically* inject its bytecode for
> > service/class/method call counting and timing purposes without requiring
> > code or config modifications?
> >
> > Thanks,
> > Otis
> > --
> > Performance Monitoring * Log Analytics * Search Analytics
> > Solr & Elasticsearch Support * http://sematext.com/
> >
>
Re: Sirona roadmap: automatic bytecode instrumentation?
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi
in one word: yes.
Actually sirona already has a javaagent and can even instrument
HttpUrlConnection of the JVM. This will be useful for Path Tracking we are
working on.
Romain Manni-Bucau
Twitter: @rmannibucau
Blog: http://rmannibucau.wordpress.com/
LinkedIn: http://fr.linkedin.com/in/rmannibucau
Github: https://github.com/rmannibucau
2014-06-07 15:42 GMT+02:00 Otis Gospodnetic <ot...@gmail.com>:
> Hi,
>
> Quick intro for context. I'm an Apache committer/member of many years,
> used to be more actively involved with Lucene and Solr. Now working on SPM
> <http://sematext.com/spm/>, a performance monitoring tool.
>
> I've been aware of Sirona for the last few months, but I'm still not 100%
> sure what it does or, rather, what the plan/roadmap/direction is. More
> specifically, what I am wondering is if the plan is to have Sirona be
> capable of *automatic* JVM instrumentation in order to do bytecode
> injection around specific key methods in an array of popular frameworks and
> libraries, for purposes of being able to trace call execution, timing
> execution of each step, and thus being able to provide information about
> which component/class/method is slow?
>
> Is that Sirona's direction?
>
> I see there is ability to do some instrumentation of Spring, and the
> ability for one to monitor Webapps by specifying Sirona's Servlet Filter in
> web.xml, as well as monitoring JDBC by replacing the JDBC driver with
> Sirona's.....
>
> ... but this all requires one to change the config and explicitly refer to
> Sirona's classes, jars, etc.
>
> What I'm wondering is whether the plan is for Sirona to be run as
> "javaagent" and *automatically* inject its bytecode for
> service/class/method call counting and timing purposes without requiring
> code or config modifications?
>
> Thanks,
> Otis
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>