You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by Avery Ching <av...@gmail.com> on 2013/04/08 19:27:51 UTC
Re: Review Request: YourKit profiling API for easy profiling of giraph
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10147/#review18781
-----------------------------------------------------------
Overall, I think it's great that we make toolkits like Yourkit easy to use in Giraph and this is a good first step. Personally I feel a little strange though with adding the actual logging calls in the usercode. Also, I am a bit hesitant on the maven dependencies that we will have to maintain going forward...What do others think here?
giraph-core/pom.xml
<https://reviews.apache.org/r/10147/#comment39201>
Same here?
pom.xml
<https://reviews.apache.org/r/10147/#comment39200>
Are we maintaining this package?
- Avery Ching
On March 27, 2013, 7:14 a.m., Nitay Joffe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10147/
> -----------------------------------------------------------
>
> (Updated March 27, 2013, 7:14 a.m.)
>
>
> Review request for giraph.
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/GIRAPH-592
>
>
> Diffs
> -----
>
> giraph-core/pom.xml 3580d0cd3cd943e0762b3eb069183a31fc690300
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java ffcae6e79cf0bd96745ef9bc9fc420f5b28494b2
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java 7882d06fddd70eb9fef7a8d0463c6618b39d3d4d
> giraph-core/src/main/java/org/apache/giraph/utils/YourKitContext.java PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/utils/YourKitProfiler.java PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java 74c1f874eea819b5e4fafd684a88f17eaded7dc5
> pom.xml e576e4b5e55be507225860e7d1c1fc55629f1761
>
> Diff: https://reviews.apache.org/r/10147/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Nitay Joffe
>
>
Re: Review Request: YourKit profiling API for easy profiling of giraph
Posted by Nitay Joffe <ni...@apache.org>.
> On April 8, 2013, 5:27 p.m., Avery Ching wrote:
> > Overall, I think it's great that we make toolkits like Yourkit easy to use in Giraph and this is a good first step. Personally I feel a little strange though with adding the actual logging calls in the usercode. Also, I am a bit hesitant on the maven dependencies that we will have to maintain going forward...What do others think here?
>
> Nitay Joffe wrote:
> Do you have any ideas of how to make it cleaner in terms of the adding code? The thought that I had was to be able to easily toggle profiling various blocks of code (vertex input, edge input, superstep, message passing, vertex shuffling, etc). In terms of the maven dependency - I am maintaining the com.facebook.thirdparty.yourkit-api package. It is very easy to maintain as it takes a couple of minutes to add another version of YourKit, and they don't change often. Also this jar is very minimal (it's like 6 classes) because it is only the API, and has not changed substantially over many versions. The real code gets hooked in at runtime when you run Giraph.
>
> Avery Ching wrote:
> Yeah, so I am a bit concerned about user maintained code. Does YourKit not produce their own api jars on maven? Would it be possible to hide these calls in our already existing metrics work?
I think I was unclear. These API jars are directly from YourKit. They do not deploy to maven, but on their site you can download them. For reference, here is my repository for this dependency: https://github.com/nitay/sonatype-yourkit. As you can see it is all binary, I am not maintaining their code.
We could add profiling to the metrics but I'm not sure that makes sense as I see the two as disjoint things that shouldn't effect each other? Maybe you could explain further what you envision?
- Nitay
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10147/#review18781
-----------------------------------------------------------
On March 27, 2013, 7:14 a.m., Nitay Joffe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10147/
> -----------------------------------------------------------
>
> (Updated March 27, 2013, 7:14 a.m.)
>
>
> Review request for giraph.
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/GIRAPH-592
>
>
> Diffs
> -----
>
> giraph-core/pom.xml 3580d0cd3cd943e0762b3eb069183a31fc690300
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java ffcae6e79cf0bd96745ef9bc9fc420f5b28494b2
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java 7882d06fddd70eb9fef7a8d0463c6618b39d3d4d
> giraph-core/src/main/java/org/apache/giraph/utils/YourKitContext.java PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/utils/YourKitProfiler.java PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java 74c1f874eea819b5e4fafd684a88f17eaded7dc5
> pom.xml e576e4b5e55be507225860e7d1c1fc55629f1761
>
> Diff: https://reviews.apache.org/r/10147/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Nitay Joffe
>
>
Re: Review Request: YourKit profiling API for easy profiling of giraph
Posted by Avery Ching <av...@gmail.com>.
> On April 8, 2013, 5:27 p.m., Avery Ching wrote:
> > Overall, I think it's great that we make toolkits like Yourkit easy to use in Giraph and this is a good first step. Personally I feel a little strange though with adding the actual logging calls in the usercode. Also, I am a bit hesitant on the maven dependencies that we will have to maintain going forward...What do others think here?
>
> Nitay Joffe wrote:
> Do you have any ideas of how to make it cleaner in terms of the adding code? The thought that I had was to be able to easily toggle profiling various blocks of code (vertex input, edge input, superstep, message passing, vertex shuffling, etc). In terms of the maven dependency - I am maintaining the com.facebook.thirdparty.yourkit-api package. It is very easy to maintain as it takes a couple of minutes to add another version of YourKit, and they don't change often. Also this jar is very minimal (it's like 6 classes) because it is only the API, and has not changed substantially over many versions. The real code gets hooked in at runtime when you run Giraph.
Yeah, so I am a bit concerned about user maintained code. Does YourKit not produce their own api jars on maven? Would it be possible to hide these calls in our already existing metrics work?
- Avery
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10147/#review18781
-----------------------------------------------------------
On March 27, 2013, 7:14 a.m., Nitay Joffe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10147/
> -----------------------------------------------------------
>
> (Updated March 27, 2013, 7:14 a.m.)
>
>
> Review request for giraph.
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/GIRAPH-592
>
>
> Diffs
> -----
>
> giraph-core/pom.xml 3580d0cd3cd943e0762b3eb069183a31fc690300
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java ffcae6e79cf0bd96745ef9bc9fc420f5b28494b2
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java 7882d06fddd70eb9fef7a8d0463c6618b39d3d4d
> giraph-core/src/main/java/org/apache/giraph/utils/YourKitContext.java PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/utils/YourKitProfiler.java PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java 74c1f874eea819b5e4fafd684a88f17eaded7dc5
> pom.xml e576e4b5e55be507225860e7d1c1fc55629f1761
>
> Diff: https://reviews.apache.org/r/10147/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Nitay Joffe
>
>
Re: Review Request: YourKit profiling API for easy profiling of giraph
Posted by Nitay Joffe <ni...@apache.org>.
> On April 8, 2013, 5:27 p.m., Avery Ching wrote:
> > Overall, I think it's great that we make toolkits like Yourkit easy to use in Giraph and this is a good first step. Personally I feel a little strange though with adding the actual logging calls in the usercode. Also, I am a bit hesitant on the maven dependencies that we will have to maintain going forward...What do others think here?
Do you have any ideas of how to make it cleaner in terms of the adding code? The thought that I had was to be able to easily toggle profiling various blocks of code (vertex input, edge input, superstep, message passing, vertex shuffling, etc). In terms of the maven dependency - I am maintaining the com.facebook.thirdparty.yourkit-api package. It is very easy to maintain as it takes a couple of minutes to add another version of YourKit, and they don't change often. Also this jar is very minimal (it's like 6 classes) because it is only the API, and has not changed substantially over many versions. The real code gets hooked in at runtime when you run Giraph.
- Nitay
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10147/#review18781
-----------------------------------------------------------
On March 27, 2013, 7:14 a.m., Nitay Joffe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10147/
> -----------------------------------------------------------
>
> (Updated March 27, 2013, 7:14 a.m.)
>
>
> Review request for giraph.
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/GIRAPH-592
>
>
> Diffs
> -----
>
> giraph-core/pom.xml 3580d0cd3cd943e0762b3eb069183a31fc690300
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java ffcae6e79cf0bd96745ef9bc9fc420f5b28494b2
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java 7882d06fddd70eb9fef7a8d0463c6618b39d3d4d
> giraph-core/src/main/java/org/apache/giraph/utils/YourKitContext.java PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/utils/YourKitProfiler.java PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java 74c1f874eea819b5e4fafd684a88f17eaded7dc5
> pom.xml e576e4b5e55be507225860e7d1c1fc55629f1761
>
> Diff: https://reviews.apache.org/r/10147/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Nitay Joffe
>
>