You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by Nitay Joffe <ni...@apache.org> on 2012/11/06 19:19:44 UTC
Review Request: GIRAPH-407: Update Metrics
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7900/
-----------------------------------------------------------
Review request for giraph.
Description
-------
https://issues.apache.org/jira/browse/GIRAPH-407
Updated metrics. Main logic changes are:
- Split into per-job and per-superstep metrics.
- The per-superstep metrics change their group (e.g. giraph.superstep.1.compute to giraph.superstep.2.compute) at beginning of each superstep.
- Any class using per-superstep metrics registers itself and gets a callback to reset its metrics objects on each new superstep.
- Cleanup Time hierarchy with Times class that has helper methods for all Time implementors.
By default metrics are disabled, to enable them use -Dgiraph.metrics.enable=true.
When enabled the metrics are always sent to JMX.
If you pass -Dgiraph.metrics.dump.at.end=true then this output will be produced on each worker at the end of the job: https://gist.github.com/38b8f527504f7792485c
If you pass -Dgiraph.metrics.print=true then this output will be produced on each worker at the end of each superstep: https://gist.github.com/3e3fad650e3960ad9edc
Diffs
-----
giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 8ec78cd10be41145fab50773aa6f28d43b307fdc
giraph/src/main/java/org/apache/giraph/comm/netty/ByteCounter.java 0a5f9264d0896218cfcba9d88e491bfa64ce1d5e
giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 7f5dd3f7a47b1e323784025448766af284f1af17
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java b3aec677a73312dd2725ec46bac2021bc1aff7ef
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java 6db2ae8f389083012f8a63727a3dcba2eefa884c
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java 2aa0611e485683c928113d0b3ed34830d86f5b77
giraph/src/main/java/org/apache/giraph/graph/BspServiceMaster.java b9542263aea28d8591c0bf0fdebb7731a8fa6b1e
giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 52e7006db5cc6ba1edf1737bd5bf1d0749b98ce0
giraph/src/main/java/org/apache/giraph/graph/ComputeCallable.java 4938169ec3155ec72aecd5b563e3c2717de02a3e
giraph/src/main/java/org/apache/giraph/graph/EdgeInputSplitsCallable.java 4e7e7f530b8ba1ebd0e240898f0d53201d7bc1cf
giraph/src/main/java/org/apache/giraph/graph/GraphMapper.java 92046b5f32741d4819a9c7c27bab5c8f2fe56ff5
giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallable.java 97cc222ff53e9a8a5ebbe07abdd7f44475416a59
giraph/src/main/java/org/apache/giraph/graph/MasterThread.java d5144de1a4ffc1f738c6dab0d14c5803f08d7398
giraph/src/main/java/org/apache/giraph/graph/VertexInputSplitsCallable.java 2551d4a544174d7ada9a02fc99a60ff173094bd7
giraph/src/main/java/org/apache/giraph/metrics/ClearableGauge.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/DoubleGauge.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/GiraphMetrics.java 9bb6f546d7dfb2cd97d82279d5784104b66dcacb
giraph/src/main/java/org/apache/giraph/metrics/GiraphMetricsRegistry.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/LongGauge.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/ResetSuperstepMetricsObserver.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/SuperstepMetricsRegistry.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/ValueGauge.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/utils/FakeTime.java 1f0a892140ea2502323c512be33ec0db984e16d5
giraph/src/main/java/org/apache/giraph/utils/MemoryUtils.java a6eba674e53aa71c90df8b23ffc9b9654f7244d8
giraph/src/main/java/org/apache/giraph/utils/SystemTime.java 7207d80e2404af3934e82831ba5d209eaa10a6bc
giraph/src/main/java/org/apache/giraph/utils/Time.java 1933a1de8b9ca0294d21324f4f55010df5edacb7
giraph/src/main/java/org/apache/giraph/utils/Times.java PRE-CREATION
giraph/src/test/java/org/apache/giraph/TestPredicateLock.java 73c508fcfdd0d4c73de23e736b93604d3101ae66
giraph/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java c7ced6b0e49cfb1ba4201aafd6b7be12a8d464f2
giraph/src/test/java/org/apache/giraph/utils/BspUtilsTest.java a31c2e05827fa31763c47c623574375fcf71404b
Diff: https://reviews.apache.org/r/7900/diff/
Testing
-------
Thanks,
Nitay Joffe
Re: Review Request: GIRAPH-407: Update Metrics
Posted by Avery Ching <av...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7900/#review13209
-----------------------------------------------------------
+1
Quick pass, looks mostly good. I'll let Alessandro commit when he's ready.
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java
<https://reviews.apache.org/r/7900/#comment28415>
Maybe we should keep SystemTime.getInstance() as a private static variable, similar to other classes?
- Avery Ching
On Nov. 7, 2012, 1:41 a.m., Nitay Joffe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7900/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2012, 1:41 a.m.)
>
>
> Review request for giraph.
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/GIRAPH-407
>
>
> Diffs
> -----
>
> giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 8ec78cd10be41145fab50773aa6f28d43b307fdc
> giraph/src/main/java/org/apache/giraph/comm/netty/ByteCounter.java 0a5f9264d0896218cfcba9d88e491bfa64ce1d5e
> giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 7f5dd3f7a47b1e323784025448766af284f1af17
> giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java b3aec677a73312dd2725ec46bac2021bc1aff7ef
> giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java 6db2ae8f389083012f8a63727a3dcba2eefa884c
> giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java 2aa0611e485683c928113d0b3ed34830d86f5b77
> giraph/src/main/java/org/apache/giraph/graph/BspServiceMaster.java b9542263aea28d8591c0bf0fdebb7731a8fa6b1e
> giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 52e7006db5cc6ba1edf1737bd5bf1d0749b98ce0
> giraph/src/main/java/org/apache/giraph/graph/ComputeCallable.java 4938169ec3155ec72aecd5b563e3c2717de02a3e
> giraph/src/main/java/org/apache/giraph/graph/EdgeInputSplitsCallable.java 4e7e7f530b8ba1ebd0e240898f0d53201d7bc1cf
> giraph/src/main/java/org/apache/giraph/graph/GraphMapper.java 92046b5f32741d4819a9c7c27bab5c8f2fe56ff5
> giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallable.java 97cc222ff53e9a8a5ebbe07abdd7f44475416a59
> giraph/src/main/java/org/apache/giraph/graph/MasterThread.java d5144de1a4ffc1f738c6dab0d14c5803f08d7398
> giraph/src/main/java/org/apache/giraph/graph/VertexInputSplitsCallable.java 2551d4a544174d7ada9a02fc99a60ff173094bd7
> giraph/src/main/java/org/apache/giraph/metrics/GiraphMetrics.java 9bb6f546d7dfb2cd97d82279d5784104b66dcacb
> giraph/src/main/java/org/apache/giraph/metrics/GiraphMetricsRegistry.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/ResetSuperstepMetricsObserver.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/SuperstepMetricsRegistry.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/ValueGauge.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/utils/FakeTime.java 1f0a892140ea2502323c512be33ec0db984e16d5
> giraph/src/main/java/org/apache/giraph/utils/MemoryUtils.java a6eba674e53aa71c90df8b23ffc9b9654f7244d8
> giraph/src/main/java/org/apache/giraph/utils/SystemTime.java 7207d80e2404af3934e82831ba5d209eaa10a6bc
> giraph/src/main/java/org/apache/giraph/utils/Time.java 1933a1de8b9ca0294d21324f4f55010df5edacb7
> giraph/src/main/java/org/apache/giraph/utils/Times.java PRE-CREATION
> giraph/src/test/java/org/apache/giraph/TestPredicateLock.java 73c508fcfdd0d4c73de23e736b93604d3101ae66
> giraph/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java c7ced6b0e49cfb1ba4201aafd6b7be12a8d464f2
> giraph/src/test/java/org/apache/giraph/utils/BspUtilsTest.java a31c2e05827fa31763c47c623574375fcf71404b
>
> Diff: https://reviews.apache.org/r/7900/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Nitay Joffe
>
>
Re: Review Request: GIRAPH-407: Update Metrics
Posted by Alessandro Presta <al...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7900/#review13201
-----------------------------------------------------------
Anyone else have comments on this?
Otherwise, I'll commit tomorrow.
- Alessandro Presta
On Nov. 7, 2012, 1:41 a.m., Nitay Joffe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7900/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2012, 1:41 a.m.)
>
>
> Review request for giraph.
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/GIRAPH-407
>
>
> Diffs
> -----
>
> giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 8ec78cd10be41145fab50773aa6f28d43b307fdc
> giraph/src/main/java/org/apache/giraph/comm/netty/ByteCounter.java 0a5f9264d0896218cfcba9d88e491bfa64ce1d5e
> giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 7f5dd3f7a47b1e323784025448766af284f1af17
> giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java b3aec677a73312dd2725ec46bac2021bc1aff7ef
> giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java 6db2ae8f389083012f8a63727a3dcba2eefa884c
> giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java 2aa0611e485683c928113d0b3ed34830d86f5b77
> giraph/src/main/java/org/apache/giraph/graph/BspServiceMaster.java b9542263aea28d8591c0bf0fdebb7731a8fa6b1e
> giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 52e7006db5cc6ba1edf1737bd5bf1d0749b98ce0
> giraph/src/main/java/org/apache/giraph/graph/ComputeCallable.java 4938169ec3155ec72aecd5b563e3c2717de02a3e
> giraph/src/main/java/org/apache/giraph/graph/EdgeInputSplitsCallable.java 4e7e7f530b8ba1ebd0e240898f0d53201d7bc1cf
> giraph/src/main/java/org/apache/giraph/graph/GraphMapper.java 92046b5f32741d4819a9c7c27bab5c8f2fe56ff5
> giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallable.java 97cc222ff53e9a8a5ebbe07abdd7f44475416a59
> giraph/src/main/java/org/apache/giraph/graph/MasterThread.java d5144de1a4ffc1f738c6dab0d14c5803f08d7398
> giraph/src/main/java/org/apache/giraph/graph/VertexInputSplitsCallable.java 2551d4a544174d7ada9a02fc99a60ff173094bd7
> giraph/src/main/java/org/apache/giraph/metrics/GiraphMetrics.java 9bb6f546d7dfb2cd97d82279d5784104b66dcacb
> giraph/src/main/java/org/apache/giraph/metrics/GiraphMetricsRegistry.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/ResetSuperstepMetricsObserver.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/SuperstepMetricsRegistry.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/ValueGauge.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/utils/FakeTime.java 1f0a892140ea2502323c512be33ec0db984e16d5
> giraph/src/main/java/org/apache/giraph/utils/MemoryUtils.java a6eba674e53aa71c90df8b23ffc9b9654f7244d8
> giraph/src/main/java/org/apache/giraph/utils/SystemTime.java 7207d80e2404af3934e82831ba5d209eaa10a6bc
> giraph/src/main/java/org/apache/giraph/utils/Time.java 1933a1de8b9ca0294d21324f4f55010df5edacb7
> giraph/src/main/java/org/apache/giraph/utils/Times.java PRE-CREATION
> giraph/src/test/java/org/apache/giraph/TestPredicateLock.java 73c508fcfdd0d4c73de23e736b93604d3101ae66
> giraph/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java c7ced6b0e49cfb1ba4201aafd6b7be12a8d464f2
> giraph/src/test/java/org/apache/giraph/utils/BspUtilsTest.java a31c2e05827fa31763c47c623574375fcf71404b
>
> Diff: https://reviews.apache.org/r/7900/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Nitay Joffe
>
>
Re: Review Request: GIRAPH-407: Update Metrics
Posted by Nitay Joffe <ni...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7900/
-----------------------------------------------------------
(Updated Nov. 7, 2012, 9:18 a.m.)
Review request for giraph.
Changes
-------
Avery's comments
Description (updated)
-------
GIRAPH-407: Update Metrics
Diffs (updated)
-----
giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 8ec78cd10be41145fab50773aa6f28d43b307fdc
giraph/src/main/java/org/apache/giraph/comm/netty/ByteCounter.java 0a5f9264d0896218cfcba9d88e491bfa64ce1d5e
giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 7f5dd3f7a47b1e323784025448766af284f1af17
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java b3aec677a73312dd2725ec46bac2021bc1aff7ef
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java 6db2ae8f389083012f8a63727a3dcba2eefa884c
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestInfo.java 6aaf3d36b66e97fc1e542e61d7f18aee31e9e5bf
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java 2aa0611e485683c928113d0b3ed34830d86f5b77
giraph/src/main/java/org/apache/giraph/graph/BspServiceMaster.java b9542263aea28d8591c0bf0fdebb7731a8fa6b1e
giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 52e7006db5cc6ba1edf1737bd5bf1d0749b98ce0
giraph/src/main/java/org/apache/giraph/graph/ComputeCallable.java 4938169ec3155ec72aecd5b563e3c2717de02a3e
giraph/src/main/java/org/apache/giraph/graph/EdgeInputSplitsCallable.java 4e7e7f530b8ba1ebd0e240898f0d53201d7bc1cf
giraph/src/main/java/org/apache/giraph/graph/GraphMapper.java 92046b5f32741d4819a9c7c27bab5c8f2fe56ff5
giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallable.java 97cc222ff53e9a8a5ebbe07abdd7f44475416a59
giraph/src/main/java/org/apache/giraph/graph/MasterThread.java d5144de1a4ffc1f738c6dab0d14c5803f08d7398
giraph/src/main/java/org/apache/giraph/graph/VertexInputSplitsCallable.java 2551d4a544174d7ada9a02fc99a60ff173094bd7
giraph/src/main/java/org/apache/giraph/metrics/GiraphMetrics.java 9bb6f546d7dfb2cd97d82279d5784104b66dcacb
giraph/src/main/java/org/apache/giraph/metrics/GiraphMetricsRegistry.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/ResetSuperstepMetricsObserver.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/SuperstepMetricsRegistry.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/ValueGauge.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/utils/FakeTime.java 1f0a892140ea2502323c512be33ec0db984e16d5
giraph/src/main/java/org/apache/giraph/utils/MemoryUtils.java a6eba674e53aa71c90df8b23ffc9b9654f7244d8
giraph/src/main/java/org/apache/giraph/utils/SystemTime.java 7207d80e2404af3934e82831ba5d209eaa10a6bc
giraph/src/main/java/org/apache/giraph/utils/Time.java 1933a1de8b9ca0294d21324f4f55010df5edacb7
giraph/src/main/java/org/apache/giraph/utils/Times.java PRE-CREATION
giraph/src/test/java/org/apache/giraph/TestPredicateLock.java 73c508fcfdd0d4c73de23e736b93604d3101ae66
giraph/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java c7ced6b0e49cfb1ba4201aafd6b7be12a8d464f2
giraph/src/test/java/org/apache/giraph/utils/BspUtilsTest.java a31c2e05827fa31763c47c623574375fcf71404b
Diff: https://reviews.apache.org/r/7900/diff/
Testing
-------
Thanks,
Nitay Joffe
Re: Review Request: GIRAPH-407: Update Metrics
Posted by Nitay Joffe <ni...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7900/
-----------------------------------------------------------
(Updated Nov. 7, 2012, 1:41 a.m.)
Review request for giraph.
Description (updated)
-------
https://issues.apache.org/jira/browse/GIRAPH-407
Diffs
-----
giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 8ec78cd10be41145fab50773aa6f28d43b307fdc
giraph/src/main/java/org/apache/giraph/comm/netty/ByteCounter.java 0a5f9264d0896218cfcba9d88e491bfa64ce1d5e
giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 7f5dd3f7a47b1e323784025448766af284f1af17
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java b3aec677a73312dd2725ec46bac2021bc1aff7ef
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java 6db2ae8f389083012f8a63727a3dcba2eefa884c
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java 2aa0611e485683c928113d0b3ed34830d86f5b77
giraph/src/main/java/org/apache/giraph/graph/BspServiceMaster.java b9542263aea28d8591c0bf0fdebb7731a8fa6b1e
giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 52e7006db5cc6ba1edf1737bd5bf1d0749b98ce0
giraph/src/main/java/org/apache/giraph/graph/ComputeCallable.java 4938169ec3155ec72aecd5b563e3c2717de02a3e
giraph/src/main/java/org/apache/giraph/graph/EdgeInputSplitsCallable.java 4e7e7f530b8ba1ebd0e240898f0d53201d7bc1cf
giraph/src/main/java/org/apache/giraph/graph/GraphMapper.java 92046b5f32741d4819a9c7c27bab5c8f2fe56ff5
giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallable.java 97cc222ff53e9a8a5ebbe07abdd7f44475416a59
giraph/src/main/java/org/apache/giraph/graph/MasterThread.java d5144de1a4ffc1f738c6dab0d14c5803f08d7398
giraph/src/main/java/org/apache/giraph/graph/VertexInputSplitsCallable.java 2551d4a544174d7ada9a02fc99a60ff173094bd7
giraph/src/main/java/org/apache/giraph/metrics/GiraphMetrics.java 9bb6f546d7dfb2cd97d82279d5784104b66dcacb
giraph/src/main/java/org/apache/giraph/metrics/GiraphMetricsRegistry.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/ResetSuperstepMetricsObserver.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/SuperstepMetricsRegistry.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/ValueGauge.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/utils/FakeTime.java 1f0a892140ea2502323c512be33ec0db984e16d5
giraph/src/main/java/org/apache/giraph/utils/MemoryUtils.java a6eba674e53aa71c90df8b23ffc9b9654f7244d8
giraph/src/main/java/org/apache/giraph/utils/SystemTime.java 7207d80e2404af3934e82831ba5d209eaa10a6bc
giraph/src/main/java/org/apache/giraph/utils/Time.java 1933a1de8b9ca0294d21324f4f55010df5edacb7
giraph/src/main/java/org/apache/giraph/utils/Times.java PRE-CREATION
giraph/src/test/java/org/apache/giraph/TestPredicateLock.java 73c508fcfdd0d4c73de23e736b93604d3101ae66
giraph/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java c7ced6b0e49cfb1ba4201aafd6b7be12a8d464f2
giraph/src/test/java/org/apache/giraph/utils/BspUtilsTest.java a31c2e05827fa31763c47c623574375fcf71404b
Diff: https://reviews.apache.org/r/7900/diff/
Testing
-------
Thanks,
Nitay Joffe
Re: Review Request: GIRAPH-407: Update Metrics
Posted by Nitay Joffe <ni...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7900/
-----------------------------------------------------------
(Updated Nov. 7, 2012, 1:40 a.m.)
Review request for giraph.
Changes
-------
Alessandro's comments
Description (updated)
-------
GIRAPH-407: Update Metrics
Diffs (updated)
-----
giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 8ec78cd10be41145fab50773aa6f28d43b307fdc
giraph/src/main/java/org/apache/giraph/comm/netty/ByteCounter.java 0a5f9264d0896218cfcba9d88e491bfa64ce1d5e
giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 7f5dd3f7a47b1e323784025448766af284f1af17
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java b3aec677a73312dd2725ec46bac2021bc1aff7ef
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java 6db2ae8f389083012f8a63727a3dcba2eefa884c
giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java 2aa0611e485683c928113d0b3ed34830d86f5b77
giraph/src/main/java/org/apache/giraph/graph/BspServiceMaster.java b9542263aea28d8591c0bf0fdebb7731a8fa6b1e
giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 52e7006db5cc6ba1edf1737bd5bf1d0749b98ce0
giraph/src/main/java/org/apache/giraph/graph/ComputeCallable.java 4938169ec3155ec72aecd5b563e3c2717de02a3e
giraph/src/main/java/org/apache/giraph/graph/EdgeInputSplitsCallable.java 4e7e7f530b8ba1ebd0e240898f0d53201d7bc1cf
giraph/src/main/java/org/apache/giraph/graph/GraphMapper.java 92046b5f32741d4819a9c7c27bab5c8f2fe56ff5
giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallable.java 97cc222ff53e9a8a5ebbe07abdd7f44475416a59
giraph/src/main/java/org/apache/giraph/graph/MasterThread.java d5144de1a4ffc1f738c6dab0d14c5803f08d7398
giraph/src/main/java/org/apache/giraph/graph/VertexInputSplitsCallable.java 2551d4a544174d7ada9a02fc99a60ff173094bd7
giraph/src/main/java/org/apache/giraph/metrics/GiraphMetrics.java 9bb6f546d7dfb2cd97d82279d5784104b66dcacb
giraph/src/main/java/org/apache/giraph/metrics/GiraphMetricsRegistry.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/ResetSuperstepMetricsObserver.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/SuperstepMetricsRegistry.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/metrics/ValueGauge.java PRE-CREATION
giraph/src/main/java/org/apache/giraph/utils/FakeTime.java 1f0a892140ea2502323c512be33ec0db984e16d5
giraph/src/main/java/org/apache/giraph/utils/MemoryUtils.java a6eba674e53aa71c90df8b23ffc9b9654f7244d8
giraph/src/main/java/org/apache/giraph/utils/SystemTime.java 7207d80e2404af3934e82831ba5d209eaa10a6bc
giraph/src/main/java/org/apache/giraph/utils/Time.java 1933a1de8b9ca0294d21324f4f55010df5edacb7
giraph/src/main/java/org/apache/giraph/utils/Times.java PRE-CREATION
giraph/src/test/java/org/apache/giraph/TestPredicateLock.java 73c508fcfdd0d4c73de23e736b93604d3101ae66
giraph/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java c7ced6b0e49cfb1ba4201aafd6b7be12a8d464f2
giraph/src/test/java/org/apache/giraph/utils/BspUtilsTest.java a31c2e05827fa31763c47c623574375fcf71404b
Diff: https://reviews.apache.org/r/7900/diff/
Testing
-------
Thanks,
Nitay Joffe
Re: Review Request: GIRAPH-407: Update Metrics
Posted by Alessandro Presta <al...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7900/#review13193
-----------------------------------------------------------
+1, I like the new design with observers.
giraph/src/main/java/org/apache/giraph/graph/GraphMapper.java
<https://reviews.apache.org/r/7900/#comment28364>
null -> zero
giraph/src/main/java/org/apache/giraph/metrics/LongGauge.java
<https://reviews.apache.org/r/7900/#comment28365>
If this is not memory-critical code, can't we implement the get/set/clear methods directly in ValueGauge? That way we don't need to write the same code for every value type (Double, Long, etc).
- Alessandro Presta
On Nov. 6, 2012, 6:19 p.m., Nitay Joffe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7900/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2012, 6:19 p.m.)
>
>
> Review request for giraph.
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/GIRAPH-407
>
> Updated metrics. Main logic changes are:
> - Split into per-job and per-superstep metrics.
> - The per-superstep metrics change their group (e.g. giraph.superstep.1.compute to giraph.superstep.2.compute) at beginning of each superstep.
> - Any class using per-superstep metrics registers itself and gets a callback to reset its metrics objects on each new superstep.
> - Cleanup Time hierarchy with Times class that has helper methods for all Time implementors.
>
> By default metrics are disabled, to enable them use -Dgiraph.metrics.enable=true.
> When enabled the metrics are always sent to JMX.
> If you pass -Dgiraph.metrics.dump.at.end=true then this output will be produced on each worker at the end of the job: https://gist.github.com/38b8f527504f7792485c
> If you pass -Dgiraph.metrics.print=true then this output will be produced on each worker at the end of each superstep: https://gist.github.com/3e3fad650e3960ad9edc
>
>
> Diffs
> -----
>
> giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 8ec78cd10be41145fab50773aa6f28d43b307fdc
> giraph/src/main/java/org/apache/giraph/comm/netty/ByteCounter.java 0a5f9264d0896218cfcba9d88e491bfa64ce1d5e
> giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 7f5dd3f7a47b1e323784025448766af284f1af17
> giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java b3aec677a73312dd2725ec46bac2021bc1aff7ef
> giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java 6db2ae8f389083012f8a63727a3dcba2eefa884c
> giraph/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java 2aa0611e485683c928113d0b3ed34830d86f5b77
> giraph/src/main/java/org/apache/giraph/graph/BspServiceMaster.java b9542263aea28d8591c0bf0fdebb7731a8fa6b1e
> giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 52e7006db5cc6ba1edf1737bd5bf1d0749b98ce0
> giraph/src/main/java/org/apache/giraph/graph/ComputeCallable.java 4938169ec3155ec72aecd5b563e3c2717de02a3e
> giraph/src/main/java/org/apache/giraph/graph/EdgeInputSplitsCallable.java 4e7e7f530b8ba1ebd0e240898f0d53201d7bc1cf
> giraph/src/main/java/org/apache/giraph/graph/GraphMapper.java 92046b5f32741d4819a9c7c27bab5c8f2fe56ff5
> giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallable.java 97cc222ff53e9a8a5ebbe07abdd7f44475416a59
> giraph/src/main/java/org/apache/giraph/graph/MasterThread.java d5144de1a4ffc1f738c6dab0d14c5803f08d7398
> giraph/src/main/java/org/apache/giraph/graph/VertexInputSplitsCallable.java 2551d4a544174d7ada9a02fc99a60ff173094bd7
> giraph/src/main/java/org/apache/giraph/metrics/ClearableGauge.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/DoubleGauge.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/GiraphMetrics.java 9bb6f546d7dfb2cd97d82279d5784104b66dcacb
> giraph/src/main/java/org/apache/giraph/metrics/GiraphMetricsRegistry.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/LongGauge.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/ResetSuperstepMetricsObserver.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/SuperstepMetricsRegistry.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/metrics/ValueGauge.java PRE-CREATION
> giraph/src/main/java/org/apache/giraph/utils/FakeTime.java 1f0a892140ea2502323c512be33ec0db984e16d5
> giraph/src/main/java/org/apache/giraph/utils/MemoryUtils.java a6eba674e53aa71c90df8b23ffc9b9654f7244d8
> giraph/src/main/java/org/apache/giraph/utils/SystemTime.java 7207d80e2404af3934e82831ba5d209eaa10a6bc
> giraph/src/main/java/org/apache/giraph/utils/Time.java 1933a1de8b9ca0294d21324f4f55010df5edacb7
> giraph/src/main/java/org/apache/giraph/utils/Times.java PRE-CREATION
> giraph/src/test/java/org/apache/giraph/TestPredicateLock.java 73c508fcfdd0d4c73de23e736b93604d3101ae66
> giraph/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java c7ced6b0e49cfb1ba4201aafd6b7be12a8d464f2
> giraph/src/test/java/org/apache/giraph/utils/BspUtilsTest.java a31c2e05827fa31763c47c623574375fcf71404b
>
> Diff: https://reviews.apache.org/r/7900/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Nitay Joffe
>
>