You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Shreyas Subramanya <sh...@gmail.com> on 2014/07/14 20:53:28 UTC
Review Request 23456: Review request for FLUME-2413
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23456/
-----------------------------------------------------------
Review request for Flume.
Bugs: FLUME-2413
https://issues.apache.org/jira/browse/FLUME-2413
Repository: flume-git
Description
-------
Add a new metric for the number of bytes transferred on a flume channel
Diffs
-----
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java 0f242d2
flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelMetrics.java PRE-CREATION
flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java b84a3e0
flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java cc391c4
flume-ng-core/src/main/java/org/apache/flume/instrumentation/ChannelCounter.java 9938c0a
flume-ng-core/src/main/java/org/apache/flume/instrumentation/ChannelCounterMBean.java 9be8c3c
flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelMetrics.java PRE-CREATION
flume-ng-core/src/test/java/org/apache/flume/instrumentation/TestMonitoredCounterGroup.java b1f637f
Diff: https://reviews.apache.org/r/23456/diff/
Testing
-------
Implemented and ran junit test cases. Also implemented and tested in our product.
Thanks,
Shreyas Subramanya
Re: Review Request 23456: Review request for FLUME-2413
Posted by Shreyas Subramanya <sh...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23456/
-----------------------------------------------------------
(Updated Sept. 17, 2014, 1 a.m.)
Review request for Flume.
Changes
-------
I have refreshed the patch here on review board
Bugs: FLUME-2413
https://issues.apache.org/jira/browse/FLUME-2413
Repository: flume-git
Description
-------
Add a new metric for the number of bytes transferred on a flume channel
Diffs
-----
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java 0f242d2
flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelMetrics.java PRE-CREATION
flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java b84a3e0
flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java cc391c4
flume-ng-core/src/main/java/org/apache/flume/instrumentation/ChannelCounter.java 9938c0a
flume-ng-core/src/main/java/org/apache/flume/instrumentation/ChannelCounterMBean.java 9be8c3c
flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelMetrics.java PRE-CREATION
flume-ng-core/src/test/java/org/apache/flume/instrumentation/TestMonitoredCounterGroup.java b1f637f
Diff: https://reviews.apache.org/r/23456/diff/
Testing
-------
Implemented and ran junit test cases. Also implemented and tested in our product.
File Attachments (updated)
----------------
FLUME-2413.patch3
https://reviews.apache.org/media/uploaded/files/2014/09/17/6186cb32-b101-49c0-ad7f-53e4feb3d3d5__FLUME-2413.patch3
Thanks,
Shreyas Subramanya
Re: Review Request 23456: Review request for FLUME-2413
Posted by Roshan Naik <ro...@hortonworks.com>.
> On Aug. 1, 2014, 11:41 p.m., Roshan Naik wrote:
> > Couple of points.
> > - From a monitoring standpoint, it is more useful to have a putCommit bytes & takeCommit bytes. instead of putBytes & putCommitBytes.
> > - The computation of event size is incorrect. It should be eventSize =bodySize + headerSize. So the reported metric will be inaccurate. Also incorporating event header's size into this will be expensive.
>
> Shreyas Subramanya wrote:
> Thanks for the comments Roshan.
> - I have added a new patch to separate out the 2 metrics - putCommit bytes and takeCommit bytes.
> - As a user, I would be mostly concerned with the number of payload bytes transmitted through a flume agent - so here we just display the body size.
could you refresh the patch here in review board ? fyi: i see some whitespace errors being flagged by RB in the old patch.
- Roshan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23456/#review49405
-----------------------------------------------------------
On July 14, 2014, 6:53 p.m., Shreyas Subramanya wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23456/
> -----------------------------------------------------------
>
> (Updated July 14, 2014, 6:53 p.m.)
>
>
> Review request for Flume.
>
>
> Bugs: FLUME-2413
> https://issues.apache.org/jira/browse/FLUME-2413
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> Add a new metric for the number of bytes transferred on a flume channel
>
>
> Diffs
> -----
>
> flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java 0f242d2
> flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelMetrics.java PRE-CREATION
> flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java b84a3e0
> flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java cc391c4
> flume-ng-core/src/main/java/org/apache/flume/instrumentation/ChannelCounter.java 9938c0a
> flume-ng-core/src/main/java/org/apache/flume/instrumentation/ChannelCounterMBean.java 9be8c3c
> flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelMetrics.java PRE-CREATION
> flume-ng-core/src/test/java/org/apache/flume/instrumentation/TestMonitoredCounterGroup.java b1f637f
>
> Diff: https://reviews.apache.org/r/23456/diff/
>
>
> Testing
> -------
>
> Implemented and ran junit test cases. Also implemented and tested in our product.
>
>
> Thanks,
>
> Shreyas Subramanya
>
>
Re: Review Request 23456: Review request for FLUME-2413
Posted by Shreyas Subramanya <sh...@gmail.com>.
> On Aug. 1, 2014, 11:41 p.m., Roshan Naik wrote:
> > Couple of points.
> > - From a monitoring standpoint, it is more useful to have a putCommit bytes & takeCommit bytes. instead of putBytes & putCommitBytes.
> > - The computation of event size is incorrect. It should be eventSize =bodySize + headerSize. So the reported metric will be inaccurate. Also incorporating event header's size into this will be expensive.
Thanks for the comments Roshan.
- I have added a new patch to separate out the 2 metrics - putCommit bytes and takeCommit bytes.
- As a user, I would be mostly concerned with the number of payload bytes transmitted through a flume agent - so here we just display the body size.
- Shreyas
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23456/#review49405
-----------------------------------------------------------
On July 14, 2014, 6:53 p.m., Shreyas Subramanya wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23456/
> -----------------------------------------------------------
>
> (Updated July 14, 2014, 6:53 p.m.)
>
>
> Review request for Flume.
>
>
> Bugs: FLUME-2413
> https://issues.apache.org/jira/browse/FLUME-2413
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> Add a new metric for the number of bytes transferred on a flume channel
>
>
> Diffs
> -----
>
> flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java 0f242d2
> flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelMetrics.java PRE-CREATION
> flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java b84a3e0
> flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java cc391c4
> flume-ng-core/src/main/java/org/apache/flume/instrumentation/ChannelCounter.java 9938c0a
> flume-ng-core/src/main/java/org/apache/flume/instrumentation/ChannelCounterMBean.java 9be8c3c
> flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelMetrics.java PRE-CREATION
> flume-ng-core/src/test/java/org/apache/flume/instrumentation/TestMonitoredCounterGroup.java b1f637f
>
> Diff: https://reviews.apache.org/r/23456/diff/
>
>
> Testing
> -------
>
> Implemented and ran junit test cases. Also implemented and tested in our product.
>
>
> Thanks,
>
> Shreyas Subramanya
>
>
Re: Review Request 23456: Review request for FLUME-2413
Posted by Roshan Naik <ro...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23456/#review49405
-----------------------------------------------------------
Couple of points.
- From a monitoring standpoint, it is more useful to have a putCommit bytes & takeCommit bytes. instead of putBytes & putCommitBytes.
- The computation of event size is incorrect. It should be eventSize =bodySize + headerSize. So the reported metric will be inaccurate. Also incorporating event header's size into this will be expensive.
- Roshan Naik
On July 14, 2014, 6:53 p.m., Shreyas Subramanya wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23456/
> -----------------------------------------------------------
>
> (Updated July 14, 2014, 6:53 p.m.)
>
>
> Review request for Flume.
>
>
> Bugs: FLUME-2413
> https://issues.apache.org/jira/browse/FLUME-2413
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> Add a new metric for the number of bytes transferred on a flume channel
>
>
> Diffs
> -----
>
> flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java 0f242d2
> flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelMetrics.java PRE-CREATION
> flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java b84a3e0
> flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java cc391c4
> flume-ng-core/src/main/java/org/apache/flume/instrumentation/ChannelCounter.java 9938c0a
> flume-ng-core/src/main/java/org/apache/flume/instrumentation/ChannelCounterMBean.java 9be8c3c
> flume-ng-core/src/test/java/org/apache/flume/channel/TestChannelMetrics.java PRE-CREATION
> flume-ng-core/src/test/java/org/apache/flume/instrumentation/TestMonitoredCounterGroup.java b1f637f
>
> Diff: https://reviews.apache.org/r/23456/diff/
>
>
> Testing
> -------
>
> Implemented and ran junit test cases. Also implemented and tested in our product.
>
>
> Thanks,
>
> Shreyas Subramanya
>
>