You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Arvind Prabhakar (Created) (JIRA)" <ji...@apache.org> on 2011/10/25 22:14:34 UTC
[jira] [Created] (FLUME-820) JDBC channel should support capacity
specification.
JDBC channel should support capacity specification.
---------------------------------------------------
Key: FLUME-820
URL: https://issues.apache.org/jira/browse/FLUME-820
Project: Flume
Issue Type: Improvement
Reporter: Arvind Prabhakar
Assignee: Arvind Prabhakar
users should be able to specify capacity for the JDBC channel in terms of total number of events stored. This will help ensure that the channel does not overwhelm the system by occupying the entire available disk space.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-820) JDBC channel should support capacity
specification.
Posted by "Arvind Prabhakar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar updated FLUME-820:
-----------------------------------
Fix Version/s: NG alpha 2
> JDBC channel should support capacity specification.
> ---------------------------------------------------
>
> Key: FLUME-820
> URL: https://issues.apache.org/jira/browse/FLUME-820
> Project: Flume
> Issue Type: Improvement
> Reporter: Arvind Prabhakar
> Assignee: Arvind Prabhakar
> Fix For: NG alpha 2
>
>
> users should be able to specify capacity for the JDBC channel in terms of total number of events stored. This will help ensure that the channel does not overwhelm the system by occupying the entire available disk space.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-820) JDBC channel should support capacity
specification.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157610#comment-13157610 ]
jiraposter@reviews.apache.org commented on FLUME-820:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2937/
-----------------------------------------------------------
Review request for Flume and Prasad Mujumdar.
Summary
-------
Added support for specifying the capacity of the JDBC channel. If a non-zero positive value is specified, the channel enforces it as the limit on total number of events persisted at any time. If a zero/negative/unspecified value is provided, the channel operates without any limit. Added a test case as well to exercise this functionality.
Another change that I am sliding in is reducing the overall number of events that tests exercise - from about 2000 to close to 200. This will greatly reduce the amount of time the tests take when running on the JDBC channel module. The test functionality remains the same.
This addresses bug FLUME-820.
https://issues.apache.org/jira/browse/FLUME-820
Diffs
-----
flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java 7e0b3ac
flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java 7b29c7d
flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java c1bc400
flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java 1065ef9
flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java 68946c9
flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/MockEventUtils.java 78eda51
flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java 9ce6d16
flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestJdbcChannelProvider.java 4c3012d
Diff: https://reviews.apache.org/r/2937/diff
Testing
-------
Ran all JDBC channel tests.
Thanks,
Arvind
> JDBC channel should support capacity specification.
> ---------------------------------------------------
>
> Key: FLUME-820
> URL: https://issues.apache.org/jira/browse/FLUME-820
> Project: Flume
> Issue Type: Improvement
> Reporter: Arvind Prabhakar
> Assignee: Arvind Prabhakar
> Fix For: NG alpha 2
>
> Attachments: FLUME-820-1.patch
>
>
> users should be able to specify capacity for the JDBC channel in terms of total number of events stored. This will help ensure that the channel does not overwhelm the system by occupying the entire available disk space.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-820) JDBC channel should support capacity
specification.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160197#comment-13160197 ]
jiraposter@reviews.apache.org commented on FLUME-820:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2937/#review3581
-----------------------------------------------------------
Ship it!
lgtm.
flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java
<https://reviews.apache.org/r/2937/#comment7994>
Not sure if Statement implements Closable, but maybe use Guava Closeables.closeQuietly(o)?
- Eric
On 2011-11-26 21:14:07, Arvind Prabhakar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2937/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-26 21:14:07)
bq.
bq.
bq. Review request for Flume and Prasad Mujumdar.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Added support for specifying the capacity of the JDBC channel. If a non-zero positive value is specified, the channel enforces it as the limit on total number of events persisted at any time. If a zero/negative/unspecified value is provided, the channel operates without any limit. Added a test case as well to exercise this functionality.
bq.
bq. Another change that I am sliding in is reducing the overall number of events that tests exercise - from about 2000 to close to 200. This will greatly reduce the amount of time the tests take when running on the JDBC channel module. The test functionality remains the same.
bq.
bq.
bq. This addresses bug FLUME-820.
bq. https://issues.apache.org/jira/browse/FLUME-820
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java 7e0b3ac
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java 7b29c7d
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java c1bc400
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java 1065ef9
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java 68946c9
bq. flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/MockEventUtils.java 78eda51
bq. flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java 9ce6d16
bq. flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestJdbcChannelProvider.java 4c3012d
bq.
bq. Diff: https://reviews.apache.org/r/2937/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Ran all JDBC channel tests.
bq.
bq.
bq. Thanks,
bq.
bq. Arvind
bq.
bq.
> JDBC channel should support capacity specification.
> ---------------------------------------------------
>
> Key: FLUME-820
> URL: https://issues.apache.org/jira/browse/FLUME-820
> Project: Flume
> Issue Type: Improvement
> Reporter: Arvind Prabhakar
> Assignee: Arvind Prabhakar
> Fix For: NG alpha 2
>
> Attachments: FLUME-820-1.patch
>
>
> users should be able to specify capacity for the JDBC channel in terms of total number of events stored. This will help ensure that the channel does not overwhelm the system by occupying the entire available disk space.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-820) JDBC channel should support capacity
specification.
Posted by "Arvind Prabhakar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar updated FLUME-820:
-----------------------------------
Status: Patch Available (was: Open)
> JDBC channel should support capacity specification.
> ---------------------------------------------------
>
> Key: FLUME-820
> URL: https://issues.apache.org/jira/browse/FLUME-820
> Project: Flume
> Issue Type: Improvement
> Reporter: Arvind Prabhakar
> Assignee: Arvind Prabhakar
> Fix For: NG alpha 2
>
> Attachments: FLUME-820-1.patch
>
>
> users should be able to specify capacity for the JDBC channel in terms of total number of events stored. This will help ensure that the channel does not overwhelm the system by occupying the entire available disk space.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-820) JDBC channel should support capacity
specification.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160218#comment-13160218 ]
jiraposter@reviews.apache.org commented on FLUME-820:
-----------------------------------------------------
bq. On 2011-11-30 18:13:08, Eric Sammer wrote:
bq. > flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java, line 1004
bq. > <https://reviews.apache.org/r/2937/diff/1/?file=60132#file60132line1004>
bq. >
bq. > Not sure if Statement implements Closable, but maybe use Guava Closeables.closeQuietly(o)?
It does not, although it does have a close method. Such boilerplate code is an eyesore and can be moved/removed once Java starts supporting closures.
- Arvind
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2937/#review3581
-----------------------------------------------------------
On 2011-11-26 21:14:07, Arvind Prabhakar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2937/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-26 21:14:07)
bq.
bq.
bq. Review request for Flume and Prasad Mujumdar.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Added support for specifying the capacity of the JDBC channel. If a non-zero positive value is specified, the channel enforces it as the limit on total number of events persisted at any time. If a zero/negative/unspecified value is provided, the channel operates without any limit. Added a test case as well to exercise this functionality.
bq.
bq. Another change that I am sliding in is reducing the overall number of events that tests exercise - from about 2000 to close to 200. This will greatly reduce the amount of time the tests take when running on the JDBC channel module. The test functionality remains the same.
bq.
bq.
bq. This addresses bug FLUME-820.
bq. https://issues.apache.org/jira/browse/FLUME-820
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java 7e0b3ac
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java 7b29c7d
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java c1bc400
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java 1065ef9
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java 68946c9
bq. flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/MockEventUtils.java 78eda51
bq. flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java 9ce6d16
bq. flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestJdbcChannelProvider.java 4c3012d
bq.
bq. Diff: https://reviews.apache.org/r/2937/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Ran all JDBC channel tests.
bq.
bq.
bq. Thanks,
bq.
bq. Arvind
bq.
bq.
> JDBC channel should support capacity specification.
> ---------------------------------------------------
>
> Key: FLUME-820
> URL: https://issues.apache.org/jira/browse/FLUME-820
> Project: Flume
> Issue Type: Improvement
> Reporter: Arvind Prabhakar
> Assignee: Arvind Prabhakar
> Fix For: NG alpha 2
>
> Attachments: FLUME-820-1.patch
>
>
> users should be able to specify capacity for the JDBC channel in terms of total number of events stored. This will help ensure that the channel does not overwhelm the system by occupying the entire available disk space.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-820) JDBC channel should support capacity
specification.
Posted by "Arvind Prabhakar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar updated FLUME-820:
-----------------------------------
Attachment: FLUME-820-1.patch
> JDBC channel should support capacity specification.
> ---------------------------------------------------
>
> Key: FLUME-820
> URL: https://issues.apache.org/jira/browse/FLUME-820
> Project: Flume
> Issue Type: Improvement
> Reporter: Arvind Prabhakar
> Assignee: Arvind Prabhakar
> Fix For: NG alpha 2
>
> Attachments: FLUME-820-1.patch
>
>
> users should be able to specify capacity for the JDBC channel in terms of total number of events stored. This will help ensure that the channel does not overwhelm the system by occupying the entire available disk space.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-820) JDBC channel should support capacity
specification.
Posted by "Arvind Prabhakar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar updated FLUME-820:
-----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
patch committed.
> JDBC channel should support capacity specification.
> ---------------------------------------------------
>
> Key: FLUME-820
> URL: https://issues.apache.org/jira/browse/FLUME-820
> Project: Flume
> Issue Type: Improvement
> Reporter: Arvind Prabhakar
> Assignee: Arvind Prabhakar
> Fix For: NG alpha 2
>
> Attachments: FLUME-820-1.patch
>
>
> users should be able to specify capacity for the JDBC channel in terms of total number of events stored. This will help ensure that the channel does not overwhelm the system by occupying the entire available disk space.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-820) JDBC channel should support capacity
specification.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160260#comment-13160260 ]
Hudson commented on FLUME-820:
------------------------------
Integrated in flume-728 #63 (See [https://builds.apache.org/job/flume-728/63/])
FLUME-820. Support for capacity specification for JDBC channel.
arvind : http://svn.apache.org/viewvc/?view=rev&rev=1208691
Files :
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/MockEventUtils.java
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestJdbcChannelProvider.java
> JDBC channel should support capacity specification.
> ---------------------------------------------------
>
> Key: FLUME-820
> URL: https://issues.apache.org/jira/browse/FLUME-820
> Project: Flume
> Issue Type: Improvement
> Reporter: Arvind Prabhakar
> Assignee: Arvind Prabhakar
> Fix For: NG alpha 2
>
> Attachments: FLUME-820-1.patch
>
>
> users should be able to specify capacity for the JDBC channel in terms of total number of events stored. This will help ensure that the channel does not overwhelm the system by occupying the entire available disk space.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira