You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org> on 2012/04/12 02:33:13 UTC
[jira] [Commented] (FLUME-1113) JDBC Channel invokes size query on
every put
[ https://issues.apache.org/jira/browse/FLUME-1113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252091#comment-13252091 ]
jiraposter@reviews.apache.org commented on FLUME-1113:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4704/
-----------------------------------------------------------
Review request for Flume.
Summary
-------
This change modifies the JDBC channel provider implementation so that it invokes the size query only during configuration to initialize it's own execution state. At runtime, the transaction instances keep track of how many events get added and removed and the delta is then added to the size maintained by the provider. Some notes about this implementation:
* It is possible that between the commit and update of the size counter by one transaction, another transaction may read stale size information.
* The size counter is only used to gate put operations and not take. This ensures that stale or incorrect information does not inhibit the draining of the channel.
Also refactored the channel tests to be run with and without foreign keys enabled.
This addresses bug FLUME-1113.
https://issues.apache.org/jira/browse/FLUME-1113
Diffs
-----
/trunk/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java 1325037
/trunk/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcTransactionFactory.java 1325037
/trunk/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcTransactionImpl.java 1325037
/trunk/flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/BaseJdbcChannelProviderTest.java PRE-CREATION
/trunk/flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestJdbcChannelProvider.java 1325037
/trunk/flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestJdbcChannelProviderNoFK.java PRE-CREATION
Diff: https://reviews.apache.org/r/4704/diff
Testing
-------
Full build and tests.
Thanks,
Arvind
> JDBC Channel invokes size query on every put
> --------------------------------------------
>
> Key: FLUME-1113
> URL: https://issues.apache.org/jira/browse/FLUME-1113
> Project: Flume
> Issue Type: Sub-task
> Components: Channel
> Reporter: Arvind Prabhakar
> Assignee: Arvind Prabhakar
>
> JDBC Channel provider invokes SchemaHandler.getChannelSize() to test for capacity. It will be better if the channel issues this query once and then uses an in-memory counter to maintain this rather than doing a query on every put.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira