You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "E. Sammer (Created) (JIRA)" <ji...@apache.org> on 2011/10/30 23:53:32 UTC
[jira] [Created] (FLUME-822) JDBC channel lock acquisition failure
during take()
JDBC channel lock acquisition failure during take()
---------------------------------------------------
Key: FLUME-822
URL: https://issues.apache.org/jira/browse/FLUME-822
Project: Flume
Issue Type: Bug
Affects Versions: NG alpha 1
Reporter: E. Sammer
Assignee: Arvind Prabhakar
Priority: Blocker
Fix For: NG alpha 2
The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
{code}
localhost.sources = avro nc
localhost.sinks = logger log2
localhost.channels = ch1 ch2
localhost.channels.ch1.type = memory
localhost.channels.ch1.capacity = 10000
localhost.sources.avro.type = avro
localhost.sources.avro.bind = 0.0.0.0
localhost.sources.avro.port = 41414
localhost.sources.avro.channels = ch1
localhost.sinks.logger.type = logger
localhost.sinks.logger.channel = ch1
#
localhost.channels.ch2.type = jdbc
localhost.sources.nc.type = netcat
localhost.sources.nc.bind = 0.0.0.0
localhost.sources.nc.port = 41415
localhost.sources.nc.name = nc
localhost.sources.nc.channels = ch2
localhost.sinks.log2.type = logger
localhost.sinks.log2.channel = ch2
{code}
The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-822) JDBC channel lock acquisition failure
during take()
Posted by "Arvind Prabhakar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar updated FLUME-822:
-----------------------------------
Status: Patch Available (was: Open)
This patch addresses some transaction related code that could potentially lead to lock problems and transaction leaks.
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
> Attachments: FLUME-822-1.patch, flume-jdbc-lock-failure.log
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-822) JDBC channel lock acquisition failure
during take()
Posted by "E. Sammer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
E. Sammer updated FLUME-822:
----------------------------
Description:
The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
==
localhost.sources = avro nc
localhost.sinks = logger log2
localhost.channels = ch1 ch2
localhost.channels.ch1.type = memory
localhost.channels.ch1.capacity = 10000
localhost.sources.avro.type = avro
localhost.sources.avro.bind = 0.0.0.0
localhost.sources.avro.port = 41414
localhost.sources.avro.channels = ch1
localhost.sinks.logger.type = logger
localhost.sinks.logger.channel = ch1
#
localhost.channels.ch2.type = jdbc
localhost.sources.nc.type = netcat
localhost.sources.nc.bind = 0.0.0.0
localhost.sources.nc.port = 41415
localhost.sources.nc.name = nc
localhost.sources.nc.channels = ch2
localhost.sinks.log2.type = logger
localhost.sinks.log2.channel = ch2
==
The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
was:
The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
{code}
localhost.sources = avro nc
localhost.sinks = logger log2
localhost.channels = ch1 ch2
localhost.channels.ch1.type = memory
localhost.channels.ch1.capacity = 10000
localhost.sources.avro.type = avro
localhost.sources.avro.bind = 0.0.0.0
localhost.sources.avro.port = 41414
localhost.sources.avro.channels = ch1
localhost.sinks.logger.type = logger
localhost.sinks.logger.channel = ch1
#
localhost.channels.ch2.type = jdbc
localhost.sources.nc.type = netcat
localhost.sources.nc.bind = 0.0.0.0
localhost.sources.nc.port = 41415
localhost.sources.nc.name = nc
localhost.sources.nc.channels = ch2
localhost.sinks.log2.type = logger
localhost.sinks.log2.channel = ch2
{code}
The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-822) JDBC channel lock acquisition failure
during take()
Posted by "E. Sammer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
E. Sammer updated FLUME-822:
----------------------------
Component/s: Channel
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-822) JDBC channel lock acquisition
failure during take()
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160189#comment-13160189 ]
jiraposter@reviews.apache.org commented on FLUME-822:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2943/#review3578
-----------------------------------------------------------
Ship it!
I'm embarrassed I never got back around to fixing these things. lgtm.
- Eric
On 2011-11-27 06:52:44, Arvind Prabhakar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2943/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-27 06:52:44)
bq.
bq.
bq. Review request for Flume and Eric Sammer.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This change cleans up some code in LoggerSink, RollingFileSink and NetcatSource dealing with transaction handling.
bq.
bq. The problem highlighted in FLUME-822 is not reproducible so far and my hope is that this clean up code will help address any isolated cases that could cause it.
bq.
bq.
bq. This addresses bug FLUME-822.
bq. https://issues.apache.org/jira/browse/FLUME-822
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-core/src/main/java/org/apache/flume/sink/LoggerSink.java f098604
bq. flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 5632b75
bq. flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java f9251a6
bq.
bq. Diff: https://reviews.apache.org/r/2943/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Ran all the tests successfully.
bq.
bq.
bq. Thanks,
bq.
bq. Arvind
bq.
bq.
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
> Attachments: FLUME-822-1.patch, flume-jdbc-lock-failure.log
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-822) JDBC channel lock acquisition
failure during take()
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157682#comment-13157682 ]
jiraposter@reviews.apache.org commented on FLUME-822:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2943/
-----------------------------------------------------------
Review request for Flume and Eric Sammer.
Summary
-------
This change cleans up some code in LoggerSink, RollingFileSink and NetcatSource dealing with transaction handling.
The problem highlighted in FLUME-822 is not reproducible so far and my hope is that this clean up code will help address any isolated cases that could cause it.
This addresses bug FLUME-822.
https://issues.apache.org/jira/browse/FLUME-822
Diffs
-----
flume-ng-core/src/main/java/org/apache/flume/sink/LoggerSink.java f098604
flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 5632b75
flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java f9251a6
Diff: https://reviews.apache.org/r/2943/diff
Testing
-------
Ran all the tests successfully.
Thanks,
Arvind
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
> Attachments: FLUME-822-1.patch, flume-jdbc-lock-failure.log
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-822) JDBC channel lock acquisition
failure during take()
Posted by "Arvind Prabhakar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156847#comment-13156847 ]
Arvind Prabhakar commented on FLUME-822:
----------------------------------------
So far I am unable to reproduce this issue. However, I will continue to test different scenarios to see if this happens in other cases.
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
> Attachments: flume-jdbc-lock-failure.log
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-822) JDBC channel lock acquisition
failure during take()
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160257#comment-13160257 ]
Hudson commented on FLUME-822:
------------------------------
Integrated in flume-728 #63 (See [https://builds.apache.org/job/flume-728/63/])
FLUME-822. Transaction code cleanup to fix potential lock timeout issues.
arvind : http://svn.apache.org/viewvc/?view=rev&rev=1208645
Files :
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/sink/LoggerSink.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
> Attachments: FLUME-822-1.patch, flume-jdbc-lock-failure.log
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-822) JDBC channel lock acquisition failure
during take()
Posted by "Arvind Prabhakar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar updated FLUME-822:
-----------------------------------
Attachment: FLUME-822-1.patch
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
> Attachments: FLUME-822-1.patch, flume-jdbc-lock-failure.log
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-822) JDBC channel lock acquisition
failure during take()
Posted by "Arvind Prabhakar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157668#comment-13157668 ]
Arvind Prabhakar commented on FLUME-822:
----------------------------------------
I have been looking through the code and found one place in the NetcatSource where there is an apparent transaction leak. This could be something that may have caused this problem. I will work on a general clean up patch for transaction handling and put it up for review soon.
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
> Attachments: flume-jdbc-lock-failure.log
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-822) JDBC channel lock acquisition failure
during take()
Posted by "E. Sammer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
E. Sammer updated FLUME-822:
----------------------------
Attachment: flume-jdbc-lock-failure.log
Attaching log with lock errors.
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
> Attachments: flume-jdbc-lock-failure.log
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-822) JDBC channel lock acquisition failure
during take()
Posted by "Arvind Prabhakar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar updated FLUME-822:
-----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch committed.
> JDBC channel lock acquisition failure during take()
> ---------------------------------------------------
>
> Key: FLUME-822
> URL: https://issues.apache.org/jira/browse/FLUME-822
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: Arvind Prabhakar
> Priority: Blocker
> Fix For: NG alpha 2
>
> Attachments: FLUME-822-1.patch, flume-jdbc-lock-failure.log
>
>
> The jdbc channel implementation (using Derby) results in a lock acquisition error upon take().
> It appears as if the event is successfully inserted by the source, but the sink fails to receive the event in the allotted time. The configuration used was as follows.
> ==
> localhost.sources = avro nc
> localhost.sinks = logger log2
> localhost.channels = ch1 ch2
> localhost.channels.ch1.type = memory
> localhost.channels.ch1.capacity = 10000
> localhost.sources.avro.type = avro
> localhost.sources.avro.bind = 0.0.0.0
> localhost.sources.avro.port = 41414
> localhost.sources.avro.channels = ch1
> localhost.sinks.logger.type = logger
> localhost.sinks.logger.channel = ch1
> #
> localhost.channels.ch2.type = jdbc
> localhost.sources.nc.type = netcat
> localhost.sources.nc.bind = 0.0.0.0
> localhost.sources.nc.port = 41415
> localhost.sources.nc.name = nc
> localhost.sources.nc.channels = ch2
> localhost.sinks.log2.type = logger
> localhost.sinks.log2.channel = ch2
> ==
> The path in use is nc(netcat) -> ch2(jdbc) -> log2(logger) with minimal configuration. I'll attach log output shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira