You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Will McQueen (Created) (JIRA)" <ji...@apache.org> on 2012/03/09 16:32:59 UTC
[jira] [Created] (FLUME-1026) Document Thread Safety Guarantees
Document Thread Safety Guarantees
---------------------------------
Key: FLUME-1026
URL: https://issues.apache.org/jira/browse/FLUME-1026
Project: Flume
Issue Type: Bug
Components: Docs
Affects Versions: v1.1.0
Reporter: Will McQueen
Fix For: v1.1.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1026) Document Thread Safety Guarantees
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233940#comment-13233940 ]
jiraposter@reviews.apache.org commented on FLUME-1026:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4400/#review6153
-----------------------------------------------------------
Juhani, looks great. I added a few suggestions around wording.
I agree with the general concerns. I'm not yet sure what I think we should do about it though.
flume-ng-core/src/main/java/org/apache/flume/Channel.java
<https://reviews.apache.org/r/4400/#comment13238>
"...they may be simultaneously accessed by" (missing "by")
flume-ng-core/src/main/java/org/apache/flume/PollableSource.java
<https://reviews.apache.org/r/4400/#comment13239>
How about:
"guaranteed to be called only by a single thread at a time, with no concurrency. Any other mechanism driving a pollable source must follow the same semantics."
This sounds a lot like a synchronized call and I wonder if it makes sense to call this out as required by PollableSource implementations.
flume-ng-core/src/main/java/org/apache/flume/Sink.java
<https://reviews.apache.org/r/4400/#comment13242>
How about switching a couple words around:
"While the Sink process call is guaranteed to only be accessed by a single thread, other calls may be concurrently accessed and should thus be protected."
- Mike
On 2012-03-19 03:59:00, Juhani Connolly wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4400/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-19 03:59:00)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. I went over the basic interfaces and added some javadoc for developers of new components.
bq.
bq. I am pretty concerned about the fact that pretty much everything is not thread safe because off the lack of guarantees provided by some of the interfaces(Configurable in particular).
bq.
bq. A lot of existing components assume thread safety(which is somewhat given by isolation of process() type methods to a single runner thread). Should we be fixing these components, or making some more guarantees about when and how configuration calls should be called(e.g. only callable on a stopped component?)
bq.
bq.
bq. This addresses bug Flume-1026.
bq. https://issues.apache.org/jira/browse/Flume-1026
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-core/src/main/java/org/apache/flume/Channel.java 9d1c14c
bq. flume-ng-core/src/main/java/org/apache/flume/PollableSource.java 8806040
bq. flume-ng-core/src/main/java/org/apache/flume/Sink.java 4a706cf
bq. flume-ng-core/src/main/java/org/apache/flume/SinkProcessor.java ea6000b
bq. flume-ng-core/src/main/java/org/apache/flume/Source.java f4e9481
bq. flume-ng-core/src/main/java/org/apache/flume/Transaction.java 5b1fee5
bq. flume-ng-core/src/main/java/org/apache/flume/conf/Configurable.java 0fa4839
bq.
bq. Diff: https://reviews.apache.org/r/4400/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Doc only patch
bq.
bq.
bq. Thanks,
bq.
bq. Juhani
bq.
bq.
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (FLUME-1026) Document Thread Safety Guarantees
Posted by "Juhani Connolly (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Juhani Connolly reassigned FLUME-1026:
--------------------------------------
Assignee: Juhani Connolly
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.1.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1026) Document Thread Safety Guarantees
Posted by "Juhani Connolly (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Juhani Connolly updated FLUME-1026:
-----------------------------------
Attachment: FLUME-1026.2.PATCH
Updated patch included in the review
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
> Attachments: FLUME-1026.2.PATCH
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1026) Document Thread Safety Guarantees
Posted by "Juhani Connolly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227351#comment-13227351 ]
Juhani Connolly commented on FLUME-1026:
----------------------------------------
Totally relevant... The problem lies in whether it is meant to be reconfigurable while running or if it should be stopped for reconfiguration. Other sinks like netcat won't switch ports when they get reconfigured.
We already know that reconfiguration while running does not work perfectly as is right now, another example is that when items are removed from a running cluster, they are not stopped(that issue is currently waiting on the rework of configuration that Hari is ddoing in FLUME-992)
The choice lies between accessing components in a thread safe manner or making all components(or at least configurable parts of them) internally thread safe. For the time being I will document this as the later, as some components do correctly change with the configuration. Whatever decision we come as to the final expected behavior of reconfiguring a running agent, a lot of components will need fixing.
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.1.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1026) Document Thread Safety Guarantees
Posted by "Jarek Jarcec Cecho (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227339#comment-13227339 ]
Jarek Jarcec Cecho commented on FLUME-1026:
-------------------------------------------
I'm not sure that it's completely relevant, but I've already reported one issue when reconfiguration breaks running agent in FLUME-862.
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.1.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1026) Document Thread Safety Guarantees
Posted by "Juhani Connolly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227328#comment-13227328 ]
Juhani Connolly commented on FLUME-1026:
----------------------------------------
I'll go over this, and post an initial pass.
There are certain aspects that I think have not been settled and will require discussion hopefully we can get a dialogue started about them.
One of these is the calling configure() on running components. IMO channels are expected to be thread safe while sinks/sources don't need to be as they are normally only accessed by a single thread. However as they are reconfigured, things have the potential to break.
Are we to expect all sinks/sources to be internally thread safe so as to be able to have configure() called at any time?
Or do we expect external locking?
A third option is that owning runners must be stopped before reconfiguration of the component and then restarted. The last method could mean a fair bit less overhead/complexity in synchronization.
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Fix For: v1.1.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1026) Document Thread Safety Guarantees
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235396#comment-13235396 ]
jiraposter@reviews.apache.org commented on FLUME-1026:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4400/#review6210
-----------------------------------------------------------
+1 from me, thanks for writing this up and we will need to resolve this pretty soon
- Mike
On 2012-03-22 04:00:33, Juhani Connolly wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4400/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-22 04:00:33)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. I went over the basic interfaces and added some javadoc for developers of new components.
bq.
bq. I am pretty concerned about the fact that pretty much everything is not thread safe because off the lack of guarantees provided by some of the interfaces(Configurable in particular).
bq.
bq. A lot of existing components assume thread safety(which is somewhat given by isolation of process() type methods to a single runner thread). Should we be fixing these components, or making some more guarantees about when and how configuration calls should be called(e.g. only callable on a stopped component?)
bq.
bq.
bq. This addresses bug Flume-1026.
bq. https://issues.apache.org/jira/browse/Flume-1026
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-core/src/main/java/org/apache/flume/Channel.java 9d1c14c
bq. flume-ng-core/src/main/java/org/apache/flume/PollableSource.java 8806040
bq. flume-ng-core/src/main/java/org/apache/flume/Sink.java 4a706cf
bq. flume-ng-core/src/main/java/org/apache/flume/SinkProcessor.java ea6000b
bq. flume-ng-core/src/main/java/org/apache/flume/Source.java f4e9481
bq. flume-ng-core/src/main/java/org/apache/flume/Transaction.java 5b1fee5
bq. flume-ng-core/src/main/java/org/apache/flume/conf/Configurable.java 0fa4839
bq.
bq. Diff: https://reviews.apache.org/r/4400/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Doc only patch
bq.
bq.
bq. Thanks,
bq.
bq. Juhani
bq.
bq.
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
> Attachments: FLUME-1026.2.PATCH
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1026) Document Thread Safety Guarantees
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235401#comment-13235401 ]
jiraposter@reviews.apache.org commented on FLUME-1026:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4400/#review6211
-----------------------------------------------------------
Ship it!
+1
- Arvind
On 2012-03-22 04:00:33, Juhani Connolly wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4400/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-22 04:00:33)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. I went over the basic interfaces and added some javadoc for developers of new components.
bq.
bq. I am pretty concerned about the fact that pretty much everything is not thread safe because off the lack of guarantees provided by some of the interfaces(Configurable in particular).
bq.
bq. A lot of existing components assume thread safety(which is somewhat given by isolation of process() type methods to a single runner thread). Should we be fixing these components, or making some more guarantees about when and how configuration calls should be called(e.g. only callable on a stopped component?)
bq.
bq.
bq. This addresses bug Flume-1026.
bq. https://issues.apache.org/jira/browse/Flume-1026
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-core/src/main/java/org/apache/flume/Channel.java 9d1c14c
bq. flume-ng-core/src/main/java/org/apache/flume/PollableSource.java 8806040
bq. flume-ng-core/src/main/java/org/apache/flume/Sink.java 4a706cf
bq. flume-ng-core/src/main/java/org/apache/flume/SinkProcessor.java ea6000b
bq. flume-ng-core/src/main/java/org/apache/flume/Source.java f4e9481
bq. flume-ng-core/src/main/java/org/apache/flume/Transaction.java 5b1fee5
bq. flume-ng-core/src/main/java/org/apache/flume/conf/Configurable.java 0fa4839
bq.
bq. Diff: https://reviews.apache.org/r/4400/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Doc only patch
bq.
bq.
bq. Thanks,
bq.
bq. Juhani
bq.
bq.
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
> Attachments: FLUME-1026.2.PATCH
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1026) Document Thread Safety Guarantees
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235415#comment-13235415 ]
Hudson commented on FLUME-1026:
-------------------------------
Integrated in flume-trunk #136 (See [https://builds.apache.org/job/flume-trunk/136/])
FLUME-1026. Document Thread Safety Guarantees.
(Juhani Connolly via Arvind Prabhakar) (Revision 1303665)
Result = SUCCESS
arvind : http://svn.apache.org/viewvc/?view=rev&rev=1303665
Files :
* /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/Channel.java
* /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/PollableSource.java
* /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/Sink.java
* /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/SinkProcessor.java
* /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/Source.java
* /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/Transaction.java
* /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/conf/Configurable.java
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
> Attachments: FLUME-1026.2.PATCH
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1026) Document Thread Safety Guarantees
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235322#comment-13235322 ]
jiraposter@reviews.apache.org commented on FLUME-1026:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4400/
-----------------------------------------------------------
(Updated 2012-03-22 04:00:33.408798)
Review request for Flume.
Changes
-------
Added in Mike's wording suggestions.
Will: yeah, I think that configuration is at the center of the whole threading issue and we need to address that before we come around to individual components.
Summary
-------
I went over the basic interfaces and added some javadoc for developers of new components.
I am pretty concerned about the fact that pretty much everything is not thread safe because off the lack of guarantees provided by some of the interfaces(Configurable in particular).
A lot of existing components assume thread safety(which is somewhat given by isolation of process() type methods to a single runner thread). Should we be fixing these components, or making some more guarantees about when and how configuration calls should be called(e.g. only callable on a stopped component?)
This addresses bug Flume-1026.
https://issues.apache.org/jira/browse/Flume-1026
Diffs (updated)
-----
flume-ng-core/src/main/java/org/apache/flume/Channel.java 9d1c14c
flume-ng-core/src/main/java/org/apache/flume/PollableSource.java 8806040
flume-ng-core/src/main/java/org/apache/flume/Sink.java 4a706cf
flume-ng-core/src/main/java/org/apache/flume/SinkProcessor.java ea6000b
flume-ng-core/src/main/java/org/apache/flume/Source.java f4e9481
flume-ng-core/src/main/java/org/apache/flume/Transaction.java 5b1fee5
flume-ng-core/src/main/java/org/apache/flume/conf/Configurable.java 0fa4839
Diff: https://reviews.apache.org/r/4400/diff
Testing
-------
Doc only patch
Thanks,
Juhani
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1026) Document Thread Safety Guarantees
Posted by "Arvind Prabhakar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar updated FLUME-1026:
------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch committed. Thanks Juhani!
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
> Attachments: FLUME-1026.2.PATCH
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1026) Document Thread Safety Guarantees
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232428#comment-13232428 ]
jiraposter@reviews.apache.org commented on FLUME-1026:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4400/
-----------------------------------------------------------
Review request for Flume.
Summary
-------
I went over the basic interfaces and added some javadoc for developers of new components.
I am pretty concerned about the fact that pretty much everything is not thread safe because off the lack of guarantees provided by some of the interfaces(Configurable in particular).
A lot of existing components assume thread safety(which is somewhat given by isolation of process() type methods to a single runner thread). Should we be fixing these components, or making some more guarantees about when and how configuration calls should be called(e.g. only callable on a stopped component?)
This addresses bug Flume-1026.
https://issues.apache.org/jira/browse/Flume-1026
Diffs
-----
flume-ng-core/src/main/java/org/apache/flume/Channel.java 9d1c14c
flume-ng-core/src/main/java/org/apache/flume/PollableSource.java 8806040
flume-ng-core/src/main/java/org/apache/flume/Sink.java 4a706cf
flume-ng-core/src/main/java/org/apache/flume/SinkProcessor.java ea6000b
flume-ng-core/src/main/java/org/apache/flume/Source.java f4e9481
flume-ng-core/src/main/java/org/apache/flume/Transaction.java 5b1fee5
flume-ng-core/src/main/java/org/apache/flume/conf/Configurable.java 0fa4839
Diff: https://reviews.apache.org/r/4400/diff
Testing
-------
Doc only patch
Thanks,
Juhani
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1026) Document Thread Safety Guarantees
Posted by "Juhani Connolly (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Juhani Connolly updated FLUME-1026:
-----------------------------------
Status: Patch Available (was: Open)
This patch just describes things as they are now.
There are general thread safety issues that should be dealt with and in my opinion, configuration is more or less at the core of those issues.
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
> Attachments: FLUME-1026.2.PATCH
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1026) Document Thread Safety Guarantees
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233948#comment-13233948 ]
jiraposter@reviews.apache.org commented on FLUME-1026:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4400/#review6156
-----------------------------------------------------------
I too am concerned about the thread safety. I'll take a look at some of the classes tonight or tomorrow, and will offer some suggestions. I'll be considering atomicity, visibility, and HB relationships per JCIP.
- Will
On 2012-03-19 03:59:00, Juhani Connolly wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4400/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-19 03:59:00)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. I went over the basic interfaces and added some javadoc for developers of new components.
bq.
bq. I am pretty concerned about the fact that pretty much everything is not thread safe because off the lack of guarantees provided by some of the interfaces(Configurable in particular).
bq.
bq. A lot of existing components assume thread safety(which is somewhat given by isolation of process() type methods to a single runner thread). Should we be fixing these components, or making some more guarantees about when and how configuration calls should be called(e.g. only callable on a stopped component?)
bq.
bq.
bq. This addresses bug Flume-1026.
bq. https://issues.apache.org/jira/browse/Flume-1026
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-core/src/main/java/org/apache/flume/Channel.java 9d1c14c
bq. flume-ng-core/src/main/java/org/apache/flume/PollableSource.java 8806040
bq. flume-ng-core/src/main/java/org/apache/flume/Sink.java 4a706cf
bq. flume-ng-core/src/main/java/org/apache/flume/SinkProcessor.java ea6000b
bq. flume-ng-core/src/main/java/org/apache/flume/Source.java f4e9481
bq. flume-ng-core/src/main/java/org/apache/flume/Transaction.java 5b1fee5
bq. flume-ng-core/src/main/java/org/apache/flume/conf/Configurable.java 0fa4839
bq.
bq. Diff: https://reviews.apache.org/r/4400/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Doc only patch
bq.
bq.
bq. Thanks,
bq.
bq. Juhani
bq.
bq.
> Document Thread Safety Guarantees
> ---------------------------------
>
> Key: FLUME-1026
> URL: https://issues.apache.org/jira/browse/FLUME-1026
> Project: Flume
> Issue Type: Bug
> Components: Docs
> Affects Versions: v1.1.0
> Reporter: Will McQueen
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira