You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Arvind Prabhakar (Created) (JIRA)" <ji...@apache.org> on 2012/02/13 04:25:06 UTC
[jira] [Created] (FLUME-966) Migrate lifecycle management to Guava
service implemetation
Migrate lifecycle management to Guava service implemetation
-----------------------------------------------------------
Key: FLUME-966
URL: https://issues.apache.org/jira/browse/FLUME-966
Project: Flume
Issue Type: Task
Affects Versions: v1.0.0
Reporter: Arvind Prabhakar
Currently flume uses a home grown lifecycle management mechanism. In order to simplify the code, we want to migrate the current implementation to use Guava libraries' service management implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-966) Migrate lifecycle management to Guava
service implemetation
Posted by "Juhani Connolly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Juhani Connolly updated FLUME-966:
----------------------------------
Attachment: FLUME-966-alpha.patch
One way to handle the reconfiguration when dealing with Suppliers.
This affect most of the framework so comes as a huge patch.
I've ironed out most of the unit tests, but I'm having issues with FileChannels tests(with the locks) that I'm not sure are related.
This isn't intended for inclusion now, but I would appreciate if people could have a look at it and think this is worth continuing on(since the changes are pretty far-reaching).
As mentioned elsewhere, this will probably need a separate branch for polish though I would like to come up with a clear target of when/how it would be merged back with trunk(I'd say after the next release would be good?)
> Migrate lifecycle management to Guava service implemetation
> -----------------------------------------------------------
>
> Key: FLUME-966
> URL: https://issues.apache.org/jira/browse/FLUME-966
> Project: Flume
> Issue Type: Task
> Affects Versions: v1.0.0
> Reporter: Arvind Prabhakar
> Assignee: Juhani Connolly
> Attachments: FLUME-966-0.patch, FLUME-966-alpha.patch, FLUME-966.patch
>
>
> Currently flume uses a home grown lifecycle management mechanism. In order to simplify the code, we want to migrate the current implementation to use Guava libraries' service management implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-966) Migrate lifecycle management to Guava
service implemetation
Posted by "Brock Noland (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated FLUME-966:
-------------------------------
Attachment: FLUME-966-0.patch
Attached patch is not for committing. Just something I put together while looking at this. It implements a Supervised and Unsupervised FlumeService.
> Migrate lifecycle management to Guava service implemetation
> -----------------------------------------------------------
>
> Key: FLUME-966
> URL: https://issues.apache.org/jira/browse/FLUME-966
> Project: Flume
> Issue Type: Task
> Affects Versions: v1.0.0
> Reporter: Arvind Prabhakar
> Attachments: FLUME-966-0.patch
>
>
> Currently flume uses a home grown lifecycle management mechanism. In order to simplify the code, we want to migrate the current implementation to use Guava libraries' service management implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-966) Migrate lifecycle management to
Guava service implemetation
Posted by "Juhani Connolly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291679#comment-13291679 ]
Juhani Connolly commented on FLUME-966:
---------------------------------------
I'm working on this atm unless Brock is keen on continuing from his earlier patch. Wanted to post current progress before the weekend but not satisfied with what I have right now.
> Migrate lifecycle management to Guava service implemetation
> -----------------------------------------------------------
>
> Key: FLUME-966
> URL: https://issues.apache.org/jira/browse/FLUME-966
> Project: Flume
> Issue Type: Task
> Affects Versions: v1.0.0
> Reporter: Arvind Prabhakar
> Assignee: Juhani Connolly
> Attachments: FLUME-966-0.patch
>
>
> Currently flume uses a home grown lifecycle management mechanism. In order to simplify the code, we want to migrate the current implementation to use Guava libraries' service management implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-966) Migrate lifecycle management to Guava
service implemetation
Posted by "Juhani Connolly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Juhani Connolly updated FLUME-966:
----------------------------------
Attachment: FLUME-966.patch
Rough implementation of lifecycle management framework for discussion.
Seems like review board is down, I'll post it up there later.
Some notes. A lot of this is just personal preference and open to change. One can note looking at the new sink runner that using the guava service api simplifies things a fair bit
- generally simplifying most stuff
- no central thread pool. Processes are expected to last a long time,
so recycling of threads is not a target
- dropping the current state/desired state model to move to the guava
model of new/starting/running/stopping/terminated.
- Supervising something will start it, and unsupervising it will stop it
- terminating a supervisor will terminate associated service
- a supervisor will create a new service using a supplier every time
the old one fails. The supplier is responsible for providing a
freshly configured service with no state.
- the component interface is just Service + NamedComponent. I would
expect Sink/Source runners to extend AbstractExecutionThreadService,
while "passive" components like channels would extend
AbstractIdleService. I'll add some base classes later along with
rudimentary tests
- a reconfiguration would probably involve unsupervising
everything(sources->sinks->channels), than configuring an item at a
time.
> Migrate lifecycle management to Guava service implemetation
> -----------------------------------------------------------
>
> Key: FLUME-966
> URL: https://issues.apache.org/jira/browse/FLUME-966
> Project: Flume
> Issue Type: Task
> Affects Versions: v1.0.0
> Reporter: Arvind Prabhakar
> Assignee: Juhani Connolly
> Attachments: FLUME-966-0.patch, FLUME-966.patch
>
>
> Currently flume uses a home grown lifecycle management mechanism. In order to simplify the code, we want to migrate the current implementation to use Guava libraries' service management implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-966) Migrate lifecycle management to
Guava service implemetation
Posted by "Arvind Prabhakar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233607#comment-13233607 ]
Arvind Prabhakar commented on FLUME-966:
----------------------------------------
Thanks for the patch Brock. Can you please post it to review board.
> Migrate lifecycle management to Guava service implemetation
> -----------------------------------------------------------
>
> Key: FLUME-966
> URL: https://issues.apache.org/jira/browse/FLUME-966
> Project: Flume
> Issue Type: Task
> Affects Versions: v1.0.0
> Reporter: Arvind Prabhakar
> Attachments: FLUME-966-0.patch
>
>
> Currently flume uses a home grown lifecycle management mechanism. In order to simplify the code, we want to migrate the current implementation to use Guava libraries' service management implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-966) Migrate lifecycle management to
Guava service implemetation
Posted by "Brock Noland (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206671#comment-13206671 ]
Brock Noland commented on FLUME-966:
------------------------------------
Good doc http://code.google.com/p/guava-libraries/wiki/ServiceExplained
> Migrate lifecycle management to Guava service implemetation
> -----------------------------------------------------------
>
> Key: FLUME-966
> URL: https://issues.apache.org/jira/browse/FLUME-966
> Project: Flume
> Issue Type: Task
> Affects Versions: v1.0.0
> Reporter: Arvind Prabhakar
>
> Currently flume uses a home grown lifecycle management mechanism. In order to simplify the code, we want to migrate the current implementation to use Guava libraries' service management implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (FLUME-966) Migrate lifecycle management to Guava
service implemetation
Posted by "Juhani Connolly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Juhani Connolly reassigned FLUME-966:
-------------------------------------
Assignee: Juhani Connolly
> Migrate lifecycle management to Guava service implemetation
> -----------------------------------------------------------
>
> Key: FLUME-966
> URL: https://issues.apache.org/jira/browse/FLUME-966
> Project: Flume
> Issue Type: Task
> Affects Versions: v1.0.0
> Reporter: Arvind Prabhakar
> Assignee: Juhani Connolly
> Attachments: FLUME-966-0.patch
>
>
> Currently flume uses a home grown lifecycle management mechanism. In order to simplify the code, we want to migrate the current implementation to use Guava libraries' service management implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira