You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "E. Sammer (JIRA)" <ji...@apache.org> on 2011/08/09 08:26:27 UTC
[jira] [Created] (FLUME-728) Flume NG refactoring
Flume NG refactoring
--------------------
Key: FLUME-728
URL: https://issues.apache.org/jira/browse/FLUME-728
Project: Flume
Issue Type: Epic
Components: Build, Docs, Master, Node, Shell, Sinks+Sources, Technical Debt, Test, Web
Reporter: E. Sammer
Assignee: E. Sammer
To solve certain known issues and limitations, Flume requires a refactoring of some core classes and systems. This bug is a parent issue to track the development of a "Flume NG" - a poorly named, but necessary refactoring. Subtasks should be added to track individual systems and components.
The following known issues are specifically to be addressed:
* Code complexity; Flume has evolved over the last few years and has a fair amount of extraneous code.
* Core component lifecycle standardization and control code (e.g. anything that can be start()ed or stop()ed, sources, sinks).
* (Static) Configuration access throughout the code base.
* Drastic simplification of common data paths (e.g. durability as an element of the source rather than a disconnected sink).
* Heartbeat and master rearchitecture.
* Renaming packages to org.apache.flume.
This is a large and far reaching set of tasks. The intent is to perform this work in a branch as to not disrupt immediate releases or short term forthcoming releases while still allowing open development in the community.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-728) Flume NG refactoring
Posted by "E. Sammer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
E. Sammer updated FLUME-728:
----------------------------
Affects Version/s: NG
> Flume NG refactoring
> --------------------
>
> Key: FLUME-728
> URL: https://issues.apache.org/jira/browse/FLUME-728
> Project: Flume
> Issue Type: Epic
> Components: Build, Docs, Master, Node, Shell, Sinks+Sources, Technical Debt, Test, Web
> Affects Versions: NG
> Reporter: E. Sammer
> Assignee: E. Sammer
> Labels: NG
>
> To solve certain known issues and limitations, Flume requires a refactoring of some core classes and systems. This bug is a parent issue to track the development of a "Flume NG" - a poorly named, but necessary refactoring. Subtasks should be added to track individual systems and components.
> The following known issues are specifically to be addressed:
> * Code complexity; Flume has evolved over the last few years and has a fair amount of extraneous code.
> * Core component lifecycle standardization and control code (e.g. anything that can be start()ed or stop()ed, sources, sinks).
> * (Static) Configuration access throughout the code base.
> * Drastic simplification of common data paths (e.g. durability as an element of the source rather than a disconnected sink).
> * Heartbeat and master rearchitecture.
> * Renaming packages to org.apache.flume.
> This is a large and far reaching set of tasks. The intent is to perform this work in a branch as to not disrupt immediate releases or short term forthcoming releases while still allowing open development in the community.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-728) Flume NG refactoring
Posted by "Arvind Prabhakar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103857#comment-13103857 ]
Arvind Prabhakar commented on FLUME-728:
----------------------------------------
@Jon: I agree - once the 728 branch is stable and has sufficient functionality, it should be put on the trunk.
> Flume NG refactoring
> --------------------
>
> Key: FLUME-728
> URL: https://issues.apache.org/jira/browse/FLUME-728
> Project: Flume
> Issue Type: Epic
> Components: Build, Docs, Master, Node, Shell, Sinks+Sources, Technical Debt, Test, Web
> Reporter: E. Sammer
> Assignee: E. Sammer
> Labels: NG
>
> To solve certain known issues and limitations, Flume requires a refactoring of some core classes and systems. This bug is a parent issue to track the development of a "Flume NG" - a poorly named, but necessary refactoring. Subtasks should be added to track individual systems and components.
> The following known issues are specifically to be addressed:
> * Code complexity; Flume has evolved over the last few years and has a fair amount of extraneous code.
> * Core component lifecycle standardization and control code (e.g. anything that can be start()ed or stop()ed, sources, sinks).
> * (Static) Configuration access throughout the code base.
> * Drastic simplification of common data paths (e.g. durability as an element of the source rather than a disconnected sink).
> * Heartbeat and master rearchitecture.
> * Renaming packages to org.apache.flume.
> This is a large and far reaching set of tasks. The intent is to perform this work in a branch as to not disrupt immediate releases or short term forthcoming releases while still allowing open development in the community.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (FLUME-728) Flume NG refactoring
Posted by "E. Sammer (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
E. Sammer resolved FLUME-728.
-----------------------------
Resolution: Fixed
I'm declaring this resolved having hit a reasonable milestone. At this point, specific bugs, feature requests, and tasks should be filed against the NG branch.
> Flume NG refactoring
> --------------------
>
> Key: FLUME-728
> URL: https://issues.apache.org/jira/browse/FLUME-728
> Project: Flume
> Issue Type: Epic
> Components: Build, Docs, Master, Node, Shell, Sinks+Sources, Technical Debt, Test, Web
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: E. Sammer
> Labels: NG
> Fix For: NG alpha 1
>
>
> To solve certain known issues and limitations, Flume requires a refactoring of some core classes and systems. This bug is a parent issue to track the development of a "Flume NG" - a poorly named, but necessary refactoring. Subtasks should be added to track individual systems and components.
> The following known issues are specifically to be addressed:
> * Code complexity; Flume has evolved over the last few years and has a fair amount of extraneous code.
> * Core component lifecycle standardization and control code (e.g. anything that can be start()ed or stop()ed, sources, sinks).
> * (Static) Configuration access throughout the code base.
> * Drastic simplification of common data paths (e.g. durability as an element of the source rather than a disconnected sink).
> * Heartbeat and master rearchitecture.
> * Renaming packages to org.apache.flume.
> This is a large and far reaching set of tasks. The intent is to perform this work in a branch as to not disrupt immediate releases or short term forthcoming releases while still allowing open development in the community.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-728) Flume NG refactoring
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132306#comment-13132306 ]
Hudson commented on FLUME-728:
------------------------------
Integrated in flume-728 #50 (See [https://builds.apache.org/job/flume-728/50/])
FLUME-728. Added a multi-threaded test to exercise JDBC channel.
arvind : http://svn.apache.org/viewvc/?view=rev&rev=1187155
Files :
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/MockEventUtils.java
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestJdbcChannelProvider.java
> Flume NG refactoring
> --------------------
>
> Key: FLUME-728
> URL: https://issues.apache.org/jira/browse/FLUME-728
> Project: Flume
> Issue Type: Epic
> Components: Build, Docs, Master, Node, Shell, Sinks+Sources, Technical Debt, Test, Web
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: E. Sammer
> Labels: NG
> Fix For: NG alpha 1
>
>
> To solve certain known issues and limitations, Flume requires a refactoring of some core classes and systems. This bug is a parent issue to track the development of a "Flume NG" - a poorly named, but necessary refactoring. Subtasks should be added to track individual systems and components.
> The following known issues are specifically to be addressed:
> * Code complexity; Flume has evolved over the last few years and has a fair amount of extraneous code.
> * Core component lifecycle standardization and control code (e.g. anything that can be start()ed or stop()ed, sources, sinks).
> * (Static) Configuration access throughout the code base.
> * Drastic simplification of common data paths (e.g. durability as an element of the source rather than a disconnected sink).
> * Heartbeat and master rearchitecture.
> * Renaming packages to org.apache.flume.
> This is a large and far reaching set of tasks. The intent is to perform this work in a branch as to not disrupt immediate releases or short term forthcoming releases while still allowing open development in the community.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-728) Flume NG refactoring
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081805#comment-13081805 ]
Jonathan Hsieh commented on FLUME-728:
--------------------------------------
Actually from reading up on svn repos, maybe we should create a 0.9.x branch for bug fixes, and do this on trunk/mainline instead.
> Flume NG refactoring
> --------------------
>
> Key: FLUME-728
> URL: https://issues.apache.org/jira/browse/FLUME-728
> Project: Flume
> Issue Type: Epic
> Components: Build, Docs, Master, Node, Shell, Sinks+Sources, Technical Debt, Test, Web
> Reporter: E. Sammer
> Assignee: E. Sammer
> Labels: NG
>
> To solve certain known issues and limitations, Flume requires a refactoring of some core classes and systems. This bug is a parent issue to track the development of a "Flume NG" - a poorly named, but necessary refactoring. Subtasks should be added to track individual systems and components.
> The following known issues are specifically to be addressed:
> * Code complexity; Flume has evolved over the last few years and has a fair amount of extraneous code.
> * Core component lifecycle standardization and control code (e.g. anything that can be start()ed or stop()ed, sources, sinks).
> * (Static) Configuration access throughout the code base.
> * Drastic simplification of common data paths (e.g. durability as an element of the source rather than a disconnected sink).
> * Heartbeat and master rearchitecture.
> * Renaming packages to org.apache.flume.
> This is a large and far reaching set of tasks. The intent is to perform this work in a branch as to not disrupt immediate releases or short term forthcoming releases while still allowing open development in the community.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-728) Flume NG refactoring
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081681#comment-13081681 ]
Jonathan Hsieh commented on FLUME-728:
--------------------------------------
+1. Let's create a new branch for this development. Maybe call it 0.10-masterng?
> Flume NG refactoring
> --------------------
>
> Key: FLUME-728
> URL: https://issues.apache.org/jira/browse/FLUME-728
> Project: Flume
> Issue Type: Epic
> Components: Build, Docs, Master, Node, Shell, Sinks+Sources, Technical Debt, Test, Web
> Reporter: E. Sammer
> Assignee: E. Sammer
> Labels: NG
>
> To solve certain known issues and limitations, Flume requires a refactoring of some core classes and systems. This bug is a parent issue to track the development of a "Flume NG" - a poorly named, but necessary refactoring. Subtasks should be added to track individual systems and components.
> The following known issues are specifically to be addressed:
> * Code complexity; Flume has evolved over the last few years and has a fair amount of extraneous code.
> * Core component lifecycle standardization and control code (e.g. anything that can be start()ed or stop()ed, sources, sinks).
> * (Static) Configuration access throughout the code base.
> * Drastic simplification of common data paths (e.g. durability as an element of the source rather than a disconnected sink).
> * Heartbeat and master rearchitecture.
> * Renaming packages to org.apache.flume.
> This is a large and far reaching set of tasks. The intent is to perform this work in a branch as to not disrupt immediate releases or short term forthcoming releases while still allowing open development in the community.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (FLUME-728) Flume NG refactoring
Posted by "E. Sammer (Closed) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
E. Sammer closed FLUME-728.
---------------------------
> Flume NG refactoring
> --------------------
>
> Key: FLUME-728
> URL: https://issues.apache.org/jira/browse/FLUME-728
> Project: Flume
> Issue Type: Epic
> Components: Build, Docs, Master, Node, Shell, Sinks+Sources, Technical Debt, Test, Web
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Assignee: E. Sammer
> Labels: NG
> Fix For: NG alpha 1
>
>
> To solve certain known issues and limitations, Flume requires a refactoring of some core classes and systems. This bug is a parent issue to track the development of a "Flume NG" - a poorly named, but necessary refactoring. Subtasks should be added to track individual systems and components.
> The following known issues are specifically to be addressed:
> * Code complexity; Flume has evolved over the last few years and has a fair amount of extraneous code.
> * Core component lifecycle standardization and control code (e.g. anything that can be start()ed or stop()ed, sources, sinks).
> * (Static) Configuration access throughout the code base.
> * Drastic simplification of common data paths (e.g. durability as an element of the source rather than a disconnected sink).
> * Heartbeat and master rearchitecture.
> * Renaming packages to org.apache.flume.
> This is a large and far reaching set of tasks. The intent is to perform this work in a branch as to not disrupt immediate releases or short term forthcoming releases while still allowing open development in the community.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira