You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Brock Noland (Created) (JIRA)" <ji...@apache.org> on 2012/02/20 00:26:34 UTC
[jira] [Created] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Context interface is too basic requiring boilerplate user code
--------------------------------------------------------------
Key: FLUME-978
URL: https://issues.apache.org/jira/browse/FLUME-978
Project: Flume
Issue Type: Improvement
Reporter: Brock Noland
Assignee: Brock Noland
Flume is filled with examples like so:
batchSize = Integer.parseInt(context.get("batch-size", String.class));
if (batchSize == null) {
batchSize = defaultBatchSize;
}
from AvroSink. The Context object should provide at a minimum:
* getBoolean(key)
* getBoolean(key, default)
* getInteger(key)
* getInteger(key, default)
* getLong(key)
* getLong(key, default)
Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. Despite this, the users of the context should be able to use the above convenience methods.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Brock Noland (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212009#comment-13212009 ]
Brock Noland commented on FLUME-978:
------------------------------------
@Arvind,
I agree that we could cache resulting parsed objects. The patch I just posted takes your suggestion that we can wait until this is an observed performance problem.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Reporter: Brock Noland
> Assignee: Brock Noland
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Arvind Prabhakar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar updated FLUME-978:
-----------------------------------
Resolution: Fixed
Fix Version/s: v1.1.0
Status: Resolved (was: Patch Available)
Patch committed to flume-728 branch. Thanks Brock!
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Fix For: v1.1.0
>
> Attachments: FLUME-978-0.patch, FLUME-978-1.patch, FLUME-978-2.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218909#comment-13218909 ]
jiraposter@reviews.apache.org commented on FLUME-978:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3968/#review5435
-----------------------------------------------------------
Ship it!
+1
- Arvind
On 2012-02-29 03:51:07, Brock Noland wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3968/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-29 03:51:07)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. 1) Changing backing structure of Context from object to String
bq. 2) Add getBoolean, getInteger, getLong to Context
bq. 3) Remove get(key, clazz)
bq. 4) Add javadoc to public methods of Context
bq. 5) Change backing map of Context to synchronized map since it seems this class could be accessed by multiple threads.
bq.
bq.
bq. This addresses bug FLUME-978.
bq. https://issues.apache.org/jira/browse/FLUME-978
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java 1cf1c0c
bq. flume-ng-core/src/main/java/org/apache/flume/Context.java f1c8f85
bq. flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorFactory.java a7d5f94
bq. flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java e79490e
bq. flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java c63d0a1
bq. flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java 1df580e
bq. flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java 5440631
bq. flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 45c031d
bq. flume-ng-core/src/main/java/org/apache/flume/sink/SinkGroup.java 1adc5ff
bq. flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 859f4fd
bq. flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 7b079f9
bq. flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
bq. flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java d205bbc
bq. flume-ng-core/src/test/java/org/apache/flume/TestContext.java a5e6aa8
bq. flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannel.java 3392dff
bq. flume-ng-core/src/test/java/org/apache/flume/sink/TestFailoverSinkProcessor.java 93ad3bf
bq. flume-ng-legacy-sources/flume-avro-source/src/main/java/org/apache/flume/source/avroLegacy/AvroLegacySource.java b1e67f7
bq. flume-ng-legacy-sources/flume-thrift-source/src/main/java/org/apache/flume/source/thriftLegacy/ThriftLegacySource.java 5fe270a
bq. flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 3da90a5
bq. flume-ng-sinks/flume-irc-sink/src/main/java/org/apache/flume/sink/irc/IRCSink.java 0a5498f
bq.
bq. Diff: https://reviews.apache.org/r/3968/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests added for all new methods and some exiting methods.
bq.
bq. All unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Brock
bq.
bq.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Fix For: v1.1.0
>
> Attachments: FLUME-978-0.patch, FLUME-978-1.patch, FLUME-978-2.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212006#comment-13212006 ]
jiraposter@reviews.apache.org commented on FLUME-978:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3968/
-----------------------------------------------------------
Review request for Flume.
Summary
-------
1) Changing backing structure of Context from object to String
2) Add getBoolean, getInteger, getLong to Context
3) Remove get(key, clazz)
4) Add javadoc to public methods of Context
5) Change backing map of Context to synchronized map since it seems this class could be accessed by multiple threads.
This addresses bug FLUME-978.
https://issues.apache.org/jira/browse/FLUME-978
Diffs
-----
flume-ng-sinks/flume-irc-sink/src/main/java/org/apache/flume/sink/irc/IRCSink.java 0a5498f
flume-ng-legacy-sources/flume-avro-source/src/main/java/org/apache/flume/source/avroLegacy/AvroLegacySource.java b1e67f7
flume-ng-legacy-sources/flume-thrift-source/src/main/java/org/apache/flume/source/thriftLegacy/ThriftLegacySource.java 5fe270a
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 3da90a5
flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java d205bbc
flume-ng-core/src/test/java/org/apache/flume/TestContext.java a5e6aa8
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 7b079f9
flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 45c031d
flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java 5440631
flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java 1df580e
flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java c63d0a1
flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java 6a17f06
flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorFactory.java a7d5f94
flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java 1cf1c0c
flume-ng-core/src/main/java/org/apache/flume/Context.java f1c8f85
Diff: https://reviews.apache.org/r/3968/diff
Testing
-------
Unit tests added for all new methods and some exiting methods.
All unit tests pass.
Thanks,
Brock
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Reporter: Brock Noland
> Assignee: Brock Noland
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218855#comment-13218855 ]
jiraposter@reviews.apache.org commented on FLUME-978:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3968/#review5427
-----------------------------------------------------------
Thanks for the patch Brock. Changes look good. A couple of suggestions:
flume-ng-core/src/main/java/org/apache/flume/Context.java
<https://reviews.apache.org/r/3968/#comment11830>
Another thing to consider while you are at it is to provide a constructor that takes in a Map<String, String>. That way the two step call:
Context c = new Context();
c.putAll(config);
Can be collapsed into one:
Context c = new Context(config);
If you decide to do this, I suggest making a copy of the map that is passed in to insulate it from accidental external modification.
flume-ng-core/src/main/java/org/apache/flume/Context.java
<https://reviews.apache.org/r/3968/#comment11829>
Since the parameters member variable is used for synchronization, you cannot replace it's reference. Replacing it's reference will break the thread safety of this class.
Alternatively, you can instantiate parameters at the point of declaration or in constructor, and the clear() could then be applied to it directly.
- Arvind
On 2012-02-28 16:49:24, Brock Noland wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3968/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-28 16:49:24)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. 1) Changing backing structure of Context from object to String
bq. 2) Add getBoolean, getInteger, getLong to Context
bq. 3) Remove get(key, clazz)
bq. 4) Add javadoc to public methods of Context
bq. 5) Change backing map of Context to synchronized map since it seems this class could be accessed by multiple threads.
bq.
bq.
bq. This addresses bug FLUME-978.
bq. https://issues.apache.org/jira/browse/FLUME-978
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java 1cf1c0c
bq. flume-ng-core/src/main/java/org/apache/flume/Context.java f1c8f85
bq. flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorFactory.java a7d5f94
bq. flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java e79490e
bq. flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java c63d0a1
bq. flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java 1df580e
bq. flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java 5440631
bq. flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 45c031d
bq. flume-ng-core/src/main/java/org/apache/flume/sink/SinkGroup.java 1adc5ff
bq. flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 859f4fd
bq. flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 7b079f9
bq. flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
bq. flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java d205bbc
bq. flume-ng-core/src/test/java/org/apache/flume/TestContext.java a5e6aa8
bq. flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannel.java 3392dff
bq. flume-ng-core/src/test/java/org/apache/flume/sink/TestFailoverSinkProcessor.java 93ad3bf
bq. flume-ng-legacy-sources/flume-avro-source/src/main/java/org/apache/flume/source/avroLegacy/AvroLegacySource.java b1e67f7
bq. flume-ng-legacy-sources/flume-thrift-source/src/main/java/org/apache/flume/source/thriftLegacy/ThriftLegacySource.java 5fe270a
bq. flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 3da90a5
bq. flume-ng-sinks/flume-irc-sink/src/main/java/org/apache/flume/sink/irc/IRCSink.java 0a5498f
bq.
bq. Diff: https://reviews.apache.org/r/3968/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests added for all new methods and some exiting methods.
bq.
bq. All unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Brock
bq.
bq.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-978-0.patch, FLUME-978-1.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Brock Noland (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated FLUME-978:
-------------------------------
Affects Version/s: v1.0.0
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-978-0.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Brock Noland (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated FLUME-978:
-------------------------------
Attachment: FLUME-978-1.patch
Attached updated patch.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-978-0.patch, FLUME-978-1.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218347#comment-13218347 ]
jiraposter@reviews.apache.org commented on FLUME-978:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3968/
-----------------------------------------------------------
(Updated 2012-02-28 16:49:24.308965)
Review request for Flume.
Changes
-------
Updated patch due to trunk commits.
Summary
-------
1) Changing backing structure of Context from object to String
2) Add getBoolean, getInteger, getLong to Context
3) Remove get(key, clazz)
4) Add javadoc to public methods of Context
5) Change backing map of Context to synchronized map since it seems this class could be accessed by multiple threads.
This addresses bug FLUME-978.
https://issues.apache.org/jira/browse/FLUME-978
Diffs (updated)
-----
flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java 1cf1c0c
flume-ng-core/src/main/java/org/apache/flume/Context.java f1c8f85
flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorFactory.java a7d5f94
flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java e79490e
flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java c63d0a1
flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java 1df580e
flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java 5440631
flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 45c031d
flume-ng-core/src/main/java/org/apache/flume/sink/SinkGroup.java 1adc5ff
flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 859f4fd
flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 7b079f9
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java d205bbc
flume-ng-core/src/test/java/org/apache/flume/TestContext.java a5e6aa8
flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannel.java 3392dff
flume-ng-core/src/test/java/org/apache/flume/sink/TestFailoverSinkProcessor.java 93ad3bf
flume-ng-legacy-sources/flume-avro-source/src/main/java/org/apache/flume/source/avroLegacy/AvroLegacySource.java b1e67f7
flume-ng-legacy-sources/flume-thrift-source/src/main/java/org/apache/flume/source/thriftLegacy/ThriftLegacySource.java 5fe270a
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 3da90a5
flume-ng-sinks/flume-irc-sink/src/main/java/org/apache/flume/sink/irc/IRCSink.java 0a5498f
Diff: https://reviews.apache.org/r/3968/diff
Testing
-------
Unit tests added for all new methods and some exiting methods.
All unit tests pass.
Thanks,
Brock
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-978-0.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218919#comment-13218919 ]
Hudson commented on FLUME-978:
------------------------------
Integrated in flume-728 #117 (See [https://builds.apache.org/job/flume-728/117/])
FLUME-978. Context interface is too basic requiring boilerplate user code.
(Brock Noland via Arvind Prabhakar) (Revision 1294969)
Result = SUCCESS
arvind : http://svn.apache.org/viewvc/?view=rev&rev=1294969
Files :
* /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/Context.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorFactory.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/sink/SinkGroup.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/test/java/org/apache/flume/TestContext.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannel.java
* /incubator/flume/branches/flume-728/flume-ng-core/src/test/java/org/apache/flume/sink/TestFailoverSinkProcessor.java
* /incubator/flume/branches/flume-728/flume-ng-legacy-sources/flume-avro-source/src/main/java/org/apache/flume/source/avroLegacy/AvroLegacySource.java
* /incubator/flume/branches/flume-728/flume-ng-legacy-sources/flume-thrift-source/src/main/java/org/apache/flume/source/thriftLegacy/ThriftLegacySource.java
* /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
* /incubator/flume/branches/flume-728/flume-ng-sinks/flume-irc-sink/src/main/java/org/apache/flume/sink/irc/IRCSink.java
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Fix For: v1.1.0
>
> Attachments: FLUME-978-0.patch, FLUME-978-1.patch, FLUME-978-2.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218856#comment-13218856 ]
jiraposter@reviews.apache.org commented on FLUME-978:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3968/#review5428
-----------------------------------------------------------
For some reason the following comment did not show up in my last review. Submitting it again.
flume-ng-core/src/main/java/org/apache/flume/Context.java
<https://reviews.apache.org/r/3968/#comment11831>
This call should be wrapped in synchronized(parameters) block as it may iterate over the contents of the map.
- Arvind
On 2012-02-28 16:49:24, Brock Noland wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3968/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-28 16:49:24)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. 1) Changing backing structure of Context from object to String
bq. 2) Add getBoolean, getInteger, getLong to Context
bq. 3) Remove get(key, clazz)
bq. 4) Add javadoc to public methods of Context
bq. 5) Change backing map of Context to synchronized map since it seems this class could be accessed by multiple threads.
bq.
bq.
bq. This addresses bug FLUME-978.
bq. https://issues.apache.org/jira/browse/FLUME-978
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java 1cf1c0c
bq. flume-ng-core/src/main/java/org/apache/flume/Context.java f1c8f85
bq. flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorFactory.java a7d5f94
bq. flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java e79490e
bq. flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java c63d0a1
bq. flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java 1df580e
bq. flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java 5440631
bq. flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 45c031d
bq. flume-ng-core/src/main/java/org/apache/flume/sink/SinkGroup.java 1adc5ff
bq. flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 859f4fd
bq. flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 7b079f9
bq. flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
bq. flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java d205bbc
bq. flume-ng-core/src/test/java/org/apache/flume/TestContext.java a5e6aa8
bq. flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannel.java 3392dff
bq. flume-ng-core/src/test/java/org/apache/flume/sink/TestFailoverSinkProcessor.java 93ad3bf
bq. flume-ng-legacy-sources/flume-avro-source/src/main/java/org/apache/flume/source/avroLegacy/AvroLegacySource.java b1e67f7
bq. flume-ng-legacy-sources/flume-thrift-source/src/main/java/org/apache/flume/source/thriftLegacy/ThriftLegacySource.java 5fe270a
bq. flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 3da90a5
bq. flume-ng-sinks/flume-irc-sink/src/main/java/org/apache/flume/sink/irc/IRCSink.java 0a5498f
bq.
bq. Diff: https://reviews.apache.org/r/3968/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests added for all new methods and some exiting methods.
bq.
bq. All unit tests pass.
bq.
bq.
bq. Thanks,
bq.
bq. Brock
bq.
bq.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-978-0.patch, FLUME-978-1.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Brock Noland (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated FLUME-978:
-------------------------------
Description:
Flume is filled with examples like so:
batchSize = Integer.parseInt(context.get("batch-size", String.class));
if (batchSize == null) {
batchSize = defaultBatchSize;
}
from AvroSink. The Context object should provide at a minimum:
* getBoolean(key)
* getBoolean(key, default)
* getInteger(key)
* getInteger(key, default)
* getLong(key)
* getLong(key, default)
Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
return Boolean.class.cast(String);
Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
was:
Flume is filled with examples like so:
batchSize = Integer.parseInt(context.get("batch-size", String.class));
if (batchSize == null) {
batchSize = defaultBatchSize;
}
from AvroSink. The Context object should provide at a minimum:
* getBoolean(key)
* getBoolean(key, default)
* getInteger(key)
* getInteger(key, default)
* getLong(key)
* getLong(key, default)
Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. Despite this, the users of the context should be able to use the above convenience methods.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Reporter: Brock Noland
> Assignee: Brock Noland
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Hari Shreedharan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218376#comment-13218376 ]
Hari Shreedharan commented on FLUME-978:
----------------------------------------
This will be very useful for the configuration refactoring I am working on. Will be great to see this checked in asap. Thanks!
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-978-0.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218882#comment-13218882 ]
jiraposter@reviews.apache.org commented on FLUME-978:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3968/
-----------------------------------------------------------
(Updated 2012-02-29 03:51:07.275144)
Review request for Flume.
Changes
-------
Updated based on review.
Summary
-------
1) Changing backing structure of Context from object to String
2) Add getBoolean, getInteger, getLong to Context
3) Remove get(key, clazz)
4) Add javadoc to public methods of Context
5) Change backing map of Context to synchronized map since it seems this class could be accessed by multiple threads.
This addresses bug FLUME-978.
https://issues.apache.org/jira/browse/FLUME-978
Diffs (updated)
-----
flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java 1cf1c0c
flume-ng-core/src/main/java/org/apache/flume/Context.java f1c8f85
flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorFactory.java a7d5f94
flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java e79490e
flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java c63d0a1
flume-ng-core/src/main/java/org/apache/flume/channel/PseudoTxnMemoryChannel.java 1df580e
flume-ng-core/src/main/java/org/apache/flume/sink/AvroSink.java 5440631
flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 45c031d
flume-ng-core/src/main/java/org/apache/flume/sink/SinkGroup.java 1adc5ff
flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 859f4fd
flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 7b079f9
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java d205bbc
flume-ng-core/src/test/java/org/apache/flume/TestContext.java a5e6aa8
flume-ng-core/src/test/java/org/apache/flume/channel/TestMemoryChannel.java 3392dff
flume-ng-core/src/test/java/org/apache/flume/sink/TestFailoverSinkProcessor.java 93ad3bf
flume-ng-legacy-sources/flume-avro-source/src/main/java/org/apache/flume/source/avroLegacy/AvroLegacySource.java b1e67f7
flume-ng-legacy-sources/flume-thrift-source/src/main/java/org/apache/flume/source/thriftLegacy/ThriftLegacySource.java 5fe270a
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 3da90a5
flume-ng-sinks/flume-irc-sink/src/main/java/org/apache/flume/sink/irc/IRCSink.java 0a5498f
Diff: https://reviews.apache.org/r/3968/diff
Testing
-------
Unit tests added for all new methods and some exiting methods.
All unit tests pass.
Thanks,
Brock
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-978-0.patch, FLUME-978-1.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Arvind Prabhakar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211650#comment-13211650 ]
Arvind Prabhakar commented on FLUME-978:
----------------------------------------
I agree that at the basic level all configuration is expressed as strings. However, once an object has been resolved, it may be beneficial to cache it within the context implementation to ensure that it can be obtained quickly if necessary. We can address the caching later too if it becomes evident that it will help with performance of the system.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Reporter: Brock Noland
> Assignee: Brock Noland
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Brock Noland (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated FLUME-978:
-------------------------------
Attachment: FLUME-978-0.patch
Attaching patch which is on review board.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-978-0.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Brock Noland (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated FLUME-978:
-------------------------------
Status: Patch Available (was: In Progress)
Marking "Patch Available" as this is ready for review.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Reporter: Brock Noland
> Assignee: Brock Noland
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Brock Noland (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated FLUME-978:
-------------------------------
Attachment: FLUME-978-2.patch
Updated patch attached.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-978-0.patch, FLUME-978-1.patch, FLUME-978-2.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Brock Noland (Work started) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on FLUME-978 started by Brock Noland.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Reporter: Brock Noland
> Assignee: Brock Noland
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (FLUME-978) Context interface is too
basic requiring boilerplate user code
Posted by "Hari Shreedharan (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218376#comment-13218376 ]
Hari Shreedharan edited comment on FLUME-978 at 2/28/12 5:29 PM:
-----------------------------------------------------------------
This will be very useful for the configuration refactoring I am working on. Will be great to see this checked in asap. Thanks Brock!
was (Author: hshreedharan):
This will be very useful for the configuration refactoring I am working on. Will be great to see this checked in asap. Thanks!
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.0.0
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: FLUME-978-0.patch, FLUME-978-1.patch
>
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-978) Context interface is too basic
requiring boilerplate user code
Posted by "Brock Noland (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211602#comment-13211602 ]
Brock Noland commented on FLUME-978:
------------------------------------
Give that the properties file will populate the Context will Strings but the users want to get fully typed objects, I think it makes sense to internally change the structure to be String storage.
> Context interface is too basic requiring boilerplate user code
> --------------------------------------------------------------
>
> Key: FLUME-978
> URL: https://issues.apache.org/jira/browse/FLUME-978
> Project: Flume
> Issue Type: Improvement
> Reporter: Brock Noland
> Assignee: Brock Noland
>
> Flume is filled with examples like so:
> batchSize = Integer.parseInt(context.get("batch-size", String.class));
> if (batchSize == null) {
> batchSize = defaultBatchSize;
> }
> from AvroSink. The Context object should provide at a minimum:
> * getBoolean(key)
> * getBoolean(key, default)
> * getInteger(key)
> * getInteger(key, default)
> * getLong(key)
> * getLong(key, default)
> Additionally, the Context object outside of tests, is populated via FlumeConfiguration which in the end is a properties file. In this common case, all the values in the Context object will be Strings. In this case, if we do the obvious and simply provide wrappers for getBoolean, we end up executing:
> return Boolean.class.cast(String);
> Users of the Context object should not care where the values come from, only that they will be returned the correct object type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira