You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Will McQueen (Created) (JIRA)" <ji...@apache.org> on 2012/02/27 18:58:48 UTC

[jira] [Created] (FLUME-1001) Support custom processors

Support custom processors
-------------------------

                 Key: FLUME-1001
                 URL: https://issues.apache.org/jira/browse/FLUME-1001
             Project: Flume
          Issue Type: Improvement
          Components: Sinks+Sources
    Affects Versions: v1.1.0
         Environment: RHEL 6.2 64-bit
            Reporter: Will McQueen
             Fix For: v1.1.0


Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274032#comment-13274032 ] 

jiraposter@reviews.apache.org commented on FLUME-1001:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4152/#review7829
-----------------------------------------------------------

Ship it!


+1


- Arvind


On 2012-05-09 09:56:14, Juhani Connolly wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4152/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-05-09 09:56:14)
bq.  
bq.  
bq.  Review request for Flume.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.
bq.  
bq.  
bq.  This addresses bug FLUME-1001.
bq.      https://issues.apache.org/jira/browse/FLUME-1001
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    flume-ng-core/src/test/java/org/apache/flume/sink/SinkProcessorFactoryTest.java PRE-CREATION 
bq.    flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java d397bd3 
bq.    flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 3ff6737 
bq.  
bq.  Diff: https://reviews.apache.org/r/4152/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment
bq.  
bq.  Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Juhani
bq.  
bq.


                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.5.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247134#comment-13247134 ] 

jiraposter@reviews.apache.org commented on FLUME-1001:
------------------------------------------------------



bq.  On 2012-04-05 03:18:45, Brock Noland wrote:
bq.  > looks good to me, just one nit. After you address to the nit please attache the patch to the JIRA.
bq.  
bq.  Juhani Connolly wrote:
bq.      Done and updated the issue.
bq.      This review is against the flume-728 so can't update the review file here, but it's just one row

Yep, I think we can just attach the patch to the JIRA since it's just a single line.


- Brock


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4152/#review6704
-----------------------------------------------------------


On 2012-03-05 03:25:20, Juhani Connolly wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4152/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-03-05 03:25:20)
bq.  
bq.  
bq.  Review request for Flume.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.
bq.  
bq.  
bq.  This addresses bug FLUME-1001.
bq.      https://issues.apache.org/jira/browse/FLUME-1001
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION 
bq.    flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b 
bq.    flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e 
bq.  
bq.  Diff: https://reviews.apache.org/r/4152/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment
bq.  
bq.  Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Juhani
bq.  
bq.


                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.3.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271259#comment-13271259 ] 

jiraposter@reviews.apache.org commented on FLUME-1001:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4152/
-----------------------------------------------------------

(Updated 2012-05-09 09:56:14.961414)


Review request for Flume.


Changes
-------

Updated against trunk


Summary
-------

Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.


This addresses bug FLUME-1001.
    https://issues.apache.org/jira/browse/FLUME-1001


Diffs (updated)
-----

  flume-ng-core/src/test/java/org/apache/flume/sink/SinkProcessorFactoryTest.java PRE-CREATION 
  flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java d397bd3 
  flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 3ff6737 

Diff: https://reviews.apache.org/r/4152/diff


Testing
-------

Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works


Thanks,

Juhani


                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.4.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (FLUME-1001) Support custom processors

Posted by "Juhani Connolly (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Juhani Connolly updated FLUME-1001:
-----------------------------------

    Attachment: FLUME-1001.4.patch

Reuploading this with a cleaner patch file, tests pass and the warn/info matter from the review https://reviews.apache.org/r/4152/ is fixed
                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.4.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (FLUME-1001) Support custom processors

Posted by "Juhani Connolly (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Juhani Connolly updated FLUME-1001:
-----------------------------------

    Attachment:     (was: FLUME-1001.2.patch)
    
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.3.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (FLUME-1001) Support custom processors

Posted by "Juhani Connolly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Juhani Connolly updated FLUME-1001:
-----------------------------------

    Attachment: FLUME-1001.5.patch

Updated version for current trunk(as posted to reviewboard)
                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.5.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (FLUME-1001) Support custom processors

Posted by "Juhani Connolly (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Juhani Connolly updated FLUME-1001:
-----------------------------------

    Status: Patch Available  (was: Open)
    
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.1.0
>
>         Attachments: FLUME-1001.2.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "Arvind Prabhakar (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268108#comment-13268108 ] 

Arvind Prabhakar commented on FLUME-1001:
-----------------------------------------

Apologies for not getting to this patch earlier. Can you please rebase the patch to the trunk? I will be happy to commit it.

                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.4.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221041#comment-13221041 ] 

jiraposter@reviews.apache.org commented on FLUME-1001:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4152/#review5557
-----------------------------------------------------------



flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java
<https://reviews.apache.org/r/4152/#comment12021>

    For consistency with the other types (SinkType, SourceType, ...), can we please put OTHER at the top of the enum listing?
    
    Also for same consistency, can we have the comment read:
    
      /**
       * Place holder for custom sink processors not part of this enumeration.
       */



flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
<https://reviews.apache.org/r/4152/#comment12035>

    Before this line:
    Preconditions.checkNotNull(context);
    Preconditions.checkNotNull(sinks);
    logger.debug("Creating instance of sink processor name {}, type {}", name, type);



flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
<https://reviews.apache.org/r/4152/#comment12032>

    => "Sink processor type {} is a custom type"
    
    (this string output matches the convention used by DefaultSourceFactory)



flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
<https://reviews.apache.org/r/4152/#comment12041>

    add "sink", remove comma:
    
    =>"Unable to create sink processor type: "
    



flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java
<https://reviews.apache.org/r/4152/#comment12028>

    Thank you for this test. It looks like it confirms that the FQDN and the enum name of the sink processor each return return the same SinkProcessor instance. I see that the intent of using the FQDN is to test the "OTHER" functionality.


- Will


On 2012-03-02 07:51:49, Juhani Connolly wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4152/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-03-02 07:51:49)
bq.  
bq.  
bq.  Review request for Flume.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.
bq.  
bq.  
bq.  This addresses bug FLUME-1001.
bq.      https://issues.apache.org/jira/browse/FLUME-1001
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b 
bq.    flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e 
bq.    flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/4152/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment
bq.  
bq.  Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Juhani
bq.  
bq.


                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.1.0
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (FLUME-1001) Support custom processors

Posted by "Juhani Connolly (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Juhani Connolly updated FLUME-1001:
-----------------------------------

    Attachment: FLUME-1001.2.patch
    
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.1.0
>
>         Attachments: FLUME-1001.2.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274038#comment-13274038 ] 

Hudson commented on FLUME-1001:
-------------------------------

Integrated in flume-trunk #203 (See [https://builds.apache.org/job/flume-trunk/203/])
    FLUME-1001. Support custom sink processors.

(Juhani Connolly via Arvind Prabhakar) (Revision 1337577)

     Result = SUCCESS
arvind : http://svn.apache.org/viewvc/?view=rev&rev=1337577
Files : 
* /incubator/flume/trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java
* /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
* /incubator/flume/trunk/flume-ng-core/src/test/java/org/apache/flume/sink/SinkProcessorFactoryTest.java

                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.5.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (FLUME-1001) Support custom processors

Posted by "Juhani Connolly (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Juhani Connolly updated FLUME-1001:
-----------------------------------

    Attachment:     (was: FLUME-1001.3.patch)
    
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (FLUME-1001) Support custom processors

Posted by "Juhani Connolly (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Juhani Connolly updated FLUME-1001:
-----------------------------------

    Attachment:     (was: FLUME-1001.4.patch)
    
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (FLUME-1001) Support custom processors

Posted by "Arvind Prabhakar (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arvind Prabhakar updated FLUME-1001:
------------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

Patch committed. Thanks Juhani!
                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.5.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247065#comment-13247065 ] 

jiraposter@reviews.apache.org commented on FLUME-1001:
------------------------------------------------------



bq.  On 2012-04-05 03:18:45, Brock Noland wrote:
bq.  > looks good to me, just one nit. After you address to the nit please attache the patch to the JIRA.

Done and updated the issue.
This review is against the flume-728 so can't update the review file here, but it's just one row


- Juhani


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4152/#review6704
-----------------------------------------------------------


On 2012-03-05 03:25:20, Juhani Connolly wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4152/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-03-05 03:25:20)
bq.  
bq.  
bq.  Review request for Flume.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.
bq.  
bq.  
bq.  This addresses bug FLUME-1001.
bq.      https://issues.apache.org/jira/browse/FLUME-1001
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION 
bq.    flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b 
bq.    flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e 
bq.  
bq.  Diff: https://reviews.apache.org/r/4152/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment
bq.  
bq.  Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Juhani
bq.  
bq.


                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.3.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (FLUME-1001) Support custom processors

Posted by "Juhani Connolly (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Juhani Connolly updated FLUME-1001:
-----------------------------------

    Attachment: FLUME-1001.3.patch

Changed the warn mentioned in the review to info
                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.3.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222151#comment-13222151 ] 

jiraposter@reviews.apache.org commented on FLUME-1001:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4152/
-----------------------------------------------------------

(Updated 2012-03-05 03:25:20.490086)


Review request for Flume.


Changes
-------

Addressed the suggested changes


Summary
-------

Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.


This addresses bug FLUME-1001.
    https://issues.apache.org/jira/browse/FLUME-1001


Diffs (updated)
-----

  flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION 
  flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b 
  flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e 

Diff: https://reviews.apache.org/r/4152/diff


Testing
-------

Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works


Thanks,

Juhani


                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.1.0
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246977#comment-13246977 ] 

jiraposter@reviews.apache.org commented on FLUME-1001:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4152/#review6704
-----------------------------------------------------------

Ship it!


looks good to me, just one nit. After you address to the nit please attache the patch to the JIRA.


flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
<https://reviews.apache.org/r/4152/#comment14508>

    INFO is more appropriate, people freak out when they see WARN.


- Brock


On 2012-03-05 03:25:20, Juhani Connolly wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4152/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-03-05 03:25:20)
bq.  
bq.  
bq.  Review request for Flume.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.
bq.  
bq.  
bq.  This addresses bug FLUME-1001.
bq.      https://issues.apache.org/jira/browse/FLUME-1001
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION 
bq.    flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b 
bq.    flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e 
bq.  
bq.  Diff: https://reviews.apache.org/r/4152/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment
bq.  
bq.  Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Juhani
bq.  
bq.


                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.2.0
>
>         Attachments: FLUME-1001.2.patch
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220765#comment-13220765 ] 

jiraposter@reviews.apache.org commented on FLUME-1001:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4152/
-----------------------------------------------------------

Review request for Flume.


Summary
-------

Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.


This addresses bug FLUME-1001.
    https://issues.apache.org/jira/browse/FLUME-1001


Diffs
-----

  flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b 
  flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e 
  flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION 

Diff: https://reviews.apache.org/r/4152/diff


Testing
-------

Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works


Thanks,

Juhani


                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.1.0
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "Juhani Connolly (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220727#comment-13220727 ] 

Juhani Connolly commented on FLUME-1001:
----------------------------------------

Seeing as no-one else has expressed interest in this, I'll put it together, following the same style as was used for sinks and sources
                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>             Fix For: v1.1.0
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Assigned] (FLUME-1001) Support custom processors

Posted by "Juhani Connolly (Assigned) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Juhani Connolly reassigned FLUME-1001:
--------------------------------------

    Assignee: Juhani Connolly
    
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.1.0
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (FLUME-1001) Support custom processors

Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222146#comment-13222146 ] 

jiraposter@reviews.apache.org commented on FLUME-1001:
------------------------------------------------------



bq.  On 2012-03-02 16:47:32, Will McQueen wrote:
bq.  > flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java, line 40
bq.  > <https://reviews.apache.org/r/4152/diff/2/?file=87333#file87333line40>
bq.  >
bq.  >     For consistency with the other types (SinkType, SourceType, ...), can we please put OTHER at the top of the enum listing?
bq.  >     
bq.  >     Also for same consistency, can we have the comment read:
bq.  >     
bq.  >       /**
bq.  >        * Place holder for custom sink processors not part of this enumeration.
bq.  >        */

Makes sense, done


bq.  On 2012-03-02 16:47:32, Will McQueen wrote:
bq.  > flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java, line 40
bq.  > <https://reviews.apache.org/r/4152/diff/2/?file=87334#file87334line40>
bq.  >
bq.  >     Before this line:
bq.  >     Preconditions.checkNotNull(context);
bq.  >     Preconditions.checkNotNull(sinks);
bq.  >     logger.debug("Creating instance of sink processor name {}, type {}", name, type);

Added precondition checks, changed the debug message a bit because processors belong to a sink group and thus themselves are not named.


bq.  On 2012-03-02 16:47:32, Will McQueen wrote:
bq.  > flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java, line 48
bq.  > <https://reviews.apache.org/r/4152/diff/2/?file=87334#file87334line48>
bq.  >
bq.  >     => "Sink processor type {} is a custom type"
bq.  >     
bq.  >     (this string output matches the convention used by DefaultSourceFactory)

ok


bq.  On 2012-03-02 16:47:32, Will McQueen wrote:
bq.  > flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java, line 66
bq.  > <https://reviews.apache.org/r/4152/diff/2/?file=87334#file87334line66>
bq.  >
bq.  >     add "sink", remove comma:
bq.  >     
bq.  >     =>"Unable to create sink processor type: "
bq.  >

done


- Juhani


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4152/#review5557
-----------------------------------------------------------


On 2012-03-02 07:51:49, Juhani Connolly wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4152/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-03-02 07:51:49)
bq.  
bq.  
bq.  Review request for Flume.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.
bq.  
bq.  
bq.  This addresses bug FLUME-1001.
bq.      https://issues.apache.org/jira/browse/FLUME-1001
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b 
bq.    flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e 
bq.    flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/4152/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment
bq.  
bq.  Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Juhani
bq.  
bq.


                
> Support custom processors
> -------------------------
>
>                 Key: FLUME-1001
>                 URL: https://issues.apache.org/jira/browse/FLUME-1001
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.1.0
>         Environment: RHEL 6.2 64-bit
>            Reporter: Will McQueen
>            Assignee: Juhani Connolly
>             Fix For: v1.1.0
>
>
> Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira