You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Mingjie Lai <mj...@gmail.com> on 2011/10/13 09:51:02 UTC

Review Request: Flume-209: UDP source

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2380/
-----------------------------------------------------------

Review request for Flume.


Summary
-------

It would be nice to have something listen on a specified udp port and convert each packet them into flume events. This could be useful for "sensors" or other simple devices on the network.


This addresses bug Flume-209.
    https://issues.apache.org/jira/browse/Flume-209


Diffs
-----

  flume-core/src/main/java/com/cloudera/flume/conf/SourceFactoryImpl.java ed7b111 
  flume-core/src/main/java/com/cloudera/flume/handlers/socket/UdpSource.java PRE-CREATION 
  flume-core/src/test/java/com/cloudera/flume/handlers/socket/TestUdpSource.java PRE-CREATION 

Diff: https://reviews.apache.org/r/2380/diff


Testing
-------

TestUdp is added and passed locally.


Thanks,

Mingjie


Re: Review Request: Flume-209: UDP source

Posted by jm...@apache.org.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2380/#review2731
-----------------------------------------------------------



flume-core/src/test/java/com/cloudera/flume/handlers/socket/TestUdpSource.java
<https://reviews.apache.org/r/2380/#comment6147>

    mvn test -Dtest=TestUdpSource
    
    This line seems to fail due to timeout.  On a good run it seems to finish in 5-10s but on a bad run it fails.  
    
    My guess is that there is udp loss due to a full udp buffer. Maybe it has the delay in the sink side is the problem? (why not process right away, and put delays on the send side?)
    
    It would great to figure out why.  
    
    
    
    
    


- jmhsieh


On 2011-10-13 18:07:50, Mingjie Lai wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2380/
> -----------------------------------------------------------
> 
> (Updated 2011-10-13 18:07:50)
> 
> 
> Review request for Flume.
> 
> 
> Summary
> -------
> 
> It would be nice to have something listen on a specified udp port and convert each packet them into flume events. This could be useful for "sensors" or other simple devices on the network.
> 
> 
> This addresses bug Flume-209.
>     https://issues.apache.org/jira/browse/Flume-209
> 
> 
> Diffs
> -----
> 
>   flume-core/src/main/java/com/cloudera/flume/conf/SourceFactoryImpl.java ed7b111 
>   flume-core/src/main/java/com/cloudera/flume/handlers/socket/UdpSource.java PRE-CREATION 
>   flume-core/src/test/java/com/cloudera/flume/handlers/socket/TestUdpSource.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/2380/diff
> 
> 
> Testing
> -------
> 
> TestUdp is added and passed locally.
> 
> 
> Thanks,
> 
> Mingjie
> 
>


Re: Review Request: Flume-209: UDP source

Posted by jm...@apache.org.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2380/#review2649
-----------------------------------------------------------

Ship it!


lgtm.

- jmhsieh


On 2011-10-13 18:07:50, Mingjie Lai wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2380/
> -----------------------------------------------------------
> 
> (Updated 2011-10-13 18:07:50)
> 
> 
> Review request for Flume.
> 
> 
> Summary
> -------
> 
> It would be nice to have something listen on a specified udp port and convert each packet them into flume events. This could be useful for "sensors" or other simple devices on the network.
> 
> 
> This addresses bug Flume-209.
>     https://issues.apache.org/jira/browse/Flume-209
> 
> 
> Diffs
> -----
> 
>   flume-core/src/main/java/com/cloudera/flume/conf/SourceFactoryImpl.java ed7b111 
>   flume-core/src/main/java/com/cloudera/flume/handlers/socket/UdpSource.java PRE-CREATION 
>   flume-core/src/test/java/com/cloudera/flume/handlers/socket/TestUdpSource.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/2380/diff
> 
> 
> Testing
> -------
> 
> TestUdp is added and passed locally.
> 
> 
> Thanks,
> 
> Mingjie
> 
>


Re: Review Request: Flume-209: UDP source

Posted by Mingjie Lai <mj...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2380/
-----------------------------------------------------------

(Updated 2011-10-13 18:07:50.050517)


Review request for Flume.


Changes
-------

Updated to fix the issues raised by Jon's reviews. Thanks, Jon. 


Summary
-------

It would be nice to have something listen on a specified udp port and convert each packet them into flume events. This could be useful for "sensors" or other simple devices on the network.


This addresses bug Flume-209.
    https://issues.apache.org/jira/browse/Flume-209


Diffs (updated)
-----

  flume-core/src/main/java/com/cloudera/flume/conf/SourceFactoryImpl.java ed7b111 
  flume-core/src/main/java/com/cloudera/flume/handlers/socket/UdpSource.java PRE-CREATION 
  flume-core/src/test/java/com/cloudera/flume/handlers/socket/TestUdpSource.java PRE-CREATION 

Diff: https://reviews.apache.org/r/2380/diff


Testing
-------

TestUdp is added and passed locally.


Thanks,

Mingjie


Re: Review Request: Flume-209: UDP source

Posted by jm...@apache.org.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2380/#review2553
-----------------------------------------------------------


Mingjie, 

Look pretty good.  

I noticed a few nits that should be addressed.


flume-core/src/main/java/com/cloudera/flume/handlers/socket/UdpSource.java
<https://reviews.apache.org/r/2380/#comment5729>

    change to apache
    
    /*
     * Licensed to the Apache Software Foundation (ASF) under one
     * or more contributor license agreements.  See the NOTICE file
     * distributed with this work for additional information
     * regarding copyright ownership.  The ASF licenses this file
     * to you under the Apache License, Version 2.0 (the
     * "License"); you may not use this file except in compliance
     * with the License.  You may obtain a copy of the License at
     *
     *     http://www.apache.org/licenses/LICENSE-2.0
     *
     * Unless required by applicable law or agreed to in writing, software
     * distributed under the License is distributed on an "AS IS" BASIS,
     * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     * See the License for the specific language governing permissions and
     * limitations under the License.
     */



flume-core/src/main/java/com/cloudera/flume/handlers/socket/UdpSource.java
<https://reviews.apache.org/r/2380/#comment5726>

    needs ')'



flume-core/src/test/java/com/cloudera/flume/handlers/socket/TestUdpSource.java
<https://reviews.apache.org/r/2380/#comment5730>

    change to apache
    



flume-core/src/test/java/com/cloudera/flume/handlers/socket/TestUdpSource.java
<https://reviews.apache.org/r/2380/#comment5727>

    final?
    



flume-core/src/test/java/com/cloudera/flume/handlers/socket/TestUdpSource.java
<https://reviews.apache.org/r/2380/#comment5728>

    Maybe put in finally so that we don't have to hang 30s if this fails or funny exceptions thrown?


- jmhsieh


On 2011-10-13 07:51:02, Mingjie Lai wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2380/
> -----------------------------------------------------------
> 
> (Updated 2011-10-13 07:51:02)
> 
> 
> Review request for Flume.
> 
> 
> Summary
> -------
> 
> It would be nice to have something listen on a specified udp port and convert each packet them into flume events. This could be useful for "sensors" or other simple devices on the network.
> 
> 
> This addresses bug Flume-209.
>     https://issues.apache.org/jira/browse/Flume-209
> 
> 
> Diffs
> -----
> 
>   flume-core/src/main/java/com/cloudera/flume/conf/SourceFactoryImpl.java ed7b111 
>   flume-core/src/main/java/com/cloudera/flume/handlers/socket/UdpSource.java PRE-CREATION 
>   flume-core/src/test/java/com/cloudera/flume/handlers/socket/TestUdpSource.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/2380/diff
> 
> 
> Testing
> -------
> 
> TestUdp is added and passed locally.
> 
> 
> Thanks,
> 
> Mingjie
> 
>