You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by wei yang <wy...@gmail.com> on 2015/05/14 10:19:48 UTC

Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/
-----------------------------------------------------------

Review request for Flume.


Bugs: FLUME-2689
    https://issues.apache.org/jira/browse/FLUME-2689


Repository: flume-git


Description
-------

Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.


Diffs
-----

  flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java e84e4b6 

Diff: https://reviews.apache.org/r/34200/diff/


Testing
-------


Thanks,

wei yang


Re: Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

Posted by wei yang <wy...@gmail.com>.

> On \u4e03\u6708 20, 2016, 1:59 p.m., Bal�zs Don�t Bessenyei wrote:
> > flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java, line 39
> > <https://reviews.apache.org/r/34200/diff/1/?file=958822#file958822line39>
> >
> >     Can you please expand this import?

Thanks for the review. I have updated patch.


- wei


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/#review142936
-----------------------------------------------------------


On \u4e03\u6708 26, 2016, 8:30 a.m., wei yang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34200/
> -----------------------------------------------------------
> 
> (Updated \u4e03\u6708 26, 2016, 8:30 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Bugs: FLUME-2689
>     https://issues.apache.org/jira/browse/FLUME-2689
> 
> 
> Repository: flume-git
> 
> 
> Description
> -------
> 
> Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
> The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.
> 
> 
> Diffs
> -----
> 
>   flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java e84e4b6 
>   flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java 22fa200 
> 
> Diff: https://reviews.apache.org/r/34200/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> wei yang
> 
>


Re: Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

Posted by Balázs Donát Bessenyei <be...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/#review142936
-----------------------------------------------------------




flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java (line 23)
<https://reviews.apache.org/r/34200/#comment208606>

    Can you please expand this import?



flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java (line 30)
<https://reviews.apache.org/r/34200/#comment208607>

    Can you please expand this import?


- Bal�zs Don�t Bessenyei


On May 14, 2015, 8:19 a.m., wei yang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34200/
> -----------------------------------------------------------
> 
> (Updated May 14, 2015, 8:19 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Bugs: FLUME-2689
>     https://issues.apache.org/jira/browse/FLUME-2689
> 
> 
> Repository: flume-git
> 
> 
> Description
> -------
> 
> Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
> The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.
> 
> 
> Diffs
> -----
> 
>   flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java e84e4b6 
> 
> Diff: https://reviews.apache.org/r/34200/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> wei yang
> 
>


Re: Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

Posted by Balázs Donát Bessenyei <be...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/#review143547
-----------------------------------------------------------




flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java (lines 130 - 133)
<https://reviews.apache.org/r/34200/#comment209387>

    Is there any way we could eliminate these Thread.sleep()-s?


- Bal�zs Don�t Bessenyei


On July 26, 2016, 8:30 a.m., wei yang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34200/
> -----------------------------------------------------------
> 
> (Updated July 26, 2016, 8:30 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Bugs: FLUME-2689
>     https://issues.apache.org/jira/browse/FLUME-2689
> 
> 
> Repository: flume-git
> 
> 
> Description
> -------
> 
> Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
> The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.
> 
> 
> Diffs
> -----
> 
>   flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java e84e4b6 
>   flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java 22fa200 
> 
> Diff: https://reviews.apache.org/r/34200/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> wei yang
> 
>


Re: Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

Posted by Balázs Donát Bessenyei <be...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/#review144520
-----------------------------------------------------------




flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java (lines 136 - 140)
<https://reviews.apache.org/r/34200/#comment210528>

    Can we maybe use an approach like this: https://github.com/apache/flume/blob/c8c0f9b8404ef60222f49aa91f74f5a846b57ca7/flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java#L220 to avoid having that long pause?


- Bal�zs Don�t Bessenyei


On July 27, 2016, 2:19 a.m., wei yang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34200/
> -----------------------------------------------------------
> 
> (Updated July 27, 2016, 2:19 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Bugs: FLUME-2689
>     https://issues.apache.org/jira/browse/FLUME-2689
> 
> 
> Repository: flume-git
> 
> 
> Description
> -------
> 
> Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
> The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.
> 
> 
> Diffs
> -----
> 
>   flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java 185c00c 
>   flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java 10ef8d8 
> 
> Diff: https://reviews.apache.org/r/34200/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> wei yang
> 
>


Re: Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

Posted by wei yang <wy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/
-----------------------------------------------------------

(Updated \u516b\u6708 4, 2016, 4:56 a.m.)


Review request for Flume.


Changes
-------

Use approach suggested by Bal�zs Don�t Bessenyei to avoid long pause.


Bugs: FLUME-2689
    https://issues.apache.org/jira/browse/FLUME-2689


Repository: flume-git


Description
-------

Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.


Diffs (updated)
-----

  flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java 185c00c 
  flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java 10ef8d8 

Diff: https://reviews.apache.org/r/34200/diff/


Testing
-------


Thanks,

wei yang


Re: Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

Posted by wei yang <wy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/
-----------------------------------------------------------

(Updated \u4e03\u6708 27, 2016, 2:19 a.m.)


Review request for Flume.


Changes
-------

patch based on trunk


Bugs: FLUME-2689
    https://issues.apache.org/jira/browse/FLUME-2689


Repository: flume-git


Description
-------

Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.


Diffs (updated)
-----

  flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java 185c00c 
  flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java 10ef8d8 

Diff: https://reviews.apache.org/r/34200/diff/


Testing
-------


Thanks,

wei yang


Re: Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

Posted by Balázs Donát Bessenyei <be...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/#review143546
-----------------------------------------------------------



wei yang, can you please rebase this patch on trunk?

also, the test seems to fail when I'm running it

- Bal�zs Don�t Bessenyei


On July 26, 2016, 8:30 a.m., wei yang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34200/
> -----------------------------------------------------------
> 
> (Updated July 26, 2016, 8:30 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Bugs: FLUME-2689
>     https://issues.apache.org/jira/browse/FLUME-2689
> 
> 
> Repository: flume-git
> 
> 
> Description
> -------
> 
> Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
> The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.
> 
> 
> Diffs
> -----
> 
>   flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java e84e4b6 
>   flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java 22fa200 
> 
> Diff: https://reviews.apache.org/r/34200/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> wei yang
> 
>


Re: Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

Posted by Mike Percy <mp...@apache.org>.

> On July 26, 2016, 1:48 a.m., Mike Percy wrote:
> > Thanks for updating the patch, tinawenqiao. I will try to let Donat take another look if he has time before I do a final review pass.
> > 
> > Regards, Mike

Oops, I am sorry, I mean Wei Yang. I mixed up this patch with another patch currently under review. Thank you.


- Mike


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/#review143498
-----------------------------------------------------------


On July 26, 2016, 1:30 a.m., wei yang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34200/
> -----------------------------------------------------------
> 
> (Updated July 26, 2016, 1:30 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Bugs: FLUME-2689
>     https://issues.apache.org/jira/browse/FLUME-2689
> 
> 
> Repository: flume-git
> 
> 
> Description
> -------
> 
> Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
> The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.
> 
> 
> Diffs
> -----
> 
>   flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java e84e4b6 
>   flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java 22fa200 
> 
> Diff: https://reviews.apache.org/r/34200/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> wei yang
> 
>


Re: Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

Posted by Mike Percy <mp...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/#review143498
-----------------------------------------------------------



Thanks for updating the patch, tinawenqiao. I will try to let Donat take another look if he has time before I do a final review pass.

Regards, Mike

- Mike Percy


On July 26, 2016, 1:30 a.m., wei yang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34200/
> -----------------------------------------------------------
> 
> (Updated July 26, 2016, 1:30 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Bugs: FLUME-2689
>     https://issues.apache.org/jira/browse/FLUME-2689
> 
> 
> Repository: flume-git
> 
> 
> Description
> -------
> 
> Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
> The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.
> 
> 
> Diffs
> -----
> 
>   flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java e84e4b6 
>   flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java 22fa200 
> 
> Diff: https://reviews.apache.org/r/34200/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> wei yang
> 
>


Re: Review Request 34200: Review request for FLUME-2689, reloading conf file leads syslogTcpSource not receives any event

Posted by wei yang <wy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34200/
-----------------------------------------------------------

(Updated \u4e03\u6708 26, 2016, 8:30 a.m.)


Review request for Flume.


Changes
-------

Expanded this import and added unit test.


Bugs: FLUME-2689
    https://issues.apache.org/jira/browse/FLUME-2689


Repository: flume-git


Description
-------

Reloading conf file will stop old syslog source and start new syslog source. Stopping syslog tcp source only closes the NioServerSocketChannel, resulting in the client sends data through the old channel. In that case, the new source never receives data. The tcpdump shows the events have received but the new source doesn't and ss shows the client connection stays same with old one.
The right way to stop syslog source is close both the NioSocketChannel and NioServerSocketChannel, and shutdown the executor.


Diffs (updated)
-----

  flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java e84e4b6 
  flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java 22fa200 

Diff: https://reviews.apache.org/r/34200/diff/


Testing
-------


Thanks,

wei yang