You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Ossi L (Created) (JIRA)" <ji...@apache.org> on 2011/12/08 14:48:40 UTC
[jira] [Created] (FLUME-874) Thirft server doesn't check if port is
free
Thirft server doesn't check if port is free
-------------------------------------------
Key: FLUME-874
URL: https://issues.apache.org/jira/browse/FLUME-874
Project: Flume
Issue Type: Improvement
Components: Sinks+Sources
Environment: RHEL 5
Reporter: Ossi L
When starting a collector with multiple flows / logical nodes configured, there might be conflict with overlapping ports.
2011-12-08 13:12:00,470 INFO org.mortbay.log: Extract /usr/lib/flume/webapps/flumeagent.war to /tmp/Jetty_0_0_0_0_35862_flumeagent.war__flumeagent__4cgrz3/webapp
2011-12-08 13:12:00,680 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:35862
2011-12-08 13:12:00,680 INFO com.cloudera.util.InternalHttpServer: Server started on port 35862
2011-12-08 13:12:05,785 INFO com.cloudera.flume.agent.LogicalNodeManager: Loading node name with FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011' snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011' flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink( "hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-" )' }
2011-12-08 13:12:05,787 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011' snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011' flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink( "hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-" )' }
2011-12-08 13:12:05,798 INFO com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking thread pool server on port 35862...
org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address 0.0.0.0/0.0.0.0:35862.
So, first flume start jetty on port 35862 and then it tries to start thrift server on the same port and it naturally fails.
Same problem will be with master if that server is running multiple logical nodes / flows.
And of course with any other service which happens to be using ports near above range.
This can be avoided by configuring http port of nodes and master to be than 35853, but it doesn't fix problem with some other services.
So, thrift should check if the port is free or not and bind only after that, if port is free.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-874) Thirft server doesn't check if port is
free
Posted by "Arvind Prabhakar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar updated FLUME-874:
-----------------------------------
Affects Version/s: v0.9.4
> Thirft server doesn't check if port is free
> -------------------------------------------
>
> Key: FLUME-874
> URL: https://issues.apache.org/jira/browse/FLUME-874
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Environment: RHEL 5
> Reporter: Ossi L
>
> When starting a collector with multiple flows / logical nodes configured, there might be conflict with overlapping ports.
> 2011-12-08 13:12:00,470 INFO org.mortbay.log: Extract /usr/lib/flume/webapps/flumeagent.war to /tmp/Jetty_0_0_0_0_35862_flumeagent.war__flumeagent__4cgrz3/webapp
> 2011-12-08 13:12:00,680 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:35862
> 2011-12-08 13:12:00,680 INFO com.cloudera.util.InternalHttpServer: Server started on port 35862
> 2011-12-08 13:12:05,785 INFO com.cloudera.flume.agent.LogicalNodeManager: Loading node name with FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011' snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011' flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink( "hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-" )' }
> 2011-12-08 13:12:05,787 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011' snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011' flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink( "hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-" )' }
> 2011-12-08 13:12:05,798 INFO com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking thread pool server on port 35862...
> org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address 0.0.0.0/0.0.0.0:35862.
> So, first flume start jetty on port 35862 and then it tries to start thrift server on the same port and it naturally fails.
> Same problem will be with master if that server is running multiple logical nodes / flows.
> And of course with any other service which happens to be using ports near above range.
> This can be avoided by configuring http port of nodes and master to be than 35853, but it doesn't fix problem with some other services.
> So, thrift should check if the port is free or not and bind only after that, if port is free.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-874) Thirft server doesn't check if port is
free
Posted by "Ossi L (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ossi L updated FLUME-874:
-------------------------
Issue Type: Bug (was: Improvement)
> Thirft server doesn't check if port is free
> -------------------------------------------
>
> Key: FLUME-874
> URL: https://issues.apache.org/jira/browse/FLUME-874
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Environment: RHEL 5
> Reporter: Ossi L
>
> When starting a collector with multiple flows / logical nodes configured, there might be conflict with overlapping ports.
> 2011-12-08 13:12:00,470 INFO org.mortbay.log: Extract /usr/lib/flume/webapps/flumeagent.war to /tmp/Jetty_0_0_0_0_35862_flumeagent.war__flumeagent__4cgrz3/webapp
> 2011-12-08 13:12:00,680 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:35862
> 2011-12-08 13:12:00,680 INFO com.cloudera.util.InternalHttpServer: Server started on port 35862
> 2011-12-08 13:12:05,785 INFO com.cloudera.flume.agent.LogicalNodeManager: Loading node name with FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011' snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011' flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink( "hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-" )' }
> 2011-12-08 13:12:05,787 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011' snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011' flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink( "hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-" )' }
> 2011-12-08 13:12:05,798 INFO com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking thread pool server on port 35862...
> org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address 0.0.0.0/0.0.0.0:35862.
> So, first flume start jetty on port 35862 and then it tries to start thrift server on the same port and it naturally fails.
> Same problem will be with master if that server is running multiple logical nodes / flows.
> And of course with any other service which happens to be using ports near above range.
> This can be avoided by configuring http port of nodes and master to be than 35853, but it doesn't fix problem with some other services.
> So, thrift should check if the port is free or not and bind only after that, if port is free.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (FLUME-874) Thirft server doesn't check if port
is free
Posted by "Alexander Lorenz-Alten (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Lorenz-Alten reopened FLUME-874:
------------------------------------------
> Thirft server doesn't check if port is free
> -------------------------------------------
>
> Key: FLUME-874
> URL: https://issues.apache.org/jira/browse/FLUME-874
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Environment: RHEL 5
> Reporter: Ossi L
>
> When starting a collector with multiple flows / logical nodes configured, there might be conflict with overlapping ports.
> 2011-12-08 13:12:00,470 INFO org.mortbay.log: Extract /usr/lib/flume/webapps/flumeagent.war to /tmp/Jetty_0_0_0_0_35862_flumeagent.war__flumeagent__4cgrz3/webapp
> 2011-12-08 13:12:00,680 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:35862
> 2011-12-08 13:12:00,680 INFO com.cloudera.util.InternalHttpServer: Server started on port 35862
> 2011-12-08 13:12:05,785 INFO com.cloudera.flume.agent.LogicalNodeManager: Loading node name with FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011' snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011' flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink( "hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-" )' }
> 2011-12-08 13:12:05,787 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011' snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011' flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink( "hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-" )' }
> 2011-12-08 13:12:05,798 INFO com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking thread pool server on port 35862...
> org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address 0.0.0.0/0.0.0.0:35862.
> So, first flume start jetty on port 35862 and then it tries to start thrift server on the same port and it naturally fails.
> Same problem will be with master if that server is running multiple logical nodes / flows.
> And of course with any other service which happens to be using ports near above range.
> This can be avoided by configuring http port of nodes and master to be than 35853, but it doesn't fix problem with some other services.
> So, thrift should check if the port is free or not and bind only after that, if port is free.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (FLUME-874) Thirft server doesn't check if port
is free
Posted by "Alexander Lorenz-Alten (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Lorenz-Alten resolved FLUME-874.
------------------------------------------
Resolution: Won't Fix
> Thirft server doesn't check if port is free
> -------------------------------------------
>
> Key: FLUME-874
> URL: https://issues.apache.org/jira/browse/FLUME-874
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Environment: RHEL 5
> Reporter: Ossi L
>
> When starting a collector with multiple flows / logical nodes configured, there might be conflict with overlapping ports.
> 2011-12-08 13:12:00,470 INFO org.mortbay.log: Extract /usr/lib/flume/webapps/flumeagent.war to /tmp/Jetty_0_0_0_0_35862_flumeagent.war__flumeagent__4cgrz3/webapp
> 2011-12-08 13:12:00,680 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:35862
> 2011-12-08 13:12:00,680 INFO com.cloudera.util.InternalHttpServer: Server started on port 35862
> 2011-12-08 13:12:05,785 INFO com.cloudera.flume.agent.LogicalNodeManager: Loading node name with FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011' snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011' flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink( "hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-" )' }
> 2011-12-08 13:12:05,787 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Thu Dec 08 12:55:34 CET 2011' snkVer:'Thu Dec 08 12:55:34 CET 2011' ts='Thu Dec 08 12:55:34 CET 2011' flowId:'dk-flow-https-fe' source:'rpcSource( 35862 )' sink:'collectorSink( "hdfs://hq-priv-01:8020/flume/fusedk/httpd-fe/%Y-%m-%d/", "%{host}-ssl-access-" )' }
> 2011-12-08 13:12:05,798 INFO com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking thread pool server on port 35862...
> org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address 0.0.0.0/0.0.0.0:35862.
> So, first flume start jetty on port 35862 and then it tries to start thrift server on the same port and it naturally fails.
> Same problem will be with master if that server is running multiple logical nodes / flows.
> And of course with any other service which happens to be using ports near above range.
> This can be avoided by configuring http port of nodes and master to be than 35853, but it doesn't fix problem with some other services.
> So, thrift should check if the port is free or not and bind only after that, if port is free.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira