You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Ashish Paliwal (JIRA)" <ji...@apache.org> on 2014/11/05 12:23:34 UTC
[jira] [Resolved] (FLUME-929) Exception "too many open files" in
ThriftEventSource (with infinite loop as result)
[ https://issues.apache.org/jira/browse/FLUME-929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashish Paliwal resolved FLUME-929.
----------------------------------
Resolution: Won't Fix
Fix Version/s: v0.9.5
Won't fix. 0.X branch not maintained anymore
> Exception "too many open files" in ThriftEventSource (with infinite loop as result)
> -----------------------------------------------------------------------------------
>
> Key: FLUME-929
> URL: https://issues.apache.org/jira/browse/FLUME-929
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Environment: Flume 0.9.4-cdh3u2 on Linux Squeeze, System with one master and two nodes
> Reporter: bert Passek
> Priority: Blocker
> Fix For: v0.9.5
>
>
> We use ThriftEventSource for logging huge amount of data. However, an exception is thrown randomly and not reproducable which results in an infinite loop in one of the worker threads. We had to stop all flume nodes because the log file was dramatically increasing due to logged exceptions.
> This is just a small part of the log file:
> 2012-01-11 14:57:30,487 ERROR com.cloudera.flume.core.connector.DirectDriver: Exiting driver logicalNode newsletterImpressionLog01-21 in error state ThriftEventSource | CassandraSink because sleep interrupted
> 2012-01-11 17:18:14,808 WARN org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred during acceptance of message.
> org.apache.thrift.transport.TTransportException: java.net.SocketException: Too many open files
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:139)
> at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
> at org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
> Caused by: java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
> at java.net.ServerSocket.accept(ServerSocket.java:430)
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:134)
> ... 2 more
> 2012-01-11 17:18:14,809 WARN org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred during acceptance of message.
> org.apache.thrift.transport.TTransportException: java.net.SocketException: Too many open files
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:139)
> at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
> at org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
> Caused by: java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
> at java.net.ServerSocket.accept(ServerSocket.java:430)
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:134)
> ... 2 more
> 2012-01-11 17:18:14,809 WARN org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred during acceptance of message.
> org.apache.thrift.transport.TTransportException: java.net.SocketException: Too many open files
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:139)
> at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
> at org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
> Caused by: java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
> at java.net.ServerSocket.accept(ServerSocket.java:430)
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:134)
> ... 2 more
> 2012-01-11 17:18:14,809 WARN org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred during acceptance of message.
> org.apache.thrift.transport.TTransportException: java.net.SocketException: Too many open files
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:139)
> at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
> at org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
> Caused by: java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
> at java.net.ServerSocket.accept(ServerSocket.java:430)
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:134)
> ... 2 more
> 2012-01-11 17:18:14,810 WARN org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred during acceptance of message.
> org.apache.thrift.transport.TTransportException: java.net.SocketException: Too many open files
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:139)
> at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
> at org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
> Caused by: java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
> at java.net.ServerSocket.accept(ServerSocket.java:430)
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:134)
> ... 2 more
> 2012-01-11 17:18:14,810 WARN org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred during acceptance of message.
> org.apache.thrift.transport.TTransportException: java.net.SocketException: Too many open files
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:139)
> at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
> at org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
> Caused by: java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
> at java.net.ServerSocket.accept(ServerSocket.java:430)
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:134)
> ... 2 more
> 2012-01-11 17:18:14,810 WARN org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred during acceptance of message.
> org.apache.thrift.transport.TTransportException: java.net.SocketException: Too many open files
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:139)
> at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
> at org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
> Caused by: java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
> at java.net.ServerSocket.accept(ServerSocket.java:430)
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:134)
> ... 2 more
> So there are actually 2 issues to get fixed:
> 1. Why are there "too many open files"?
> 2. The worker thread must shutdown correctly and not ending in an infinite loop.
> This is a blocker issue for us because with this kind of error we are not able to use flume furthermore.
> Best Regards
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)