You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Bharath Ravi <bh...@gmail.com> on 2012/04/05 03:52:40 UTC
Disable flume logging?
Hi all,
The logs generated by flume (especially the master) run to a few GBs in a
few minutes
on my setup.
How do I disable logging on flume? I did some brief looking around the user
guide and related
docs, but couldn't find a knob that turns it off.
Thanks a lot!
--
Bharath Ravi
Re: Disable flume logging?
Posted by Brock Noland <br...@cloudera.com>.
Sorry, I see you referenced the master. Disregard my reply.
On Wed, Apr 4, 2012 at 9:59 PM, Brock Noland <br...@cloudera.com> wrote:
> NG or OG? With at least NG you should be able to change DEBUG to INFO
> (or even higher) in the log4j.properties file.
>
> Brock
>
> On Wed, Apr 4, 2012 at 9:52 PM, Bharath Ravi <bh...@gmail.com> wrote:
>> Hi all,
>>
>> The logs generated by flume (especially the master) run to a few GBs in a
>> few minutes
>> on my setup.
>>
>> How do I disable logging on flume? I did some brief looking around the user
>> guide and related
>> docs, but couldn't find a knob that turns it off.
>>
>> Thanks a lot!
>> --
>> Bharath Ravi
>>
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: Disable flume logging?
Posted by Brock Noland <br...@cloudera.com>.
NG or OG? With at least NG you should be able to change DEBUG to INFO
(or even higher) in the log4j.properties file.
Brock
On Wed, Apr 4, 2012 at 9:52 PM, Bharath Ravi <bh...@gmail.com> wrote:
> Hi all,
>
> The logs generated by flume (especially the master) run to a few GBs in a
> few minutes
> on my setup.
>
> How do I disable logging on flume? I did some brief looking around the user
> guide and related
> docs, but couldn't find a knob that turns it off.
>
> Thanks a lot!
> --
> Bharath Ravi
>
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: Disable flume logging?
Posted by Bharath Ravi <bh...@gmail.com>.
Ah, I'm sorry I wasn't clear. I should have mentioned that the 16 nodes are
distributed on 16 different machines.
The master, which hits these limits is on one of these 16 machines.
That said, I have increased the limits on the master machine, and haven't
run into problems since.
Thanks a lot!
--
Bharath Ravi
On 6 April 2012 15:40, alo alt <wg...@googlemail.com> wrote:
> Yes, it is much to low. For 16 nodes use a number around 16k
> That means the open file limit for the whole system! 1024 are very very
> low.
>
>
> - Alex
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Apr 6, 2012, at 9:27 PM, Bharath Ravi wrote:
>
> > I currently have an open file limit of 1024 as root. Forgive my
> ignorance, but
> > it was a surprise to me that the master could actually hit this limit
> while configuring 16 nodes
> > (there were a few other processes running but not too many files open).
> >
> > Further I don't always hit this limit: it only happens occasionally.
> > Is this a problem that flume users have experienced before? Is 1024
> typically too low a number?
> >
> > Thanks again!
> > --
> > Bharath Ravi
> >
> >
> >
> > On 6 April 2012 03:20, Alexander Lorenz <wg...@googlemail.com>
> wrote:
> > to many open files means that your OS is blocking new threads. Simply
> tune /etc/security/limits.conf and tweak the
> > soft and hard limits to fit your installation.
> >
> > -Alex
> >
> > sent via my mobile device
> >
> > On Apr 6, 2012, at 5:24 AM, Bharath Ravi <bh...@gmail.com> wrote:
> >
> >> Thanks Alex! My log4j does seem to be on a verbose configuration.
> >>
> >> Now the problem my log is growing is because it's flooded with the
> following exception:
> >>
> >> 2012-04-05 22:45:00,381 WARN
> org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred
> during acceptance of message.
> >> org.apache.thrift.transport.TTransportException:
> java.net.SocketException: Too many open files
> >> at
> org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:137)
> >> at
> org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
> >> at
> org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
> >> Caused by: java.net.SocketException: Too many open files
> >> at java.net.PlainSocketImpl.socketAccept(Native Method)
> >> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
> >> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
> >> at java.net.ServerSocket.accept(ServerSocket.java:430)
> >> at
> org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:132)
> >> ... 2 more
> >>
> >> This happens when I attempt to configure around 16 nodes through the
> master through async "submit" commands
> >> to the flume shell. I'm not sure why the master would have too many
> sockets open for such a small number of nodes:
> >> any pointers to this?
> >>
> >> Thanks again!
> >>
> >> --
> >> Bharath Ravi
> >>
> >>
> >>
> >> On 5 April 2012 02:44, alo alt <wg...@googlemail.com> wrote:
> >> Hi,
> >>
> >> take a look into flume's conf directory, here you should find alog4j
> config. Could be that you here have a verbose logging configured.
> >> Before you delete the logs, what let them blow up? That amount of logs
> are really unusual.
> >>
> >> - Alex
> >>
> >> --
> >> Alexander Lorenz
> >> http://mapredit.blogspot.com
> >>
> >> On Apr 5, 2012, at 3:52 AM, Bharath Ravi wrote:
> >>
> >> > Hi all,
> >> >
> >> > The logs generated by flume (especially the master) run to a few GBs
> in a few minutes
> >> > on my setup.
> >> >
> >> > How do I disable logging on flume? I did some brief looking around
> the user guide and related
> >> > docs, but couldn't find a knob that turns it off.
> >> >
> >> > Thanks a lot!
> >> > --
> >> > Bharath Ravi
> >> >
> >>
> >>
> >
>
>
Re: Disable flume logging?
Posted by alo alt <wg...@googlemail.com>.
Yes, it is much to low. For 16 nodes use a number around 16k
That means the open file limit for the whole system! 1024 are very very low.
- Alex
--
Alexander Lorenz
http://mapredit.blogspot.com
On Apr 6, 2012, at 9:27 PM, Bharath Ravi wrote:
> I currently have an open file limit of 1024 as root. Forgive my ignorance, but
> it was a surprise to me that the master could actually hit this limit while configuring 16 nodes
> (there were a few other processes running but not too many files open).
>
> Further I don't always hit this limit: it only happens occasionally.
> Is this a problem that flume users have experienced before? Is 1024 typically too low a number?
>
> Thanks again!
> --
> Bharath Ravi
>
>
>
> On 6 April 2012 03:20, Alexander Lorenz <wg...@googlemail.com> wrote:
> to many open files means that your OS is blocking new threads. Simply tune /etc/security/limits.conf and tweak the
> soft and hard limits to fit your installation.
>
> -Alex
>
> sent via my mobile device
>
> On Apr 6, 2012, at 5:24 AM, Bharath Ravi <bh...@gmail.com> wrote:
>
>> Thanks Alex! My log4j does seem to be on a verbose configuration.
>>
>> Now the problem my log is growing is because it's flooded with the following exception:
>>
>> 2012-04-05 22:45:00,381 WARN org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred during acceptance of message.
>> org.apache.thrift.transport.TTransportException: java.net.SocketException: Too many open files
>> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:137)
>> at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
>> at org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
>> Caused by: java.net.SocketException: Too many open files
>> at java.net.PlainSocketImpl.socketAccept(Native Method)
>> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
>> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
>> at java.net.ServerSocket.accept(ServerSocket.java:430)
>> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:132)
>> ... 2 more
>>
>> This happens when I attempt to configure around 16 nodes through the master through async "submit" commands
>> to the flume shell. I'm not sure why the master would have too many sockets open for such a small number of nodes:
>> any pointers to this?
>>
>> Thanks again!
>>
>> --
>> Bharath Ravi
>>
>>
>>
>> On 5 April 2012 02:44, alo alt <wg...@googlemail.com> wrote:
>> Hi,
>>
>> take a look into flume's conf directory, here you should find alog4j config. Could be that you here have a verbose logging configured.
>> Before you delete the logs, what let them blow up? That amount of logs are really unusual.
>>
>> - Alex
>>
>> --
>> Alexander Lorenz
>> http://mapredit.blogspot.com
>>
>> On Apr 5, 2012, at 3:52 AM, Bharath Ravi wrote:
>>
>> > Hi all,
>> >
>> > The logs generated by flume (especially the master) run to a few GBs in a few minutes
>> > on my setup.
>> >
>> > How do I disable logging on flume? I did some brief looking around the user guide and related
>> > docs, but couldn't find a knob that turns it off.
>> >
>> > Thanks a lot!
>> > --
>> > Bharath Ravi
>> >
>>
>>
>
Re: Disable flume logging?
Posted by Bharath Ravi <bh...@gmail.com>.
I currently have an open file limit of 1024 as root. Forgive my ignorance,
but
it was a surprise to me that the master could actually hit this limit while
configuring 16 nodes
(there were a few other processes running but not too many files open).
Further I don't always hit this limit: it only happens occasionally.
Is this a problem that flume users have experienced before? Is 1024
typically too low a number?
Thanks again!
--
Bharath Ravi
On 6 April 2012 03:20, Alexander Lorenz <wg...@googlemail.com> wrote:
> to many open files means that your OS is blocking new threads. Simply tune
> /etc/security/limits.conf and tweak the
> soft and hard limits to fit your installation.
>
> -Alex
>
> sent via my mobile device
>
> On Apr 6, 2012, at 5:24 AM, Bharath Ravi <bh...@gmail.com> wrote:
>
> Thanks Alex! My log4j does seem to be on a verbose configuration.
>
> Now the problem my log is growing is because it's flooded with the
> following exception:
>
> 2012-04-05 22:45:00,381 WARN
> org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred
> during acceptance of message.
> org.apache.thrift.transport.TTransportException: java.net.SocketException:
> Too many open files
> at
> org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:137)
> at
> org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
> at
> org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
> Caused by: java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
> at java.net.ServerSocket.accept(ServerSocket.java:430)
> at
> org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:132)
> ... 2 more
>
> This happens when I attempt to configure around 16 nodes through the
> master through async "submit" commands
> to the flume shell. I'm not sure why the master would have too many
> sockets open for such a small number of nodes:
> any pointers to this?
>
> Thanks again!
>
> --
> Bharath Ravi
>
>
>
> On 5 April 2012 02:44, alo alt <wg...@googlemail.com> wrote:
>
>> Hi,
>>
>> take a look into flume's conf directory, here you should find alog4j
>> config. Could be that you here have a verbose logging configured.
>> Before you delete the logs, what let them blow up? That amount of logs
>> are really unusual.
>>
>> - Alex
>>
>> --
>> Alexander Lorenz
>> http://mapredit.blogspot.com
>>
>> On Apr 5, 2012, at 3:52 AM, Bharath Ravi wrote:
>>
>> > Hi all,
>> >
>> > The logs generated by flume (especially the master) run to a few GBs in
>> a few minutes
>> > on my setup.
>> >
>> > How do I disable logging on flume? I did some brief looking around the
>> user guide and related
>> > docs, but couldn't find a knob that turns it off.
>> >
>> > Thanks a lot!
>> > --
>> > Bharath Ravi
>> >
>>
>>
>
Re: Disable flume logging?
Posted by Alexander Lorenz <wg...@googlemail.com>.
to many open files means that your OS is blocking new threads. Simply tune /etc/security/limits.conf and tweak the
soft and hard limits to fit your installation.
-Alex
sent via my mobile device
On Apr 6, 2012, at 5:24 AM, Bharath Ravi <bh...@gmail.com> wrote:
> Thanks Alex! My log4j does seem to be on a verbose configuration.
>
> Now the problem my log is growing is because it's flooded with the following exception:
>
> 2012-04-05 22:45:00,381 WARN org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred during acceptance of message.
> org.apache.thrift.transport.TTransportException: java.net.SocketException: Too many open files
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:137)
> at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
> at org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
> Caused by: java.net.SocketException: Too many open files
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
> at java.net.ServerSocket.implAccept(ServerSocket.java:462)
> at java.net.ServerSocket.accept(ServerSocket.java:430)
> at org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:132)
> ... 2 more
>
> This happens when I attempt to configure around 16 nodes through the master through async "submit" commands
> to the flume shell. I'm not sure why the master would have too many sockets open for such a small number of nodes:
> any pointers to this?
>
> Thanks again!
>
> --
> Bharath Ravi
>
>
>
> On 5 April 2012 02:44, alo alt <wg...@googlemail.com> wrote:
> Hi,
>
> take a look into flume's conf directory, here you should find alog4j config. Could be that you here have a verbose logging configured.
> Before you delete the logs, what let them blow up? That amount of logs are really unusual.
>
> - Alex
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Apr 5, 2012, at 3:52 AM, Bharath Ravi wrote:
>
> > Hi all,
> >
> > The logs generated by flume (especially the master) run to a few GBs in a few minutes
> > on my setup.
> >
> > How do I disable logging on flume? I did some brief looking around the user guide and related
> > docs, but couldn't find a knob that turns it off.
> >
> > Thanks a lot!
> > --
> > Bharath Ravi
> >
>
>
Re: Disable flume logging?
Posted by Bharath Ravi <bh...@gmail.com>.
Thanks Alex! My log4j does seem to be on a verbose configuration.
Now the problem my log is growing is because it's flooded with the
following exception:
2012-04-05 22:45:00,381 WARN
org.apache.thrift.server.TSaneThreadPoolServer: Transport error occurred
during acceptance of message.
org.apache.thrift.transport.TTransportException: java.net.SocketException:
Too many open files
at
org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:137)
at
org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
at
org.apache.thrift.server.TSaneThreadPoolServer$1.run(TSaneThreadPoolServer.java:175)
Caused by: java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at java.net.ServerSocket.accept(ServerSocket.java:430)
at
org.apache.thrift.transport.TSaneServerSocket.acceptImpl(TSaneServerSocket.java:132)
... 2 more
This happens when I attempt to configure around 16 nodes through the master
through async "submit" commands
to the flume shell. I'm not sure why the master would have too many sockets
open for such a small number of nodes:
any pointers to this?
Thanks again!
--
Bharath Ravi
On 5 April 2012 02:44, alo alt <wg...@googlemail.com> wrote:
> Hi,
>
> take a look into flume's conf directory, here you should find alog4j
> config. Could be that you here have a verbose logging configured.
> Before you delete the logs, what let them blow up? That amount of logs are
> really unusual.
>
> - Alex
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Apr 5, 2012, at 3:52 AM, Bharath Ravi wrote:
>
> > Hi all,
> >
> > The logs generated by flume (especially the master) run to a few GBs in
> a few minutes
> > on my setup.
> >
> > How do I disable logging on flume? I did some brief looking around the
> user guide and related
> > docs, but couldn't find a knob that turns it off.
> >
> > Thanks a lot!
> > --
> > Bharath Ravi
> >
>
>
Re: Disable flume logging?
Posted by alo alt <wg...@googlemail.com>.
Hi,
take a look into flume's conf directory, here you should find a log4j config. Could be that you here have a verbose logging configured.
Before you delete the logs, what let them blow up? That amount of logs are really unusual.
- Alex
--
Alexander Lorenz
http://mapredit.blogspot.com
On Apr 5, 2012, at 3:52 AM, Bharath Ravi wrote:
> Hi all,
>
> The logs generated by flume (especially the master) run to a few GBs in a few minutes
> on my setup.
>
> How do I disable logging on flume? I did some brief looking around the user guide and related
> docs, but couldn't find a knob that turns it off.
>
> Thanks a lot!
> --
> Bharath Ravi
>