You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Guillaume Polaert <gp...@cyres.fr> on 2012/05/21 15:03:57 UTC

RollingFile Sink problems

Hi everybody,

I've problems with RollingFile sink.
First, I'm seeking about the sink's properties. I've found two properties in RollingFileSink.java : rollInterval and directory. Are there anymore ?

Then, I've this error when I'm running the node :

...
SINKS: {sink=ComponentConfiguration[sink]
  CONFIG: {directory=/tmp/log, type=file_roll}
  RUNNER:   ComponentConfiguration[runner]
    CONFIG: {}


, log-sink1=ComponentConfiguration[log-sink1]
  CONFIG: {type=logger, channel=ch1}
  RUNNER:   ComponentConfiguration[runner]
    CONFIG: {}

}
...
2012-05-21 08:29:15,246 (conf-file-poller-0) [INFO - org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:69)] Creating instance of sink sink typefile_roll
2012-05-21 08:29:15,253 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows.
java.lang.IllegalArgumentException: Directory may not be null
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
        at org.apache.flume.sink.RollingFileSink.configure(RollingFileSink.java:74)
        at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
...

Flume.conf

# Define a memory channel called ch1 on agent1
agent1.channels.ch1.type = memory

# Define an Avro source called avro-source1 on agent1 and tell it
# to bind to 0.0.0.0:41414. Connect it to channel ch1.
agent1.sources.avro-source1.channels = ch1
agent1.sources.avro-source1.type = avro
agent1.sources.avro-source1.bind = 0.0.0.0
agent1.sources.avro-source1.port = 41414

# Define a logger sink that simply logs all events it receives
# and connect it to the other end of the same channel.
agent1.sinks.log-sink1.channel = ch1
agent1.sinks.log-sink1.type = logger
agent1.sinks.sink.channel = ch1
agent1.sinks.sink.type = file_roll
agent1.sinks.sink.directory=/tmp/log

# Finally, now that we've defined all of our components, tell
# agent1 which ones we want to activate.
agent1.channels = ch1
agent1.sources = avro-source1
agent1.sinks = sink

What is wrong ? I don't understand the problem.

Guillaume

Re: RollingFile Sink problems

Posted by alo alt <wg...@googlemail.com>.
The .sink was in the main config available, but the identifier not. 

@ Guillaume
agent1.sinks.file-sink-1.sink.channel = ch1
agent1.sinks.file-sink-1.sink.directory = /foo/bar
agent1.sinks.file-sink-1.sink.type = file_roll

works.

Not the "file-sink-1" identifier after .sinks.

I was reopened the Jira, but it is a doc gap, so I close again (sorry Mike). I'll file a doc jira about.

Thanks
- Alex


--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

On May 22, 2012, at 10:11 AM, Mike Percy wrote:

> Some of the flume NG config options are a little strange. This is one of them.
> 
> You have to specify agent.sinks.file-sink.sink.directory = /some/dir
> 
> It doesn't make a lot of sense, but it's historical.
> 
> HTH,
> Mike
> 
> On Monday, May 21, 2012 at 11:57 PM, Guillaume Polaert wrote:
> 
>> Thank you for helping me out.
>> 
>> Guillaume Polaert | Cyrès Conseil
>> 
>> 
>> -----Message d'origine-----
>> De : alo alt [mailto:wget.null@googlemail.com]
>> Envoyé : mardi 22 mai 2012 08:34
>> À : flume-user@incubator.apache.org
>> Objet : Re: RollingFile Sink problems
>> 
>> created https://issues.apache.org/jira/browse/FLUME-1222
>> 
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>> 
>> On May 21, 2012, at 6:27 PM, alo alt wrote:
>> 
>>> Looks like a config.parser issue in RollingFileSink.java or Configuration.java. Please open a Jira for.
>>> The error messages comes from the Preconditions check (Google Guava library) in RollingFileSink.java, indeed the variable for the directory is empty.
>>> 
>>> - Alex
>>> 
>>> --
>>> Alexander Alten-Lorenz
>>> http://mapredit.blogspot.com
>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>> 
>>> On May 21, 2012, at 3:50 PM, Guillaume Polaert wrote:
>>> 
>>>> Thank you for the link.
>>>> I've created a empty directory /tmp/log (mkdir+chmod).
>>>> 
>>>> Do I have to create other things ?
>>>> 
>>>> -----Message d'origine-----
>>>> De : alo alt [mailto:wget.null@googlemail.com] Envoyé : lundi 21 mai
>>>> 2012 15:31 À : flume-user@incubator.apache.org Objet : Re:
>>>> RollingFile Sink problems
>>>> 
>>>> Hi,
>>>> 
>>>> http://archive.cloudera.com/cdh4/cdh/4/flume-ng-1.1.0-cdh4.0.0b2/Flum
>>>> eUserGuide.html#h.frlqql6vu68
>>>> 
>>>> Means, /tmp/log is empty or non existent, isn't it?
>>>> 
>>>> - Alex
>>>> 
>>>> --
>>>> Alexander Alten-Lorenz
>>>> http://mapredit.blogspot.com
>>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>>> 
>>>> On May 21, 2012, at 3:03 PM, Guillaume Polaert wrote:
>>>> 
>>>>> Hi everybody,
>>>>> 
>>>>> I've problems with RollingFile sink.
>>>>> First, I'm seeking about the sink's properties. I've found two properties in RollingFileSink.java : rollInterval and directory. Are there anymore ?
>>>>> 
>>>>> Then, I've this error when I'm running the node :
>>>>> 
>>>>> .
>>>>> SINKS: {sink=ComponentConfiguration[sink]
>>>>> CONFIG: {directory=/tmp/log, type=file_roll}
>>>>> RUNNER: ComponentConfiguration[runner]
>>>>> CONFIG: {}
>>>>> 
>>>>> 
>>>>> , log-sink1=ComponentConfiguration[log-sink1]
>>>>> CONFIG: {type=logger, channel=ch1}
>>>>> RUNNER: ComponentConfiguration[runner]
>>>>> CONFIG: {}
>>>>> 
>>>>> }
>>>>> .
>>>>> 2012-05-21 08:29:15,246 (conf-file-poller-0) [INFO -
>>>>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.j
>>>>> av a:69)] Creating instance of sink sink typefile_roll
>>>>> 2012-05-21 08:29:15,253 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows.
>>>>> java.lang.IllegalArgumentException: Directory may not be null
>>>>> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
>>>>> at org.apache.flume.sink.RollingFileSink.configure(RollingFileSink.java:74)
>>>>> at
>>>>> org.apache.flume.conf.Configurables.configure(Configurables.java:41)
>>>>> .
>>>>> 
>>>>> Flume.conf
>>>>> 
>>>>> # Define a memory channel called ch1 on agent1
>>>>> agent1.channels.ch1.type = memory
>>>>> 
>>>>> # Define an Avro source called avro-source1 on agent1 and tell it #
>>>>> to bind to 0.0.0.0:41414. Connect it to channel ch1.
>>>>> agent1.sources.avro-source1.channels = ch1
>>>>> agent1.sources.avro-source1.type = avro
>>>>> agent1.sources.avro-source1.bind = 0.0.0.0
>>>>> agent1.sources.avro-source1.port = 41414
>>>>> 
>>>>> # Define a logger sink that simply logs all events it receives # and
>>>>> connect it to the other end of the same channel.
>>>>> agent1.sinks.log-sink1.channel = ch1 agent1.sinks.log-sink1.type =
>>>>> logger agent1.sinks.sink.channel = ch1 agent1.sinks.sink.type =
>>>>> file_roll agent1.sinks.sink.directory=/tmp/log
>>>>> 
>>>>> # Finally, now that we've defined all of our components, tell #
>>>>> agent1 which ones we want to activate.
>>>>> agent1.channels = ch1
>>>>> agent1.sources = avro-source1
>>>>> agent1.sinks = sink
>>>>> 
>>>>> What is wrong ? I don't understand the problem.
>>>>> 
>>>>> Guillaume
> 


Re: RollingFile Sink problems

Posted by Mike Percy <mp...@cloudera.com>.
Some of the flume NG config options are a little strange. This is one of them.

You have to specify agent.sinks.file-sink.sink.directory = /some/dir

It doesn't make a lot of sense, but it's historical.

HTH,
Mike


On Monday, May 21, 2012 at 11:57 PM, Guillaume Polaert wrote:

> Thank you for helping me out.
>  
> Guillaume Polaert | Cyrès Conseil  
>  
>  
> -----Message d'origine-----
> De : alo alt [mailto:wget.null@googlemail.com]  
> Envoyé : mardi 22 mai 2012 08:34
> À : flume-user@incubator.apache.org (mailto:flume-user@incubator.apache.org)
> Objet : Re: RollingFile Sink problems
>  
> created https://issues.apache.org/jira/browse/FLUME-1222
>  
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>  
> On May 21, 2012, at 6:27 PM, alo alt wrote:
>  
> > Looks like a config.parser issue in RollingFileSink.java or Configuration.java. Please open a Jira for.
> > The error messages comes from the Preconditions check (Google Guava library) in RollingFileSink.java, indeed the variable for the directory is empty.  
> >  
> > - Alex
> >  
> > --
> > Alexander Alten-Lorenz
> > http://mapredit.blogspot.com
> > German Hadoop LinkedIn Group: http://goo.gl/N8pCF
> >  
> > On May 21, 2012, at 3:50 PM, Guillaume Polaert wrote:
> >  
> > > Thank you for the link.
> > > I've created a empty directory /tmp/log (mkdir+chmod).  
> > >  
> > > Do I have to create other things ?
> > >  
> > > -----Message d'origine-----
> > > De : alo alt [mailto:wget.null@googlemail.com] Envoyé : lundi 21 mai  
> > > 2012 15:31 À : flume-user@incubator.apache.org (mailto:flume-user@incubator.apache.org) Objet : Re:  
> > > RollingFile Sink problems
> > >  
> > > Hi,
> > >  
> > > http://archive.cloudera.com/cdh4/cdh/4/flume-ng-1.1.0-cdh4.0.0b2/Flum
> > > eUserGuide.html#h.frlqql6vu68
> > >  
> > > Means, /tmp/log is empty or non existent, isn't it?
> > >  
> > > - Alex
> > >  
> > > --
> > > Alexander Alten-Lorenz
> > > http://mapredit.blogspot.com
> > > German Hadoop LinkedIn Group: http://goo.gl/N8pCF
> > >  
> > > On May 21, 2012, at 3:03 PM, Guillaume Polaert wrote:
> > >  
> > > > Hi everybody,
> > > >  
> > > > I've problems with RollingFile sink.
> > > > First, I'm seeking about the sink's properties. I've found two properties in RollingFileSink.java : rollInterval and directory. Are there anymore ?
> > > >  
> > > > Then, I've this error when I'm running the node :
> > > >  
> > > > .
> > > > SINKS: {sink=ComponentConfiguration[sink]
> > > > CONFIG: {directory=/tmp/log, type=file_roll}
> > > > RUNNER: ComponentConfiguration[runner]
> > > > CONFIG: {}
> > > >  
> > > >  
> > > > , log-sink1=ComponentConfiguration[log-sink1]
> > > > CONFIG: {type=logger, channel=ch1}
> > > > RUNNER: ComponentConfiguration[runner]
> > > > CONFIG: {}
> > > >  
> > > > }
> > > > .
> > > > 2012-05-21 08:29:15,246 (conf-file-poller-0) [INFO -  
> > > > org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.j
> > > > av a:69)] Creating instance of sink sink typefile_roll
> > > > 2012-05-21 08:29:15,253 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows.
> > > > java.lang.IllegalArgumentException: Directory may not be null
> > > > at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
> > > > at org.apache.flume.sink.RollingFileSink.configure(RollingFileSink.java:74)
> > > > at
> > > > org.apache.flume.conf.Configurables.configure(Configurables.java:41)
> > > > .
> > > >  
> > > > Flume.conf
> > > >  
> > > > # Define a memory channel called ch1 on agent1  
> > > > agent1.channels.ch1.type = memory
> > > >  
> > > > # Define an Avro source called avro-source1 on agent1 and tell it #  
> > > > to bind to 0.0.0.0:41414. Connect it to channel ch1.
> > > > agent1.sources.avro-source1.channels = ch1  
> > > > agent1.sources.avro-source1.type = avro  
> > > > agent1.sources.avro-source1.bind = 0.0.0.0  
> > > > agent1.sources.avro-source1.port = 41414
> > > >  
> > > > # Define a logger sink that simply logs all events it receives # and  
> > > > connect it to the other end of the same channel.
> > > > agent1.sinks.log-sink1.channel = ch1 agent1.sinks.log-sink1.type =  
> > > > logger agent1.sinks.sink.channel = ch1 agent1.sinks.sink.type =  
> > > > file_roll agent1.sinks.sink.directory=/tmp/log
> > > >  
> > > > # Finally, now that we've defined all of our components, tell #  
> > > > agent1 which ones we want to activate.
> > > > agent1.channels = ch1
> > > > agent1.sources = avro-source1
> > > > agent1.sinks = sink
> > > >  
> > > > What is wrong ? I don't understand the problem.
> > > >  
> > > > Guillaume  


RE: RollingFile Sink problems

Posted by Guillaume Polaert <gp...@cyres.fr>.
Thank you for helping me out.

Guillaume Polaert | Cyrès Conseil 


-----Message d'origine-----
De : alo alt [mailto:wget.null@googlemail.com] 
Envoyé : mardi 22 mai 2012 08:34
À : flume-user@incubator.apache.org
Objet : Re: RollingFile Sink problems

created https://issues.apache.org/jira/browse/FLUME-1222

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

On May 21, 2012, at 6:27 PM, alo alt wrote:

> Looks like a config.parser issue in RollingFileSink.java or Configuration.java. Please open a Jira for.
> The error messages comes from the Preconditions check (Google Guava library) in RollingFileSink.java, indeed the variable for the directory is empty. 
> 
> - Alex
> 
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
> 
> On May 21, 2012, at 3:50 PM, Guillaume Polaert wrote:
> 
>> Thank you for the link.
>> I've created a empty directory /tmp/log (mkdir+chmod). 
>> 
>> Do I have to create other things ?
>> 
>> -----Message d'origine-----
>> De : alo alt [mailto:wget.null@googlemail.com] Envoyé : lundi 21 mai 
>> 2012 15:31 À : flume-user@incubator.apache.org Objet : Re: 
>> RollingFile Sink problems
>> 
>> Hi,
>> 
>> http://archive.cloudera.com/cdh4/cdh/4/flume-ng-1.1.0-cdh4.0.0b2/Flum
>> eUserGuide.html#h.frlqql6vu68
>> 
>> Means, /tmp/log is empty or non existent, isn't it?
>> 
>> - Alex
>> 
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>> 
>> On May 21, 2012, at 3:03 PM, Guillaume Polaert wrote:
>> 
>>> Hi everybody,
>>> 
>>> I've problems with RollingFile sink.
>>> First, I'm seeking about the sink's properties. I've found two properties in RollingFileSink.java : rollInterval and directory. Are there anymore ?
>>> 
>>> Then, I've this error when I'm running the node :
>>> 
>>> .
>>> SINKS: {sink=ComponentConfiguration[sink]
>>> CONFIG: {directory=/tmp/log, type=file_roll}
>>> RUNNER:   ComponentConfiguration[runner]
>>>   CONFIG: {}
>>> 
>>> 
>>> , log-sink1=ComponentConfiguration[log-sink1]
>>> CONFIG: {type=logger, channel=ch1}
>>> RUNNER:   ComponentConfiguration[runner]
>>>   CONFIG: {}
>>> 
>>> }
>>> .
>>> 2012-05-21 08:29:15,246 (conf-file-poller-0) [INFO - 
>>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.j
>>> av a:69)] Creating instance of sink sink typefile_roll
>>> 2012-05-21 08:29:15,253 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows.
>>> java.lang.IllegalArgumentException: Directory may not be null
>>>       at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
>>>       at org.apache.flume.sink.RollingFileSink.configure(RollingFileSink.java:74)
>>>       at
>>> org.apache.flume.conf.Configurables.configure(Configurables.java:41)
>>> .
>>> 
>>> Flume.conf
>>> 
>>> # Define a memory channel called ch1 on agent1 
>>> agent1.channels.ch1.type = memory
>>> 
>>> # Define an Avro source called avro-source1 on agent1 and tell it # 
>>> to bind to 0.0.0.0:41414. Connect it to channel ch1.
>>> agent1.sources.avro-source1.channels = ch1 
>>> agent1.sources.avro-source1.type = avro 
>>> agent1.sources.avro-source1.bind = 0.0.0.0 
>>> agent1.sources.avro-source1.port = 41414
>>> 
>>> # Define a logger sink that simply logs all events it receives # and 
>>> connect it to the other end of the same channel.
>>> agent1.sinks.log-sink1.channel = ch1 agent1.sinks.log-sink1.type = 
>>> logger agent1.sinks.sink.channel = ch1 agent1.sinks.sink.type = 
>>> file_roll agent1.sinks.sink.directory=/tmp/log
>>> 
>>> # Finally, now that we've defined all of our components, tell # 
>>> agent1 which ones we want to activate.
>>> agent1.channels = ch1
>>> agent1.sources = avro-source1
>>> agent1.sinks = sink
>>> 
>>> What is wrong ? I don't understand the problem.
>>> 
>>> Guillaume
>> 
> 


Re: RollingFile Sink problems

Posted by alo alt <wg...@googlemail.com>.
created https://issues.apache.org/jira/browse/FLUME-1222

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

On May 21, 2012, at 6:27 PM, alo alt wrote:

> Looks like a config.parser issue in RollingFileSink.java or Configuration.java. Please open a Jira for.
> The error messages comes from the Preconditions check (Google Guava library) in RollingFileSink.java, indeed the variable for the directory is empty. 
> 
> - Alex
> 
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
> 
> On May 21, 2012, at 3:50 PM, Guillaume Polaert wrote:
> 
>> Thank you for the link.
>> I've created a empty directory /tmp/log (mkdir+chmod). 
>> 
>> Do I have to create other things ?
>> 
>> -----Message d'origine-----
>> De : alo alt [mailto:wget.null@googlemail.com] 
>> Envoyé : lundi 21 mai 2012 15:31
>> À : flume-user@incubator.apache.org
>> Objet : Re: RollingFile Sink problems
>> 
>> Hi,
>> 
>> http://archive.cloudera.com/cdh4/cdh/4/flume-ng-1.1.0-cdh4.0.0b2/FlumeUserGuide.html#h.frlqql6vu68
>> 
>> Means, /tmp/log is empty or non existent, isn't it?
>> 
>> - Alex
>> 
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>> 
>> On May 21, 2012, at 3:03 PM, Guillaume Polaert wrote:
>> 
>>> Hi everybody,
>>> 
>>> I've problems with RollingFile sink.
>>> First, I'm seeking about the sink's properties. I've found two properties in RollingFileSink.java : rollInterval and directory. Are there anymore ?
>>> 
>>> Then, I've this error when I'm running the node :
>>> 
>>> .
>>> SINKS: {sink=ComponentConfiguration[sink]
>>> CONFIG: {directory=/tmp/log, type=file_roll}
>>> RUNNER:   ComponentConfiguration[runner]
>>>   CONFIG: {}
>>> 
>>> 
>>> , log-sink1=ComponentConfiguration[log-sink1]
>>> CONFIG: {type=logger, channel=ch1}
>>> RUNNER:   ComponentConfiguration[runner]
>>>   CONFIG: {}
>>> 
>>> }
>>> .
>>> 2012-05-21 08:29:15,246 (conf-file-poller-0) [INFO - 
>>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.jav
>>> a:69)] Creating instance of sink sink typefile_roll
>>> 2012-05-21 08:29:15,253 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows.
>>> java.lang.IllegalArgumentException: Directory may not be null
>>>       at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
>>>       at org.apache.flume.sink.RollingFileSink.configure(RollingFileSink.java:74)
>>>       at 
>>> org.apache.flume.conf.Configurables.configure(Configurables.java:41)
>>> .
>>> 
>>> Flume.conf
>>> 
>>> # Define a memory channel called ch1 on agent1 
>>> agent1.channels.ch1.type = memory
>>> 
>>> # Define an Avro source called avro-source1 on agent1 and tell it # to 
>>> bind to 0.0.0.0:41414. Connect it to channel ch1.
>>> agent1.sources.avro-source1.channels = ch1 
>>> agent1.sources.avro-source1.type = avro 
>>> agent1.sources.avro-source1.bind = 0.0.0.0 
>>> agent1.sources.avro-source1.port = 41414
>>> 
>>> # Define a logger sink that simply logs all events it receives # and 
>>> connect it to the other end of the same channel.
>>> agent1.sinks.log-sink1.channel = ch1
>>> agent1.sinks.log-sink1.type = logger
>>> agent1.sinks.sink.channel = ch1
>>> agent1.sinks.sink.type = file_roll
>>> agent1.sinks.sink.directory=/tmp/log
>>> 
>>> # Finally, now that we've defined all of our components, tell # agent1 
>>> which ones we want to activate.
>>> agent1.channels = ch1
>>> agent1.sources = avro-source1
>>> agent1.sinks = sink
>>> 
>>> What is wrong ? I don't understand the problem.
>>> 
>>> Guillaume
>> 
> 


Re: RollingFile Sink problems

Posted by alo alt <wg...@googlemail.com>.
Looks like a config.parser issue in RollingFileSink.java or Configuration.java. Please open a Jira for.
The error messages comes from the Preconditions check (Google Guava library) in RollingFileSink.java, indeed the variable for the directory is empty. 

- Alex

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

On May 21, 2012, at 3:50 PM, Guillaume Polaert wrote:

> Thank you for the link.
> I've created a empty directory /tmp/log (mkdir+chmod). 
> 
> Do I have to create other things ?
> 
> -----Message d'origine-----
> De : alo alt [mailto:wget.null@googlemail.com] 
> Envoyé : lundi 21 mai 2012 15:31
> À : flume-user@incubator.apache.org
> Objet : Re: RollingFile Sink problems
> 
> Hi,
> 
> http://archive.cloudera.com/cdh4/cdh/4/flume-ng-1.1.0-cdh4.0.0b2/FlumeUserGuide.html#h.frlqql6vu68
> 
> Means, /tmp/log is empty or non existent, isn't it?
> 
> - Alex
> 
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
> 
> On May 21, 2012, at 3:03 PM, Guillaume Polaert wrote:
> 
>> Hi everybody,
>> 
>> I've problems with RollingFile sink.
>> First, I'm seeking about the sink's properties. I've found two properties in RollingFileSink.java : rollInterval and directory. Are there anymore ?
>> 
>> Then, I've this error when I'm running the node :
>> 
>> .
>> SINKS: {sink=ComponentConfiguration[sink]
>>  CONFIG: {directory=/tmp/log, type=file_roll}
>>  RUNNER:   ComponentConfiguration[runner]
>>    CONFIG: {}
>> 
>> 
>> , log-sink1=ComponentConfiguration[log-sink1]
>>  CONFIG: {type=logger, channel=ch1}
>>  RUNNER:   ComponentConfiguration[runner]
>>    CONFIG: {}
>> 
>> }
>> .
>> 2012-05-21 08:29:15,246 (conf-file-poller-0) [INFO - 
>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.jav
>> a:69)] Creating instance of sink sink typefile_roll
>> 2012-05-21 08:29:15,253 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows.
>> java.lang.IllegalArgumentException: Directory may not be null
>>        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
>>        at org.apache.flume.sink.RollingFileSink.configure(RollingFileSink.java:74)
>>        at 
>> org.apache.flume.conf.Configurables.configure(Configurables.java:41)
>> .
>> 
>> Flume.conf
>> 
>> # Define a memory channel called ch1 on agent1 
>> agent1.channels.ch1.type = memory
>> 
>> # Define an Avro source called avro-source1 on agent1 and tell it # to 
>> bind to 0.0.0.0:41414. Connect it to channel ch1.
>> agent1.sources.avro-source1.channels = ch1 
>> agent1.sources.avro-source1.type = avro 
>> agent1.sources.avro-source1.bind = 0.0.0.0 
>> agent1.sources.avro-source1.port = 41414
>> 
>> # Define a logger sink that simply logs all events it receives # and 
>> connect it to the other end of the same channel.
>> agent1.sinks.log-sink1.channel = ch1
>> agent1.sinks.log-sink1.type = logger
>> agent1.sinks.sink.channel = ch1
>> agent1.sinks.sink.type = file_roll
>> agent1.sinks.sink.directory=/tmp/log
>> 
>> # Finally, now that we've defined all of our components, tell # agent1 
>> which ones we want to activate.
>> agent1.channels = ch1
>> agent1.sources = avro-source1
>> agent1.sinks = sink
>> 
>> What is wrong ? I don't understand the problem.
>> 
>> Guillaume
> 


RE: RollingFile Sink problems

Posted by Guillaume Polaert <gp...@cyres.fr>.
Thank you for the link.
I've created a empty directory /tmp/log (mkdir+chmod). 

Do I have to create other things ?

-----Message d'origine-----
De : alo alt [mailto:wget.null@googlemail.com] 
Envoyé : lundi 21 mai 2012 15:31
À : flume-user@incubator.apache.org
Objet : Re: RollingFile Sink problems

Hi,

http://archive.cloudera.com/cdh4/cdh/4/flume-ng-1.1.0-cdh4.0.0b2/FlumeUserGuide.html#h.frlqql6vu68

Means, /tmp/log is empty or non existent, isn't it?

- Alex

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

On May 21, 2012, at 3:03 PM, Guillaume Polaert wrote:

> Hi everybody,
>  
> I've problems with RollingFile sink.
> First, I'm seeking about the sink's properties. I've found two properties in RollingFileSink.java : rollInterval and directory. Are there anymore ?
>  
> Then, I've this error when I'm running the node :
>  
> .
> SINKS: {sink=ComponentConfiguration[sink]
>   CONFIG: {directory=/tmp/log, type=file_roll}
>   RUNNER:   ComponentConfiguration[runner]
>     CONFIG: {}
>  
>  
> , log-sink1=ComponentConfiguration[log-sink1]
>   CONFIG: {type=logger, channel=ch1}
>   RUNNER:   ComponentConfiguration[runner]
>     CONFIG: {}
>  
> }
> .
> 2012-05-21 08:29:15,246 (conf-file-poller-0) [INFO - 
> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.jav
> a:69)] Creating instance of sink sink typefile_roll
> 2012-05-21 08:29:15,253 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows.
> java.lang.IllegalArgumentException: Directory may not be null
>         at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
>         at org.apache.flume.sink.RollingFileSink.configure(RollingFileSink.java:74)
>         at 
> org.apache.flume.conf.Configurables.configure(Configurables.java:41)
> .
>  
> Flume.conf
>  
> # Define a memory channel called ch1 on agent1 
> agent1.channels.ch1.type = memory
>  
> # Define an Avro source called avro-source1 on agent1 and tell it # to 
> bind to 0.0.0.0:41414. Connect it to channel ch1.
> agent1.sources.avro-source1.channels = ch1 
> agent1.sources.avro-source1.type = avro 
> agent1.sources.avro-source1.bind = 0.0.0.0 
> agent1.sources.avro-source1.port = 41414
>  
> # Define a logger sink that simply logs all events it receives # and 
> connect it to the other end of the same channel.
> agent1.sinks.log-sink1.channel = ch1
> agent1.sinks.log-sink1.type = logger
> agent1.sinks.sink.channel = ch1
> agent1.sinks.sink.type = file_roll
> agent1.sinks.sink.directory=/tmp/log
>  
> # Finally, now that we've defined all of our components, tell # agent1 
> which ones we want to activate.
> agent1.channels = ch1
> agent1.sources = avro-source1
> agent1.sinks = sink
>  
> What is wrong ? I don't understand the problem.
>  
> Guillaume


Re: RollingFile Sink problems

Posted by alo alt <wg...@googlemail.com>.
Hi,

http://archive.cloudera.com/cdh4/cdh/4/flume-ng-1.1.0-cdh4.0.0b2/FlumeUserGuide.html#h.frlqql6vu68

Means, /tmp/log is empty or non existent, isn't it?

- Alex

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

On May 21, 2012, at 3:03 PM, Guillaume Polaert wrote:

> Hi everybody,
>  
> I’ve problems with RollingFile sink.
> First, I’m seeking about the sink’s properties. I’ve found two properties in RollingFileSink.java : rollInterval and directory. Are there anymore ?
>  
> Then, I’ve this error when I’m running the node :
>  
> …
> SINKS: {sink=ComponentConfiguration[sink]
>   CONFIG: {directory=/tmp/log, type=file_roll}
>   RUNNER:   ComponentConfiguration[runner]
>     CONFIG: {}
>  
>  
> , log-sink1=ComponentConfiguration[log-sink1]
>   CONFIG: {type=logger, channel=ch1}
>   RUNNER:   ComponentConfiguration[runner]
>     CONFIG: {}
>  
> }
> …
> 2012-05-21 08:29:15,246 (conf-file-poller-0) [INFO - org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:69)] Creating instance of sink sink typefile_roll
> 2012-05-21 08:29:15,253 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows.
> java.lang.IllegalArgumentException: Directory may not be null
>         at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
>         at org.apache.flume.sink.RollingFileSink.configure(RollingFileSink.java:74)
>         at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
> …
>  
> Flume.conf
>  
> # Define a memory channel called ch1 on agent1
> agent1.channels.ch1.type = memory
>  
> # Define an Avro source called avro-source1 on agent1 and tell it
> # to bind to 0.0.0.0:41414. Connect it to channel ch1.
> agent1.sources.avro-source1.channels = ch1
> agent1.sources.avro-source1.type = avro
> agent1.sources.avro-source1.bind = 0.0.0.0
> agent1.sources.avro-source1.port = 41414
>  
> # Define a logger sink that simply logs all events it receives
> # and connect it to the other end of the same channel.
> agent1.sinks.log-sink1.channel = ch1
> agent1.sinks.log-sink1.type = logger
> agent1.sinks.sink.channel = ch1
> agent1.sinks.sink.type = file_roll
> agent1.sinks.sink.directory=/tmp/log
>  
> # Finally, now that we've defined all of our components, tell
> # agent1 which ones we want to activate.
> agent1.channels = ch1
> agent1.sources = avro-source1
> agent1.sinks = sink
>  
> What is wrong ? I don’t understand the problem.
>  
> Guillaume