You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Will McQueen (Commented) (JIRA)" <ji...@apache.org> on 2011/11/22 01:21:40 UTC
[jira] [Commented] (FLUME-842) NPE when using jdbc channel with 2
local nodes communicating through avro
[ https://issues.apache.org/jira/browse/FLUME-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154761#comment-13154761 ]
Will McQueen commented on FLUME-842:
------------------------------------
I narrowed this issues down to being a case where there is more than one JDBC channel running on the same machine (where each JDBC channel is contained within a separate Flume agent). The DerbySchemaHandler.schemaExists() fails when attempting to get a connection. It fails with:
java.sql.SQLException: Failed to start database '/home/will/.flume/jdbc-channel/db'. Another instance of Derby may have already booted the database /home/will/.flume/jdbc-channel/db.
The first process launched with this connectUrl:
jdbc:derby:/home/will/.flume/jdbc-channel/db;create=true
For the 2nd process, what's the minimum change to the connectUrl that needs to be made? Would it be just to change the db name from "db" to some other name, "db2"?
> NPE when using jdbc channel with 2 local nodes communicating through avro
> -------------------------------------------------------------------------
>
> Key: FLUME-842
> URL: https://issues.apache.org/jira/browse/FLUME-842
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: NG alpha 1
> Environment: flume-728 branch, commit object 0014756b8462ed2711d2e3af4ccabcabff4fdc70
> Props file "flume.properties":
> host1.channels = jdbcChannel
> host1.sources = avroSource
> host1.sinks = avroSink
> #
> host1.channels.jdbcChannel.type=jdbc
> #
> host1.sources.avroSource.type=avro
> host1.sources.avroSource.channels=jdbcChannel
> host1.sources.avroSource.bind=0.0.0.0
> host1.sources.avroSource.port=41414
> #
> host1.sinks.avroSink.type=avro
> host1.sinks.avroSink.channel=jdbcChannel
> host1.sinks.avroSink.hostname=localhost
> host1.sinks.avroSink.port=41415
> host1.sinks.avroSink.batch-size=100
> #-----
> host2.channels=jdbcChannel
> host2.sources=avroSource
> host2.sinks=loggerSink
> #
> host2.channels.jdbcChannel.type=jdbc
> #
> host2.sources.avroSource.type=avro
> host2.sources.avroSource.channels=jdbcChannel
> host2.sources.avroSource.bind=0.0.0.0
> host2.sources.avroSource.port=41415
> #
> host2.sinks.loggerSink.type=logger
> host2.sinks.loggerSink.channel=jdbcChannel
> Reporter: Will McQueen
>
> Expected comm looks like this:
> AvroCLIClient ==(41414)==> avroSource=>jdbcChannel=>avroSink ==(41415)==> avroSource=>jdbcChannel=>loggerSink
> Shorthand:
> AvroCLIClient ==(41414)==> host1 ==(41415)==> host2
> Steps:
> 1) Run host2 with main class org.apache.flume.node.Application and args:
> -f /home/will/git/flume-asf/conf/flume.properties -n host2
> 2) Run host1 with same main class, and args:
> -f /home/will/git/flume-asf/conf/flume.properties -n host1
> 3) Run Avro client with main class org.apache.flume.client.avro.AvroCLIClient and args:
> -H localhost -p 41414 -F /etc/passwd
> I see the following NPE on host1 after step 2 (and before step 3). There's no problem when switching to memory channel for both hosts.
> 2011-11-07 11:00:42,103 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:205)] Failed to load configuration data. Exception follows.
> java.lang.NullPointerException
> at org.apache.flume.channel.jdbc.impl.DerbySchemaHandler.schemaExists(DerbySchemaHandler.java:321)
> at org.apache.flume.channel.jdbc.impl.JdbcChannelProviderImpl.initializeSchema(JdbcChannelProviderImpl.java:108)
> at org.apache.flume.channel.jdbc.impl.JdbcChannelProviderImpl.initialize(JdbcChannelProviderImpl.java:95)
> at org.apache.flume.channel.jdbc.JdbcChannelProviderFactory.getProvider(JdbcChannelProviderFactory.java:35)
> at org.apache.flume.channel.jdbc.JdbcChannel.configure(JdbcChannel.java:81)
> at org.apache.flume.conf.Configurables.configure(Configurables.java:22)
> at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadChannels(PropertiesFileConfigurationProvider.java:223)
> at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:184)
> at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:124)
> at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$1(AbstractFileConfigurationProvider.java:114)
> at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:203)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira