You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Shiti Saxena <ss...@gmail.com> on 2016/01/12 07:14:52 UTC
Kafka Connect usage
Hi,
I tried executing the following,
bin/connect-standalone.sh config/connect-standalone.properties
config/connect-file-source.properties config/connect-console-sink.properties
I created a file text.txt in kafka directory but get the error,
ERROR Failed to flush WorkerSourceTask{id=local-file-source-0}, timed out
while waiting for producer to flush outstanding messages, 1 left
({ProducerRecord(topic=connect-test, partition=null, key=[B@71bb594d,
value=[B@7f7ca90a=ProducerRecord(topic=connect-test, partition=null,
key=[B@71bb594d, value=[B@7f7ca90a})
(org.apache.kafka.connect.runtime.WorkerSourceTask:237)
[2016-01-12 11:43:51,948] ERROR Failed to commit offsets for
WorkerSourceTask{id=local-file-source-0}
(org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter:112)
Is any other configuration required?
Thanks,
Shiti
Re: Kafka Connect usage
Posted by Shiti Saxena <ss...@gmail.com>.
Hi,
Thanks for replying soon. It would be nice if an error was thrown saying
Kafka server not available. I kept looking at the code in workerSourceTask
and couldn't understand what was wrong.
Thanks,
Shiti
On 12 Jan 2016 23:54, "Liquan Pei" <li...@gmail.com> wrote:
> Hi Shiti,
>
> You need to start Kafka server and Zookeeper before running Kafka Connect.
>
> Thanks,
> Liquan
>
> On Tue, Jan 12, 2016 at 10:22 AM, Shiti Saxena <ss...@gmail.com>
> wrote:
>
> > Hi Alex,
> >
> > I am using the default files.
> >
> > Do we need to start Kafka server and zookeeper separately before starting
> > Kafka connect?
> >
> > Thanks,
> > Shiti
> > On 12 Jan 2016 23:11, "Alex Loddengaard" <al...@confluent.io> wrote:
> >
> > > Hi Shiti, I'm not able to reproduce the problem with the default
> > > *.properties files you're passing to connect-standalone.sh. Can you
> share
> > > these three files?
> > >
> > > Thanks,
> > >
> > > Alex
> > >
> > > On Mon, Jan 11, 2016 at 10:14 PM, Shiti Saxena <ss...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I tried executing the following,
> > > >
> > > > bin/connect-standalone.sh config/connect-standalone.properties
> > > > config/connect-file-source.properties
> > > > config/connect-console-sink.properties
> > > >
> > > > I created a file text.txt in kafka directory but get the error,
> > > >
> > > > ERROR Failed to flush WorkerSourceTask{id=local-file-source-0}, timed
> > out
> > > > while waiting for producer to flush outstanding messages, 1 left
> > > > ({ProducerRecord(topic=connect-test, partition=null, key=[B@71bb594d
> ,
> > > > value=[B@7f7ca90a=ProducerRecord(topic=connect-test, partition=null,
> > > > key=[B@71bb594d, value=[B@7f7ca90a})
> > > > (org.apache.kafka.connect.runtime.WorkerSourceTask:237)
> > > > [2016-01-12 11:43:51,948] ERROR Failed to commit offsets for
> > > > WorkerSourceTask{id=local-file-source-0}
> > > > (org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter:112)
> > > >
> > > > Is any other configuration required?
> > > >
> > > > Thanks,
> > > > Shiti
> > > >
> > >
> > >
> > >
> > > --
> > > *Alex Loddengaard | **Solutions Architect | Confluent*
> > > *Download Apache Kafka and Confluent Platform:
> www.confluent.io/download
> > > <http://www.confluent.io/download>*
> > >
> >
>
>
>
> --
> Liquan Pei
> Department of Physics
> University of Massachusetts Amherst
>
Re: Kafka Connect usage
Posted by Liquan Pei <li...@gmail.com>.
Hi Shiti,
You need to start Kafka server and Zookeeper before running Kafka Connect.
Thanks,
Liquan
On Tue, Jan 12, 2016 at 10:22 AM, Shiti Saxena <ss...@gmail.com>
wrote:
> Hi Alex,
>
> I am using the default files.
>
> Do we need to start Kafka server and zookeeper separately before starting
> Kafka connect?
>
> Thanks,
> Shiti
> On 12 Jan 2016 23:11, "Alex Loddengaard" <al...@confluent.io> wrote:
>
> > Hi Shiti, I'm not able to reproduce the problem with the default
> > *.properties files you're passing to connect-standalone.sh. Can you share
> > these three files?
> >
> > Thanks,
> >
> > Alex
> >
> > On Mon, Jan 11, 2016 at 10:14 PM, Shiti Saxena <ss...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I tried executing the following,
> > >
> > > bin/connect-standalone.sh config/connect-standalone.properties
> > > config/connect-file-source.properties
> > > config/connect-console-sink.properties
> > >
> > > I created a file text.txt in kafka directory but get the error,
> > >
> > > ERROR Failed to flush WorkerSourceTask{id=local-file-source-0}, timed
> out
> > > while waiting for producer to flush outstanding messages, 1 left
> > > ({ProducerRecord(topic=connect-test, partition=null, key=[B@71bb594d,
> > > value=[B@7f7ca90a=ProducerRecord(topic=connect-test, partition=null,
> > > key=[B@71bb594d, value=[B@7f7ca90a})
> > > (org.apache.kafka.connect.runtime.WorkerSourceTask:237)
> > > [2016-01-12 11:43:51,948] ERROR Failed to commit offsets for
> > > WorkerSourceTask{id=local-file-source-0}
> > > (org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter:112)
> > >
> > > Is any other configuration required?
> > >
> > > Thanks,
> > > Shiti
> > >
> >
> >
> >
> > --
> > *Alex Loddengaard | **Solutions Architect | Confluent*
> > *Download Apache Kafka and Confluent Platform: www.confluent.io/download
> > <http://www.confluent.io/download>*
> >
>
--
Liquan Pei
Department of Physics
University of Massachusetts Amherst
Re: Kafka Connect usage
Posted by Shiti Saxena <ss...@gmail.com>.
Hi Alex,
I am using the default files.
Do we need to start Kafka server and zookeeper separately before starting
Kafka connect?
Thanks,
Shiti
On 12 Jan 2016 23:11, "Alex Loddengaard" <al...@confluent.io> wrote:
> Hi Shiti, I'm not able to reproduce the problem with the default
> *.properties files you're passing to connect-standalone.sh. Can you share
> these three files?
>
> Thanks,
>
> Alex
>
> On Mon, Jan 11, 2016 at 10:14 PM, Shiti Saxena <ss...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I tried executing the following,
> >
> > bin/connect-standalone.sh config/connect-standalone.properties
> > config/connect-file-source.properties
> > config/connect-console-sink.properties
> >
> > I created a file text.txt in kafka directory but get the error,
> >
> > ERROR Failed to flush WorkerSourceTask{id=local-file-source-0}, timed out
> > while waiting for producer to flush outstanding messages, 1 left
> > ({ProducerRecord(topic=connect-test, partition=null, key=[B@71bb594d,
> > value=[B@7f7ca90a=ProducerRecord(topic=connect-test, partition=null,
> > key=[B@71bb594d, value=[B@7f7ca90a})
> > (org.apache.kafka.connect.runtime.WorkerSourceTask:237)
> > [2016-01-12 11:43:51,948] ERROR Failed to commit offsets for
> > WorkerSourceTask{id=local-file-source-0}
> > (org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter:112)
> >
> > Is any other configuration required?
> >
> > Thanks,
> > Shiti
> >
>
>
>
> --
> *Alex Loddengaard | **Solutions Architect | Confluent*
> *Download Apache Kafka and Confluent Platform: www.confluent.io/download
> <http://www.confluent.io/download>*
>
Re: Kafka Connect usage
Posted by Alex Loddengaard <al...@confluent.io>.
Hi Shiti, I'm not able to reproduce the problem with the default
*.properties files you're passing to connect-standalone.sh. Can you share
these three files?
Thanks,
Alex
On Mon, Jan 11, 2016 at 10:14 PM, Shiti Saxena <ss...@gmail.com>
wrote:
> Hi,
>
> I tried executing the following,
>
> bin/connect-standalone.sh config/connect-standalone.properties
> config/connect-file-source.properties
> config/connect-console-sink.properties
>
> I created a file text.txt in kafka directory but get the error,
>
> ERROR Failed to flush WorkerSourceTask{id=local-file-source-0}, timed out
> while waiting for producer to flush outstanding messages, 1 left
> ({ProducerRecord(topic=connect-test, partition=null, key=[B@71bb594d,
> value=[B@7f7ca90a=ProducerRecord(topic=connect-test, partition=null,
> key=[B@71bb594d, value=[B@7f7ca90a})
> (org.apache.kafka.connect.runtime.WorkerSourceTask:237)
> [2016-01-12 11:43:51,948] ERROR Failed to commit offsets for
> WorkerSourceTask{id=local-file-source-0}
> (org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter:112)
>
> Is any other configuration required?
>
> Thanks,
> Shiti
>
--
*Alex Loddengaard | **Solutions Architect | Confluent*
*Download Apache Kafka and Confluent Platform: www.confluent.io/download
<http://www.confluent.io/download>*