You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Andre F de Miranda <af...@fucs.org> on 2016/09/11 12:02:44 UTC
Anyone having issues running nifi-kafka-0-9-processors junits?
Hi there,
is anyone else having frequent errors on some of the Kafka processors?
Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 22.248 sec
<<< FAILURE! - in org.apache.nifi.processors.kafka.pubsub.KafkaPublisherTest
validateSuccessfulSendAsDelimited(org.apache.nifi.processors.kafka.pubsub.KafkaPublisherTest)
Time elapsed: 1.413 sec <<< ERROR!
kafka.consumer.ConsumerTimeoutException: null
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
at
org.apache.nifi.processors.kafka.pubsub.KafkaPublisherTest.validateSuccessfulSendAsDelimited(KafkaPublisherTest.java:115)
It seems to happen only (or at least more frequently) when build is
performed with -T 2.0C
Cheers
Re: Anyone having issues running nifi-kafka-0-9-processors junits?
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Ok, I’ll take a look at it.
> On Sep 12, 2016, at 9:20 AM, Joe Skora <js...@gmail.com> wrote:
>
> I've been seeing it for a week for sure, not sure about previously.
>
> But it is intermittent, a re-run usually succeeds.
>
> On Mon, Sep 12, 2016 at 8:58 AM, Andre <an...@fucs.org> wrote:
>
>> Oleg,
>>
>> Yes, it is intermittent and it only seen it happening when using -T
>>
>> I would say it happens 1 out of 5 times so we can say it is intermittent.
>> If I recall correctly, Kafka 0.9 is the most common failure but give me a
>> few days and I will try to confirm that.
>>
>> Cheers
>>
>>
>>
>> On Mon, Sep 12, 2016 at 7:54 AM, Oleg Zhurakousky <
>> ozhurakousky@hortonworks.com> wrote:
>>
>>> Andre
>>>
>>> Couple of questions
>>> 1. Since KafkaPublisher class exists in all 3 Kafka processors (0.9 and
>>> 0.10 Kafka clients) under the same package name, which one does it happen
>>> in? I am assuming its intermittent.
>>> 2. How often does it happen?
>>>
>>> My suspicion is that this test is using embedded Kafka together with NiFi
>>> test mocks and there may be slight possibility of a timing issue under
>> the
>>> load of the build which means the test may need to be tuned. But if this
>>> test fails on 0.10 only it could also mean that we may need to look a
>>> little deeper (since I can’t remember seeng this failure yet) which could
>>> relate to some of the changes in Kafka 10 clients
>>>
>>> Cheers
>>> Oleg
>>>
>>>> On Sep 11, 2016, at 8:02 AM, Andre F de Miranda <af...@fucs.org>
>> wrote:
>>>>
>>>> Hi there,
>>>>
>>>> is anyone else having frequent errors on some of the Kafka processors?
>>>>
>>>> Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 22.248
>>> sec
>>>> <<< FAILURE! - in org.apache.nifi.processors.kafka.pubsub.
>>> KafkaPublisherTest
>>>> validateSuccessfulSendAsDelimited(org.apache.nifi.
>>> processors.kafka.pubsub.KafkaPublisherTest)
>>>> Time elapsed: 1.413 sec <<< ERROR!
>>>> kafka.consumer.ConsumerTimeoutException: null
>>>> at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
>>>> at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
>>>> at kafka.utils.IteratorTemplate.maybeComputeNext(
>>> IteratorTemplate.scala:66)
>>>> at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
>>>> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
>>>> at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
>>>> at
>>>> org.apache.nifi.processors.kafka.pubsub.KafkaPublisherTest.
>>> validateSuccessfulSendAsDelimited(KafkaPublisherTest.java:115)
>>>>
>>>> It seems to happen only (or at least more frequently) when build is
>>>> performed with -T 2.0C
>>>>
>>>>
>>>> Cheers
>>>
>>>
>>
Re: Anyone having issues running nifi-kafka-0-9-processors junits?
Posted by Joe Skora <js...@gmail.com>.
I've been seeing it for a week for sure, not sure about previously.
But it is intermittent, a re-run usually succeeds.
On Mon, Sep 12, 2016 at 8:58 AM, Andre <an...@fucs.org> wrote:
> Oleg,
>
> Yes, it is intermittent and it only seen it happening when using -T
>
> I would say it happens 1 out of 5 times so we can say it is intermittent.
> If I recall correctly, Kafka 0.9 is the most common failure but give me a
> few days and I will try to confirm that.
>
> Cheers
>
>
>
> On Mon, Sep 12, 2016 at 7:54 AM, Oleg Zhurakousky <
> ozhurakousky@hortonworks.com> wrote:
>
> > Andre
> >
> > Couple of questions
> > 1. Since KafkaPublisher class exists in all 3 Kafka processors (0.9 and
> > 0.10 Kafka clients) under the same package name, which one does it happen
> > in? I am assuming its intermittent.
> > 2. How often does it happen?
> >
> > My suspicion is that this test is using embedded Kafka together with NiFi
> > test mocks and there may be slight possibility of a timing issue under
> the
> > load of the build which means the test may need to be tuned. But if this
> > test fails on 0.10 only it could also mean that we may need to look a
> > little deeper (since I can’t remember seeng this failure yet) which could
> > relate to some of the changes in Kafka 10 clients
> >
> > Cheers
> > Oleg
> >
> > > On Sep 11, 2016, at 8:02 AM, Andre F de Miranda <af...@fucs.org>
> wrote:
> > >
> > > Hi there,
> > >
> > > is anyone else having frequent errors on some of the Kafka processors?
> > >
> > > Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 22.248
> > sec
> > > <<< FAILURE! - in org.apache.nifi.processors.kafka.pubsub.
> > KafkaPublisherTest
> > > validateSuccessfulSendAsDelimited(org.apache.nifi.
> > processors.kafka.pubsub.KafkaPublisherTest)
> > > Time elapsed: 1.413 sec <<< ERROR!
> > > kafka.consumer.ConsumerTimeoutException: null
> > > at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
> > > at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
> > > at kafka.utils.IteratorTemplate.maybeComputeNext(
> > IteratorTemplate.scala:66)
> > > at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
> > > at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
> > > at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
> > > at
> > > org.apache.nifi.processors.kafka.pubsub.KafkaPublisherTest.
> > validateSuccessfulSendAsDelimited(KafkaPublisherTest.java:115)
> > >
> > > It seems to happen only (or at least more frequently) when build is
> > > performed with -T 2.0C
> > >
> > >
> > > Cheers
> >
> >
>
Re: Anyone having issues running nifi-kafka-0-9-processors junits?
Posted by Andre <an...@fucs.org>.
Oleg,
Yes, it is intermittent and it only seen it happening when using -T
I would say it happens 1 out of 5 times so we can say it is intermittent.
If I recall correctly, Kafka 0.9 is the most common failure but give me a
few days and I will try to confirm that.
Cheers
On Mon, Sep 12, 2016 at 7:54 AM, Oleg Zhurakousky <
ozhurakousky@hortonworks.com> wrote:
> Andre
>
> Couple of questions
> 1. Since KafkaPublisher class exists in all 3 Kafka processors (0.9 and
> 0.10 Kafka clients) under the same package name, which one does it happen
> in? I am assuming its intermittent.
> 2. How often does it happen?
>
> My suspicion is that this test is using embedded Kafka together with NiFi
> test mocks and there may be slight possibility of a timing issue under the
> load of the build which means the test may need to be tuned. But if this
> test fails on 0.10 only it could also mean that we may need to look a
> little deeper (since I can’t remember seeng this failure yet) which could
> relate to some of the changes in Kafka 10 clients
>
> Cheers
> Oleg
>
> > On Sep 11, 2016, at 8:02 AM, Andre F de Miranda <af...@fucs.org> wrote:
> >
> > Hi there,
> >
> > is anyone else having frequent errors on some of the Kafka processors?
> >
> > Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 22.248
> sec
> > <<< FAILURE! - in org.apache.nifi.processors.kafka.pubsub.
> KafkaPublisherTest
> > validateSuccessfulSendAsDelimited(org.apache.nifi.
> processors.kafka.pubsub.KafkaPublisherTest)
> > Time elapsed: 1.413 sec <<< ERROR!
> > kafka.consumer.ConsumerTimeoutException: null
> > at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
> > at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
> > at kafka.utils.IteratorTemplate.maybeComputeNext(
> IteratorTemplate.scala:66)
> > at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
> > at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
> > at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
> > at
> > org.apache.nifi.processors.kafka.pubsub.KafkaPublisherTest.
> validateSuccessfulSendAsDelimited(KafkaPublisherTest.java:115)
> >
> > It seems to happen only (or at least more frequently) when build is
> > performed with -T 2.0C
> >
> >
> > Cheers
>
>
Re: Anyone having issues running nifi-kafka-0-9-processors junits?
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Andre
Couple of questions
1. Since KafkaPublisher class exists in all 3 Kafka processors (0.9 and 0.10 Kafka clients) under the same package name, which one does it happen in? I am assuming its intermittent.
2. How often does it happen?
My suspicion is that this test is using embedded Kafka together with NiFi test mocks and there may be slight possibility of a timing issue under the load of the build which means the test may need to be tuned. But if this test fails on 0.10 only it could also mean that we may need to look a little deeper (since I can’t remember seeng this failure yet) which could relate to some of the changes in Kafka 10 clients
Cheers
Oleg
> On Sep 11, 2016, at 8:02 AM, Andre F de Miranda <af...@fucs.org> wrote:
>
> Hi there,
>
> is anyone else having frequent errors on some of the Kafka processors?
>
> Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 22.248 sec
> <<< FAILURE! - in org.apache.nifi.processors.kafka.pubsub.KafkaPublisherTest
> validateSuccessfulSendAsDelimited(org.apache.nifi.processors.kafka.pubsub.KafkaPublisherTest)
> Time elapsed: 1.413 sec <<< ERROR!
> kafka.consumer.ConsumerTimeoutException: null
> at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
> at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
> at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
> at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
> at kafka.utils.IteratorTemplate.next(IteratorTemplate.scala:38)
> at kafka.consumer.ConsumerIterator.next(ConsumerIterator.scala:46)
> at
> org.apache.nifi.processors.kafka.pubsub.KafkaPublisherTest.validateSuccessfulSendAsDelimited(KafkaPublisherTest.java:115)
>
> It seems to happen only (or at least more frequently) when build is
> performed with -T 2.0C
>
>
> Cheers