You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com> on 2016/03/16 16:01:35 UTC
Strange Timeout error with PutKafka
I say strange because the timeout (63ms) is so very short. The communication timeout I’ve set is 30 sec. Has anyone overseen this?
2016-03-16 14:41:38,227 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=852c8d42-a2fa-3478-b06b-84ceb6\
6f8b0b] Failed to send StandardFlowFileRecord[uuid=a0074162-0066-49e7-918b-cea1cfc5a955,claim=StandardContentClaim [resourceClaim=StandardResour\
ceClaim[id=1458079089737-67, container=default, section=67], offset=377796, length=743],offset=0,name=2349680613178720,size=743] to Kafka; routi\
ng to 'failure'; last failure reason reported was org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.;: org.\
apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.
Re: Strange Timeout error with PutKafka
Posted by Joe Witt <jo...@gmail.com>.
Chris,
If your the adventurous build from master type that does have the good
stuff now which works against 0.8 and 0.9 brokers.
I can send you a build of that if you'd like to try it out and let us
know if it helps.
Thanks
Joe
On Wed, Mar 16, 2016 at 11:41 AM, McDermott, Chris Kevin (MSDU -
STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
> Joe,
>
> I’ll checkout the disk-space. We are running 0.9. If disk space is not the issue we’ll give 0.8 a try.
>
> Thanks very much for your quick reply.
>
> Cheers,
> Chris
>
>
>
> On 3/16/16, 11:04 AM, "Joe Witt" <jo...@gmail.com> wrote:
>
>>Chris,
>>I have seen that when the diskspace kafka relies on is full. We've
>>seen a number of interesting exceptions recently in testing various
>>configurations. But recommend checking that.
>>
>>Also, what version of Kafka broker are you using? With Apache NiFi
>>0.5.x we moved to the kafka client 0.9. In doing that we messed up
>>support for 0.8. So...with the upcoming release we will move back to
>>the 0.8 client and thus it works great with Kafka 0.8 and 0.9 brokers
>>albeit without the new SSL and Kerberos support they added in their
>>0.9 work. We have a JIRA item to go after that for our next feature
>>bearing release.
>>
>>Thanks
>>Joe
>>
>>On Wed, Mar 16, 2016 at 11:01 AM, McDermott, Chris Kevin (MSDU -
>>STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>>> I say strange because the timeout (63ms) is so very short. The communication timeout I’ve set is 30 sec. Has anyone overseen this?
>>>
>>> 2016-03-16 14:41:38,227 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=852c8d42-a2fa-3478-b06b-84ceb6\
>>> 6f8b0b] Failed to send StandardFlowFileRecord[uuid=a0074162-0066-49e7-918b-cea1cfc5a955,claim=StandardContentClaim [resourceClaim=StandardResour\
>>> ceClaim[id=1458079089737-67, container=default, section=67], offset=377796, length=743],offset=0,name=2349680613178720,size=743] to Kafka; routi\
>>> ng to 'failure'; last failure reason reported was org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.;: org.\
>>> apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.
Re: Strange Timeout error with PutKafka
Posted by Joe Witt <jo...@gmail.com>.
Chris,
Assuming the client can validate #2 i am with you. Please do feel
free to fire up a JIRA for this.
Thanks
Joe
On Wed, Mar 16, 2016 at 1:24 PM, McDermott, Chris Kevin (MSDU -
STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
> It turns out the root cause of the problem was an invalid topic name. Strange error for that!
>
> I think there are a couple of improvements could be made to PutKafka.
>
> 1. Check the validity of the the expression in the topic property.
> 2. Check the validity of the topic name before attempting to write to the topic.
>
> Chris
>
>
>
> On 3/16/16, 11:41 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com> wrote:
>
>>Joe,
>>
>>I’ll checkout the disk-space. We are running 0.9. If disk space is not the issue we’ll give 0.8 a try.
>>
>>Thanks very much for your quick reply.
>>
>>Cheers,
>>Chris
>>
>>
>>
>>On 3/16/16, 11:04 AM, "Joe Witt" <jo...@gmail.com> wrote:
>>
>>>Chris,
>>>I have seen that when the diskspace kafka relies on is full. We've
>>>seen a number of interesting exceptions recently in testing various
>>>configurations. But recommend checking that.
>>>
>>>Also, what version of Kafka broker are you using? With Apache NiFi
>>>0.5.x we moved to the kafka client 0.9. In doing that we messed up
>>>support for 0.8. So...with the upcoming release we will move back to
>>>the 0.8 client and thus it works great with Kafka 0.8 and 0.9 brokers
>>>albeit without the new SSL and Kerberos support they added in their
>>>0.9 work. We have a JIRA item to go after that for our next feature
>>>bearing release.
>>>
>>>Thanks
>>>Joe
>>>
>>>On Wed, Mar 16, 2016 at 11:01 AM, McDermott, Chris Kevin (MSDU -
>>>STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>>>> I say strange because the timeout (63ms) is so very short. The communication timeout I’ve set is 30 sec. Has anyone overseen this?
>>>>
>>>> 2016-03-16 14:41:38,227 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=852c8d42-a2fa-3478-b06b-84ceb6\
>>>> 6f8b0b] Failed to send StandardFlowFileRecord[uuid=a0074162-0066-49e7-918b-cea1cfc5a955,claim=StandardContentClaim [resourceClaim=StandardResour\
>>>> ceClaim[id=1458079089737-67, container=default, section=67], offset=377796, length=743],offset=0,name=2349680613178720,size=743] to Kafka; routi\
>>>> ng to 'failure'; last failure reason reported was org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.;: org.\
>>>> apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.
Re: Strange Timeout error with PutKafka
Posted by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>.
It turns out the root cause of the problem was an invalid topic name. Strange error for that!
I think there are a couple of improvements could be made to PutKafka.
1. Check the validity of the the expression in the topic property.
2. Check the validity of the topic name before attempting to write to the topic.
Chris
On 3/16/16, 11:41 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com> wrote:
>Joe,
>
>I’ll checkout the disk-space. We are running 0.9. If disk space is not the issue we’ll give 0.8 a try.
>
>Thanks very much for your quick reply.
>
>Cheers,
>Chris
>
>
>
>On 3/16/16, 11:04 AM, "Joe Witt" <jo...@gmail.com> wrote:
>
>>Chris,
>>I have seen that when the diskspace kafka relies on is full. We've
>>seen a number of interesting exceptions recently in testing various
>>configurations. But recommend checking that.
>>
>>Also, what version of Kafka broker are you using? With Apache NiFi
>>0.5.x we moved to the kafka client 0.9. In doing that we messed up
>>support for 0.8. So...with the upcoming release we will move back to
>>the 0.8 client and thus it works great with Kafka 0.8 and 0.9 brokers
>>albeit without the new SSL and Kerberos support they added in their
>>0.9 work. We have a JIRA item to go after that for our next feature
>>bearing release.
>>
>>Thanks
>>Joe
>>
>>On Wed, Mar 16, 2016 at 11:01 AM, McDermott, Chris Kevin (MSDU -
>>STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>>> I say strange because the timeout (63ms) is so very short. The communication timeout I’ve set is 30 sec. Has anyone overseen this?
>>>
>>> 2016-03-16 14:41:38,227 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=852c8d42-a2fa-3478-b06b-84ceb6\
>>> 6f8b0b] Failed to send StandardFlowFileRecord[uuid=a0074162-0066-49e7-918b-cea1cfc5a955,claim=StandardContentClaim [resourceClaim=StandardResour\
>>> ceClaim[id=1458079089737-67, container=default, section=67], offset=377796, length=743],offset=0,name=2349680613178720,size=743] to Kafka; routi\
>>> ng to 'failure'; last failure reason reported was org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.;: org.\
>>> apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.
Re: Strange Timeout error with PutKafka
Posted by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>.
Joe,
I’ll checkout the disk-space. We are running 0.9. If disk space is not the issue we’ll give 0.8 a try.
Thanks very much for your quick reply.
Cheers,
Chris
On 3/16/16, 11:04 AM, "Joe Witt" <jo...@gmail.com> wrote:
>Chris,
>I have seen that when the diskspace kafka relies on is full. We've
>seen a number of interesting exceptions recently in testing various
>configurations. But recommend checking that.
>
>Also, what version of Kafka broker are you using? With Apache NiFi
>0.5.x we moved to the kafka client 0.9. In doing that we messed up
>support for 0.8. So...with the upcoming release we will move back to
>the 0.8 client and thus it works great with Kafka 0.8 and 0.9 brokers
>albeit without the new SSL and Kerberos support they added in their
>0.9 work. We have a JIRA item to go after that for our next feature
>bearing release.
>
>Thanks
>Joe
>
>On Wed, Mar 16, 2016 at 11:01 AM, McDermott, Chris Kevin (MSDU -
>STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>> I say strange because the timeout (63ms) is so very short. The communication timeout I’ve set is 30 sec. Has anyone overseen this?
>>
>> 2016-03-16 14:41:38,227 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=852c8d42-a2fa-3478-b06b-84ceb6\
>> 6f8b0b] Failed to send StandardFlowFileRecord[uuid=a0074162-0066-49e7-918b-cea1cfc5a955,claim=StandardContentClaim [resourceClaim=StandardResour\
>> ceClaim[id=1458079089737-67, container=default, section=67], offset=377796, length=743],offset=0,name=2349680613178720,size=743] to Kafka; routi\
>> ng to 'failure'; last failure reason reported was org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.;: org.\
>> apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.
Re: Strange Timeout error with PutKafka
Posted by Joe Witt <jo...@gmail.com>.
Chris,
I have seen that when the diskspace kafka relies on is full. We've
seen a number of interesting exceptions recently in testing various
configurations. But recommend checking that.
Also, what version of Kafka broker are you using? With Apache NiFi
0.5.x we moved to the kafka client 0.9. In doing that we messed up
support for 0.8. So...with the upcoming release we will move back to
the 0.8 client and thus it works great with Kafka 0.8 and 0.9 brokers
albeit without the new SSL and Kerberos support they added in their
0.9 work. We have a JIRA item to go after that for our next feature
bearing release.
Thanks
Joe
On Wed, Mar 16, 2016 at 11:01 AM, McDermott, Chris Kevin (MSDU -
STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
> I say strange because the timeout (63ms) is so very short. The communication timeout I’ve set is 30 sec. Has anyone overseen this?
>
> 2016-03-16 14:41:38,227 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=852c8d42-a2fa-3478-b06b-84ceb6\
> 6f8b0b] Failed to send StandardFlowFileRecord[uuid=a0074162-0066-49e7-918b-cea1cfc5a955,claim=StandardContentClaim [resourceClaim=StandardResour\
> ceClaim[id=1458079089737-67, container=default, section=67], offset=377796, length=743],offset=0,name=2349680613178720,size=743] to Kafka; routi\
> ng to 'failure'; last failure reason reported was org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.;: org.\
> apache.kafka.common.errors.TimeoutException: Failed to update metadata after 63 ms.