You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by 38797715 <38...@qq.com> on 2021/03/03 09:31:00 UTC
[2.9.1]Failed to find DHT update future for deferred update response
Hi team,
When using DataStreamer to write a large amount of data at high speed,
if one server node fails, then other nodes will appear a lot of
following information, and finally they will also fail:
[2021-03-03 15:49:44,516][WARN ][sys-stripe-6-#7][atomic] Failed to find
DHT update future for deferred update response [futId=142939814,
nodeId=bfe0d1a9-8e0c-4a9e-8b62-041f2f252a80,
res=GridDhtAtomicDeferredUpdateResponse [futIds=GridLongList [idx=256,
arr=[142939814,142731432,142827914,142939816,142731434,142939818,142939820,142939822,142939824,142939826,142939828,142731436,142939830,142731438,142939832,142939834,142731440,142827916,142731442,142827918,142827920,142939836,142827922,142939838,142827924,142827926,142939840,142939842,142827928,142731444,142939844,142827930,142731446,142939846,142827932,142939848,142731448,142827934,142939850,142827936,142939852,142827938,142827940,142939854,142827942,142939856,142939858,142827944,142827946,142939860,142827948,142939862,142827950,142731450,142939864,142827952,142731452,142939866,142939868,142827954,142731454,142827956,142731456,142827958,142939870,142827960,142731458,142939872,142827962,142939874,142731460,142939876,142939878,142731462,142731464,142939880,142731466,142939882,142939884,142731468,142939886,142731470,142939888,142731472,142731474,142939890,142731476,142939892,142731478,142939894,142939896,142731480,142731482,142731484,142731486,142731488,142731490,142731492,142731494,142939898,142939900,142731496,142731498,142939902,142939904,142731500,142939906,142939908,142731502,142939910,142731504,142939912,142939914,142731506,142939916,142731508,142731510,142939918,142939920,142939922,142731512,142939924,142731514,142939926,142731516,142939928,142731518,142939930,142731520,142939932,142939934,142939936,142731522,142939938,142731524,142939940,142731526,142939942,142939944,142731528,142939946,142731530,142939948,142939950,142731532,142939952,142939954,142731534,142939956,142731536,142939958,142731538,142939960,142731540,142731542,142939962,142939964,142731544,142731546,142731548,142939966,142731550,142939968,142731552,142939970,142731554,142731556,142939972,142731558,142731560,142731562,142939974,142731564,142939976,142939978,142939980,
What I want to ask is:
1.Are these logs related to node failures when the DataStreamer writes?
2.Does DataStreamer have a mechanism for fail over? We know that
DataStreamer sends data to specified nodes in batches. When a node
fails, what is the behavior of DataStreamer?
Re: [2.9.1]Failed to find DHT update future for deferred update response
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I believe that DataStreamer has limited failover support, such as: when you
do streamer.close(), either it will return normally, in which case the data
was written to caches, or it will throw an exception, in which case only an
unknown subset of data were written, and you need to repeat the operation.
I'm not sure that "Failed to find DHT update future for deferred update
response" may cause subsequent node failure. Can you provide complete logs
from all server nodes in the cluster?
Regards,
--
Ilya Kasnacheev
пт, 5 мар. 2021 г. в 16:16, 38797715 <38...@qq.com>:
> Hi Ilya,
>
> It's not easy to build a reproducible environment, which is probably a
> problem in use, not necessarily a bug.
>
> In particular, I want to know the behavior of DataStreamer in case of node
> failure and whether it has the function of fail over.
> 在 2021/3/5 下午8:16, Ilya Kasnacheev 写道:
>
> Hello!
>
> Do you happen to have a reproducer for this issue? I've not seen anything
> similar.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 3 мар. 2021 г. в 12:31, 38797715 <38...@qq.com>:
>
>> Hi team,
>>
>> When using DataStreamer to write a large amount of data at high speed, if
>> one server node fails, then other nodes will appear a lot of following
>> information, and finally they will also fail:
>>
>> [2021-03-03 15:49:44,516][WARN ][sys-stripe-6-#7][atomic] Failed to find
>> DHT update future for deferred update response [futId=142939814,
>> nodeId=bfe0d1a9-8e0c-4a9e-8b62-041f2f252a80,
>> res=GridDhtAtomicDeferredUpdateResponse [futIds=GridLongList [idx=256,
>> arr=[142939814,142731432,142827914,142939816,142731434,142939818,142939820,142939822,142939824,142939826,142939828,142731436,142939830,142731438,142939832,142939834,142731440,142827916,142731442,142827918,142827920,142939836,142827922,142939838,142827924,142827926,142939840,142939842,142827928,142731444,142939844,142827930,142731446,142939846,142827932,142939848,142731448,142827934,142939850,142827936,142939852,142827938,142827940,142939854,142827942,142939856,142939858,142827944,142827946,142939860,142827948,142939862,142827950,142731450,142939864,142827952,142731452,142939866,142939868,142827954,142731454,142827956,142731456,142827958,142939870,142827960,142731458,142939872,142827962,142939874,142731460,142939876,142939878,142731462,142731464,142939880,142731466,142939882,142939884,142731468,142939886,142731470,142939888,142731472,142731474,142939890,142731476,142939892,142731478,142939894,142939896,142731480,142731482,142731484,142731486,142731488,142731490,142731492,142731494,142939898,142939900,142731496,142731498,142939902,142939904,142731500,142939906,142939908,142731502,142939910,142731504,142939912,142939914,142731506,142939916,142731508,142731510,142939918,142939920,142939922,142731512,142939924,142731514,142939926,142731516,142939928,142731518,142939930,142731520,142939932,142939934,142939936,142731522,142939938,142731524,142939940,142731526,142939942,142939944,142731528,142939946,142731530,142939948,142939950,142731532,142939952,142939954,142731534,142939956,142731536,142939958,142731538,142939960,142731540,142731542,142939962,142939964,142731544,142731546,142731548,142939966,142731550,142939968,142731552,142939970,142731554,142731556,142939972,142731558,142731560,142731562,142939974,142731564,142939976,142939978,142939980,
>>
>> What I want to ask is:
>>
>> 1.Are these logs related to node failures when the DataStreamer writes?
>>
>> 2.Does DataStreamer have a mechanism for fail over? We know that
>> DataStreamer sends data to specified nodes in batches. When a node
>> fails, what is the behavior of DataStreamer?
>>
>
Re: [2.9.1]Failed to find DHT update future for deferred update
response
Posted by 38797715 <38...@qq.com>.
Hi Ilya,
It's not easy to build a reproducible environment, which is probably a
problem in use, not necessarily a bug.
In particular, I want to know the behavior of DataStreamer in case of
node failure and whether it has the function of fail over.
在 2021/3/5 下午8:16, Ilya Kasnacheev 写道:
> Hello!
>
> Do you happen to have a reproducer for this issue? I've not seen
> anything similar.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 3 мар. 2021 г. в 12:31, 38797715 <38797715@qq.com
> <ma...@qq.com>>:
>
> Hi team,
>
> When using DataStreamer to write a large amount of data at high
> speed, if one server node fails, then other nodes will appear a
> lot of following information, and finally they will also fail:
>
> [2021-03-03 15:49:44,516][WARN ][sys-stripe-6-#7][atomic] Failed
> to find DHT update future for deferred update response
> [futId=142939814, nodeId=bfe0d1a9-8e0c-4a9e-8b62-041f2f252a80,
> res=GridDhtAtomicDeferredUpdateResponse [futIds=GridLongList
> [idx=256,
> arr=[142939814,142731432,142827914,142939816,142731434,142939818,142939820,142939822,142939824,142939826,142939828,142731436,142939830,142731438,142939832,142939834,142731440,142827916,142731442,142827918,142827920,142939836,142827922,142939838,142827924,142827926,142939840,142939842,142827928,142731444,142939844,142827930,142731446,142939846,142827932,142939848,142731448,142827934,142939850,142827936,142939852,142827938,142827940,142939854,142827942,142939856,142939858,142827944,142827946,142939860,142827948,142939862,142827950,142731450,142939864,142827952,142731452,142939866,142939868,142827954,142731454,142827956,142731456,142827958,142939870,142827960,142731458,142939872,142827962,142939874,142731460,142939876,142939878,142731462,142731464,142939880,142731466,142939882,142939884,142731468,142939886,142731470,142939888,142731472,142731474,142939890,142731476,142939892,142731478,142939894,142939896,142731480,142731482,142731484,142731486,142731488,142731490,142731492,142731494,142939898,142939900,142731496,142731498,142939902,142939904,142731500,142939906,142939908,142731502,142939910,142731504,142939912,142939914,142731506,142939916,142731508,142731510,142939918,142939920,142939922,142731512,142939924,142731514,142939926,142731516,142939928,142731518,142939930,142731520,142939932,142939934,142939936,142731522,142939938,142731524,142939940,142731526,142939942,142939944,142731528,142939946,142731530,142939948,142939950,142731532,142939952,142939954,142731534,142939956,142731536,142939958,142731538,142939960,142731540,142731542,142939962,142939964,142731544,142731546,142731548,142939966,142731550,142939968,142731552,142939970,142731554,142731556,142939972,142731558,142731560,142731562,142939974,142731564,142939976,142939978,142939980,
>
> What I want to ask is:
>
> 1.Are these logs related to node failures when the DataStreamer
> writes?
>
> 2.Does DataStreamer have a mechanism for fail over? We know that
> DataStreamer sends data to specified nodes in batches. When a node
> fails, what is the behavior of DataStreamer?
>
Re: [2.9.1]Failed to find DHT update future for deferred update response
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Do you happen to have a reproducer for this issue? I've not seen anything
similar.
Regards,
--
Ilya Kasnacheev
ср, 3 мар. 2021 г. в 12:31, 38797715 <38...@qq.com>:
> Hi team,
>
> When using DataStreamer to write a large amount of data at high speed, if
> one server node fails, then other nodes will appear a lot of following
> information, and finally they will also fail:
>
> [2021-03-03 15:49:44,516][WARN ][sys-stripe-6-#7][atomic] Failed to find
> DHT update future for deferred update response [futId=142939814,
> nodeId=bfe0d1a9-8e0c-4a9e-8b62-041f2f252a80,
> res=GridDhtAtomicDeferredUpdateResponse [futIds=GridLongList [idx=256,
> arr=[142939814,142731432,142827914,142939816,142731434,142939818,142939820,142939822,142939824,142939826,142939828,142731436,142939830,142731438,142939832,142939834,142731440,142827916,142731442,142827918,142827920,142939836,142827922,142939838,142827924,142827926,142939840,142939842,142827928,142731444,142939844,142827930,142731446,142939846,142827932,142939848,142731448,142827934,142939850,142827936,142939852,142827938,142827940,142939854,142827942,142939856,142939858,142827944,142827946,142939860,142827948,142939862,142827950,142731450,142939864,142827952,142731452,142939866,142939868,142827954,142731454,142827956,142731456,142827958,142939870,142827960,142731458,142939872,142827962,142939874,142731460,142939876,142939878,142731462,142731464,142939880,142731466,142939882,142939884,142731468,142939886,142731470,142939888,142731472,142731474,142939890,142731476,142939892,142731478,142939894,142939896,142731480,142731482,142731484,142731486,142731488,142731490,142731492,142731494,142939898,142939900,142731496,142731498,142939902,142939904,142731500,142939906,142939908,142731502,142939910,142731504,142939912,142939914,142731506,142939916,142731508,142731510,142939918,142939920,142939922,142731512,142939924,142731514,142939926,142731516,142939928,142731518,142939930,142731520,142939932,142939934,142939936,142731522,142939938,142731524,142939940,142731526,142939942,142939944,142731528,142939946,142731530,142939948,142939950,142731532,142939952,142939954,142731534,142939956,142731536,142939958,142731538,142939960,142731540,142731542,142939962,142939964,142731544,142731546,142731548,142939966,142731550,142939968,142731552,142939970,142731554,142731556,142939972,142731558,142731560,142731562,142939974,142731564,142939976,142939978,142939980,
>
> What I want to ask is:
>
> 1.Are these logs related to node failures when the DataStreamer writes?
>
> 2.Does DataStreamer have a mechanism for fail over? We know that
> DataStreamer sends data to specified nodes in batches. When a node fails,
> what is the behavior of DataStreamer?
>