[GitHub] [incubator-doris] XShun opened a new issue #2271: Does routine load will make oom?

[GitBox <gi...@apache.org>]

XShun opened a new issue #2271: Does routine load will make oom?
URL: https://github.com/apache/incubator-doris/issues/2271
 
 
   These days, I tried to load data to doris from kafka. I created a routine load and I wrote some data to kafka. Then I saw the status of load is RUNNING, and I can execute query from the destination table. Because it is just test, so I just wrote about 200,000 rows, and I didn't stop the routine load. Actually, when 200,000 rows load finished, there is no other data for 'routine load job' to consume, and I didn't stop the 'routine load job'. 
   The next day, I found BE down. Then I looked over the BE log and I saw a lot of 'kafka consume timeout: xxxxxxxx-xxxxxxxx', and there is an oom error in dmesg, and BE was killed because of oom. 
   So, does routine load will make oom when there is no other data in kafka for 'routine load job' to consume? Should I stop the routine load when there is no data to kafka to avoid this error?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org