You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@carbondata.apache.org by litao <li...@126.com> on 2019/01/24 08:09:18 UTC

【Discuss】load data cause GC overhead limit exceeded

when i change driver size 5GB, excutor size 5GB, but forget to change lru
cache, it still 20GB.
when i do the query with select count(*) from.
I get the error, may be we can protect about that.

0: carbon_query_x@jdbc:hive2://x..> select count(*) from x;

Error: java.io.IOException: Problem in loading segment blocks: GC overhead
limit exceeded (state=,code=0)



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: 【Discuss】load data cause GC overhead limit exceeded

Posted by litao <li...@126.com>.

i have code for that,and test finished.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: 【Discuss】load data cause GC overhead limit exceeded

Posted by David CaiQiang <da...@gmail.com>.

maybe we can validate this property and limit it to less than the total
memory size (or 60%...)  of the driver side




-----
Best Regards
David Cai
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/