You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@carbondata.apache.org by David CaiQiang <da...@gmail.com> on 2020/05/08 01:58:04 UTC

Re: Carbon over-use cluster resources

Hi, manhua
  Now no_sort reuse the loading flow of local_sort. It is not a good
solution and led to the situation which you have mentioned. In my opinion,
we need to adjust the loading flow of no_sort, maybe like global_sort
finally.

  In addition, the producer-consumer pattern in data encoding and
compression also can be optimized for no_sort and global_sort, maybe just
prefetch one page and process it instead of using a thread pool.



-----
Best Regards
David Cai
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/