[GitHub] [incubator-pinot] jackjlli opened a new pull request #4253: Add pre-compute Hadoop job

[GitBox <gi...@apache.org>]

jackjlli opened a new pull request #4253: Add pre-compute Hadoop job
URL: https://github.com/apache/incubator-pinot/pull/4253
 
 
   We’ve seen some requests from client that we add partitioning, sorting and resizing functionalities in/before the actual Pinot related Hadoop jobs.
   And without any control on the number/size of the segments, we may run into issues like generating too many small segments. It’s essential to add a layer before segment creation so that we lower the chances of making too small segments; the segment size could be more unified. On the other hand, when a new use case is onboard, clients don’t need to worry about partitioning and sorting.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org