You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/08/17 17:05:26 UTC
[GitHub] [incubator-pinot] lgo opened a new issue #5879: Configuring ingestion to default to pushing metadata so controllers do not always pull segments for metadata
lgo opened a new issue #5879:
URL: https://github.com/apache/incubator-pinot/issues/5879
From Slack:
> joey: the controller is pulling the entire segment. Does that happen for every segment? Is it just pulling it to grab the metadata, or does it persist additional data?
> That seems like a gnarly bottleneck if we're pushing TBs of data from a Spark app.
> kishore: its just pulling it for metadata
> there is a way to push the metadata as well
> so that it does not pull the entire segment
> we should make that the default
cc @kishoreg
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] lgo commented on issue #5879: Configuring ingestion to default to pushing metadata so controllers do not always pull segments for metadata
Posted by GitBox <gi...@apache.org>.
lgo commented on issue #5879:
URL: https://github.com/apache/incubator-pinot/issues/5879#issuecomment-674999462
Additionally, what's the configuration parameters to enable this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #5879: Configuring ingestion to default to pushing metadata so controllers do not always pull segments for metadata
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #5879:
URL: https://github.com/apache/incubator-pinot/issues/5879#issuecomment-683484237
I will add this support into ingestion jobs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org