You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/09/11 06:57:44 UTC
[GitHub] [incubator-pinot] fx19880617 opened a new issue #6003: Review PinotFS ListFile Implementations
fx19880617 opened a new issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003
Per https://github.com/apache/incubator-pinot/pull/6002, S3 has a limit on each ListObject call response. I think this policy may also be true for other blob store.
This issue is created to review other PinotFs implementations to ensure the similar issues are not presenting there.
* [ ] AzurePinotFs
* [ ] ADLSGen2PinotFs
* [ ] GcsPinotFs
* [ ] HadoopPinotFs
* [ ] S3PinotFs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #6003: Review PinotFS ListFile Implementations
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690914135
S3PinotFs is fixed in #6002
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #6003: Review PinotFS ListFile Implementations
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690914135
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #6003: Review PinotFS ListFile Implementations
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690926043
@snleee seems that ADLSGen2 also has the problem of truncation:
https://docs.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/filesystem/list
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 removed a comment on issue #6003: Review PinotFS ListFile Implementations
Posted by GitBox <gi...@apache.org>.
fx19880617 removed a comment on issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690920467
@snleee
@elonazoulay
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #6003: Review PinotFS ListFile Implementations
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690914135
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 removed a comment on issue #6003: Review PinotFS ListFile Implementations
Posted by GitBox <gi...@apache.org>.
fx19880617 removed a comment on issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690920467
@snleee
@elonazoulay
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #6003: Review PinotFS ListFile Implementations
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690920467
@snleee
@elonazoulay
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #6003: Review PinotFS ListFile Implementations
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690927594
@elonazoulay could you help check if GcsPinotFs is good when the bucket has many objects?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 removed a comment on issue #6003: Review PinotFS ListFile Implementations
Posted by GitBox <gi...@apache.org>.
fx19880617 removed a comment on issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690920467
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 removed a comment on issue #6003: Review PinotFS ListFile Implementations
Posted by GitBox <gi...@apache.org>.
fx19880617 removed a comment on issue #6003:
URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690920467
@snleee
@elonazoulay
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org