You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/01/19 07:30:20 UTC
[GitHub] [hudi] yuzhaojing opened a new issue #4636: [SUPPORT] Sync timeline from embedded timeline server in flink pipline
yuzhaojing opened a new issue #4636:
URL: https://github.com/apache/hudi/issues/4636
At present, in the Flink-Hudi pipeline, each task will scan the meta directory to obtain the latest timeline, which will cause frequent get listing operations on HDFS and cause a lot of pressure.
A proposal is we can modify the way to get the timeline in the Flink-Hudi pipeline and pull the active timeline through the embedded timeline server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on issue #4636: [SUPPORT] Sync timeline from embedded timeline server in flink pipline
Posted by GitBox <gi...@apache.org>.
xushiyan commented on issue #4636:
URL: https://github.com/apache/hudi/issues/4636#issuecomment-1025271879
@danny0405 any suggestion on using timeline server with flink writer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on issue #4636: [SUPPORT] Sync timeline from embedded timeline server in flink pipline
Posted by GitBox <gi...@apache.org>.
danny0405 commented on issue #4636:
URL: https://github.com/apache/hudi/issues/4636#issuecomment-1032262107
+1 generally for the timeline server based instant request, we already did that for MARKER files already, what do you think @vinothchandar , do we need a new RFC here ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org