[GitHub] [hudi] bvaradar commented on pull request #2899: [HUDI-1865] Make embedded time line service singleton

[GitBox <gi...@apache.org>]

bvaradar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-839495288


   > > Looking at the usage of `createEmbeddedTimelineService()`, there are broadly two.
   > > 
   > > * AbstractHoodieClient can start (during intiialization) and stop (during close()) the embedded server view. If we return the same static instance, it will be tied to the same underlying file system view which may be stale i.e may not see the new commit. See `createInMemoryFileSystemView()` in FileSystemViewManager for e.g which is initialized during the timeline server creation.
   > > * DeltaStreamer in `DeltaSync#reInitWriteClient` actually keeps a single instance of the embedded timeline server. it would n't reinit each time.
   > > 
   > > Overall, can we think through any side effects of this change?
   > 
   > Yes, so we should reload the active timeline each time we fetch the filesystem view from the meta client, the `TimelineService` would check the timeline hashcode from client right ?
   
   Yes, this is already happening in RemoteHoodieTableFileSystemView. 
   
   > @vinothchandar Thanks, i have added the config options in the PR, i'm also wondering the details how the filesystem view was synced with a new client instant time request, could you give some help @bvaradar ?
   
   @danny0405 : The logic to determine timeline staleness in the timeline server is in https://github.com/apache/hudi/blob/master/hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/RequestHandler.java#L100
   
   Basically, the timeline-server relies on the last timeline instant known to the client to determine if it is holding a stale copy 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org