[jira] [Assigned] (HUDI-6249) Make maps in HoodieTableFileSystemView concurrent maps

["Jonathan Vexler (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Vexler reassigned HUDI-6249:
-------------------------------------

    Assignee: Jonathan Vexler

> Make maps in HoodieTableFileSystemView concurrent maps
> ------------------------------------------------------
>
>                 Key: HUDI-6249
>                 URL: https://issues.apache.org/jira/browse/HUDI-6249
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: timeline-server
>            Reporter: Jonathan Vexler
>            Assignee: Jonathan Vexler
>            Priority: Major
>
> {code:java}
> // mapping from partition paths to file groups contained within them
> protected Map<String, List<HoodieFileGroup>> partitionToFileGroupsMap;
> /**
>  * PartitionPath + File-Id to pending compaction instant time.
>  */
> protected Map<HoodieFileGroupId, Pair<String, CompactionOperation>> fgIdToPendingCompaction;
> /**
>  * PartitionPath + File-Id to pending compaction instant time.
>  */
> protected Map<HoodieFileGroupId, Pair<String, CompactionOperation>> fgIdToPendingLogCompaction;
> /**
>  * PartitionPath + File-Id to bootstrap base File (Index Only bootstrapped).
>  */
> protected Map<HoodieFileGroupId, BootstrapBaseFileMapping> fgIdToBootstrapBaseFile;
> /**
>  * Track replace time for replaced file groups.
>  */
> protected Map<HoodieFileGroupId, HoodieInstant> fgIdToReplaceInstants;
> /**
>  * Track file groups in pending clustering.
>  */
> protected Map<HoodieFileGroupId, HoodieInstant> fgIdToPendingClustering; {code}
> The maps should be ConcurrentMaps, because addFilesToView in AbstractTableFileSystem can be called on multiple partitions concurrently. 
> SpillableMapBasedFileSystemView uses ExternalSpillableMap which is not thread safe, so we cannot update HoodieTableFileSystemView because then SpillableMapBasedFileSystemView would not implement it correctly



--
This message was sent by Atlassian Jira
(v8.20.10#820010)