[jira] [Updated] (ATLAS-5205) Improve bootstrap performance

["Mandar Ambawane (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ATLAS-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mandar Ambawane updated ATLAS-5205:
-----------------------------------
    Attachment: ATLAS-5205-0615.patch

> Improve bootstrap performance
> -----------------------------
>
>                 Key: ATLAS-5205
>                 URL: https://issues.apache.org/jira/browse/ATLAS-5205
>             Project: Atlas
>          Issue Type: Improvement
>          Components:  atlas-core
>            Reporter: Madhan Neethiraj
>            Assignee: Mandar Ambawane
>            Priority: Major
>         Attachments: ATLAS-5205-0615.patch
>
>
> On the first startup, Atlas initializes its database with following:
>  * load all typedefs defined in files under \{ATLAS_HOME}/model directory
>  * create indices in backend and index store
>  * apply type patches found under \{ATLAS_HOME}/model directory
>  
> This process can take around 3 minutes, as following logs show:
> {noformat}
> 2026-02-06 20:43:49,980 [main] INFO [GraphBackedSearchIndexer.java:515] Creating indexes for graph.
> 2026-02-06 20:43:51,062 [main] INFO [GraphBackedSearchIndexer.java:522] Created index : vertex_index
> 2026-02-06 20:43:51,071 [main] INFO [GraphBackedSearchIndexer.java:528] Created index : edge_index
> 2026-02-06 20:43:51,077 [main] INFO [GraphBackedSearchIndexer.java:534] Created index : fulltext_index
> 2026-02-06 20:43:51,089 [main] INFO [AtlasJanusGraphManagement.java:321] property '__guid' is encoded to '35x_t'.
> 2026-02-06 20:43:51,089 [main] INFO [GraphBackedSearchIndexer.java:450] Created backing index for vertex property __guid of type java.lang.String
> ...
> 2026-02-06 20:43:52,815 [main] INFO [AtlasTypeDefStoreInitializer.java:314] ==> AtlasTypeDefStoreInitializer.loadBootstrapTypeDefs()
> ...
> 2026-02-06 20:43:52,825 [main] INFO [AtlasTypeDefStoreInitializer.java:353] ==> AtlasTypeDefStoreInitializer(/opt/apache-atlas-3.0.0-SNAPSHOT/models/0000-Area0)
> ...
> 2026-02-06 20:44:03,973 [main] INFO [AtlasTypeDefStoreInitializer.java:382] Created/Updated types defined in file /opt/apache-atlas-3.0.0-SNAPSHOT/models/0000-Area0/0010-base_model.json
> ...
> 2026-02-06 20:44:08,952 [main] INFO [AtlasTypeDefStoreInitializer.java:382] Created/Updated types defined in file /opt/apache-atlas-3.0.0-SNAPSHOT/models/0000-Area0/0011-glossary_model.json
> ...
> 2026-02-06 20:44:22,963 [main] INFO [AtlasTypeDefStoreInitializer.java:382] Created/Updated types defined in file /opt/apache-atlas-3.0.0-SNAPSHOT/models/1000-Hadoop/1020-fs_model.json
> ...
> 2026-02-06 20:45:50,612 [main] INFO [AtlasTypeDefStoreInitializer.java:382] Created/Updated types defined in file /opt/apache-atlas-3.0.0-SNAPSHOT/models/6000-Trino/6000-trino_model.json
> 2026-02-06 20:45:50,612 [main] INFO [AtlasTypeDefStoreInitializer.java:394] <== AtlasTypeDefStoreInitializer(/opt/apache-atlas-3.0.0-SNAPSHOT/models/6000-Trino)
> 2026-02-06 20:45:50,612 [main] INFO [AtlasTypeDefStoreInitializer.java:353] ==> AtlasTypeDefStoreInitializer(/opt/apache-atlas-3.0.0-SNAPSHOT/models)
> 2026-02-06 20:45:50,613 [main] INFO [AtlasTypeDefStoreInitializer.java:345] <== AtlasTypeDefStoreInitializer.loadBootstrapTypeDefs()
> ...
> 2026-02-06 20:45:53,621 [main] INFO [Services.java:70] Starting service org.apache.atlas.repository.patches.AtlasPatchService
> 2026-02-06 20:45:53,621 [main] INFO [AtlasPatchService.java:49] ==> AtlasPatchService.start()
> 2026-02-06 20:45:53,622 [main] INFO [AtlasPatchService.java:86] AtlasPatchService: applying patches...
> 2026-02-06 20:45:53,622 [main] INFO [AtlasPatchManager.java:64] ==> AtlasPatchManager.applyAll()
> 2026-02-06 20:45:53,622 [main] INFO [AtlasPatchManager.java:99] ==> AtlasPatchManager.init()
> 2026-02-06 20:45:53,622 [main] INFO [AtlasPatchRegistry.java:69] AtlasPatchRegistry: initializing..
> 2026-02-06 20:45:53,732 [main] INFO [AtlasPatchRegistry.java:74] AtlasPatchRegistry: found 237 patches
> 2026-02-06 20:45:53,732 [main] INFO [AtlasPatchRegistry.java:77] AtlasPatchRegistry: patchId=TYPEDEF_PATCH_0000_026, status=APPLIED
> ...
> 2026-02-06 20:45:55,101 [main] INFO [AtlasPatchManager.java:116] <== AtlasPatchManager.init()
> 2026-02-06 20:45:55,101 [main] INFO [AtlasPatchManager.java:75] Applying java handler: JAVA_PATCH_0000_001; status: UNKNOWN
> 2026-02-06 20:45:55,101 [main] INFO [ConcurrentPatchProcessor.java:217] ConcurrentPatchProcessor: atlas.patch.numWorkers=3, atlas.patch.batchSize=300
> 2026-02-06 20:45:55,101 [main] INFO [UniqueAttributePatch.java:166] Unique attributes: type: trino_table_ddl: Registered!
> ...
> 2026-02-06 20:46:05,452 [main] INFO [AtlasPatchManager.java:87] <== AtlasPatchManager.applyAll(){noformat}
> The goal is to minimize the time taken to bootstrap as much as possible, like by exploring multu-threaded loading of typedef models, eliminating patches while initializing database from clean state.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)