[I] MetadataPerspective: full tree reload on editor dirty flag freezes GUI on large projects (hop)

["mattcasters (via GitHub)" <gi...@apache.org>]

mattcasters opened a new issue, #7791:
URL: https://github.com/apache/hop/issues/7791

   ## Description
   
   When any metadata editor is first marked dirty (`MetadataEditor.setChanged()`), Hop performs a **full metadata perspective reload** (and fires `HopGuiEvents.MetadataChanged`). On large projects this freezes the UI for a noticeable time—even for trivial edits such as typing the first character of a metadata **name** field.
   
   This is especially painful when:
   
   - The project has many metadata objects (every object is fully `load()`ed during refresh)
   - Plugin listeners also react to `MetadataChanged` (e.g. rebuilding large secondary trees)
   
   Observed while working with large [hop-data-vault](https://github.com/mattcasters/hop-data-vault) resource definition groups (~hundreds of model paths), but the root cause is in Hop GUI core and affects **all** metadata types.
   
   ## Call chain
   
   ```
   MetadataEditor.setChanged()   // first dirty only
     → MetadataPerspective.updateEditor(this)
          → refresh()
               → reloadModel()   // list + load EVERY metadata object
               → renderTree()
     → fire HopGuiEvents.MetadataChanged
          → MetadataPerspective listener → refresh() again
          → any other MetadataChanged listeners (plugins)
   ```
   
   ### Relevant code (Hop UI)
   
   1. **`MetadataEditor.setChanged()`** — on first dirty, calls `updateEditor` and fires `MetadataChanged`:
   
   ```java
   public void setChanged() {
     if (!this.isChanged) {
       this.isChanged = true;
       MetadataPerspective.getInstance().updateEditor(this);
       hopGui.getEventsHandler().fire(HopGuiEvents.MetadataChanged.name());
     }
   }
   ```
   
   2. **`MetadataPerspective.updateEditor()`** — always full-refreshes the tree just to bold the tab title:
   
   ```java
   public void updateEditor(MetadataEditor<?> editor) {
     // Update TabItem font/title...
     this.refresh();  // full reloadModel + renderTree
     this.updateGui();
   }
   ```
   
   3. **`MetadataPerspective.reloadModel()`** — for every metadata type, lists names then **fully deserializes each object** only to read `virtualPath`:
   
   ```java
   for (String name : names) {
     virtualPath = Const.NVL(serializer.load(name).getVirtualPath(), "");
     // ...
   }
   ```
   
   ## Expected behavior
   
   - Marking an editor dirty should only update lightweight UI (tab title, bold font, toolbar/file capabilities).
   - **Persisted** create/update/delete/rename should refresh the metadata tree (and fire change events).
   - Tree reload should not need to fully deserialize every metadata object just for the name list / virtual path.
   
   ## Suggested directions
   
   1. **`updateEditor`**: update the open tab decoration only; do **not** call full `refresh()` / `reloadModel()`.
   2. **`setChanged`**: do **not** fire `MetadataChanged` on dirty. Fire that (or a more specific event) on successful **save / delete / create / rename** only. Dirty ≠ persisted change.
   3. **`reloadModel`**: avoid `serializer.load(name)` for every object when only name + virtual path are needed—e.g. list names only, lazy-load virtual path when expanding a type, or a cheap name→path index/cache.
   
   ## Impact
   
   - Large projects: multi-second freeze on first keystroke in any metadata editor name field.
   - Double refresh: `updateEditor` + `MetadataChanged` listener both call `refresh()`.
   - Downstream plugins that listen to `MetadataChanged` incorrectly treat “user typed a character” as “metadata store changed,” which can trigger expensive work (catalog reconnects, tree rebuilds, etc.).
   
   ## Environment
   
   - Hop 2.19.0-SNAPSHOT / current `main` line
   - Linux, Hop GUI metadata perspective
   
   ## Related
   
   Plugin-side mitigation (stop reacting to dirty-as-change) is being done in hop-data-vault; core fix still needed so all metadata editors stay responsive.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] MetadataPerspective: full tree reload on editor dirty flag freezes GUI on large projects (hop)

["hansva (via GitHub)" <gi...@apache.org>]

hansva closed issue #7791: MetadataPerspective: full tree reload on editor dirty flag freezes GUI on large projects
URL: https://github.com/apache/hop/issues/7791


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org