[I] [Feature Request]: Operating-system keystore plugin for Hop secrets (hop)

["mattcasters (via GitHub)" <gi...@apache.org>]

mattcasters opened a new issue, #9099:
URL: https://github.com/apache/hop/issues/9099

   ### What would you like to happen?
   
   Add a keystore plugin that keeps one master key in the operating-system secret store and exposes named secrets through a variable resolver. Logging into the operating-system account that stored the key makes those secrets available to every Hop process running as that user.
   
   This gives Hop GUI, `hop-run`, and pipelines one place to resolve a secret, and it gives Git a credential that survives a restart. Today the token typed in **Add project → From version control** is used only for that clone (`UsernamePasswordCredentialsProvider` with username `x-oauth-basic`) and is then discarded. Push and pull start with no credentials and open a username/password dialog. The answer is kept in memory on that Git session until Hop exits. Nothing is written to a store Hop can read the next time.
   
   #### Architecture
   
   The operating-system store holds a single master key. Named secrets are AES2 ciphertext in a file under the Hop config folder. `hop-config.json` stays the registry of projects and environment files and does not hold the key or the secrets. Project files do not hold them either.
   
   ```mermaid
   flowchart TB
     subgraph osAccount [Operating-system account]
       Store["OS secret store<br/>Windows Credential Manager<br/>macOS Keychain<br/>Linux Secret Service"]
     end
   
     subgraph hopProcess [Hop process running as that account]
       Plugin[Keystore plugin]
       File["Named secrets<br/>HOP_CONFIG_FOLDER/keystore.json<br/>AES2 ciphertext"]
       Resolver["Variable resolver<br/>#{hop-keystore:name}"]
       Encoder[AES2 password encoder]
       Dialog[Configuration perspective]
       Git[Git clone, push and pull]
       Runtime[Pipelines, workflows, metadata]
     end
   
     Store -->|master key| Plugin
     Plugin --> File
     Plugin --> Encoder
     Dialog --> Plugin
     Resolver --> Plugin
     Git --> Resolver
     Runtime --> Resolver
     Runtime --> Encoder
   
     KeyVar["HOP_AES_ENCODER_KEY"] -->|wins| Encoder
     KeyFile["HOP_AES_ENCODER_KEY_FILE"] -->|wins| Encoder
   ```
   
   Master-key lookup, first hit wins:
   
   1. `HOP_AES_ENCODER_KEY`
   2. `HOP_AES_ENCODER_KEY_FILE`
   3. The operating-system store, service name `Apache Hop`
   
   `AesEncoderKeyUtil` already implements the first two. The plugin adds the third, so password fields and environment values that are already stored as `AES2` ciphertext unlock with the same login. The default `Encrypted ` encoder is obfuscation and is not a fallback for this store.
   
   ```mermaid
   sequenceDiagram
     participant User
     participant Git as Git clone or push
     participant Resolver as hop-keystore resolver
     participant File as keystore.json
     participant OS as OS secret store
   
     User->>Git: https://github.com/org/repo.git
     Git->>Resolver: #{hop-keystore:git/github.com}
     Resolver->>OS: read master key
     OS-->>Resolver: master key of the logged-in account
     Resolver->>File: read AES2 ciphertext for git/github.com
     File-->>Resolver: ciphertext
     Resolver-->>Git: token
     Git->>User: clone or push, no password dialog
   ```
   
   When the name is missing, Git keeps the token field and the username/password dialog it has today. A **Remember for this host** action writes `git/<host>` through the keystore dialog's save path.
   
   #### Requirements
   
   **Plugin and resolver**
   
   * New plugin, with a `@VariableResolverPlugin`. Expression: `#{hop-keystore:<name>}`, for example `#{hop-keystore:git/github.com}`.
   * Names are chosen by the user. `git/<host>` is the convention the Git plugin uses. It is not a special case inside the resolver.
   * A missing name, a locked store, or no store leaves the expression unresolved and logs one line. The line does not contain the secret or the master key. An empty string is not a successful resolution.
   * The same resolver works in Hop GUI, `hop-run`, Hop Server, and Hop Web. There is no separate Hop login that unlocks it.
   
   **Master key**
   
   * Generate a random key the first time Hop GUI needs one and the OS store is available. Store it under the service name `Apache Hop`.
   * Windows Credential Manager opens with the Windows login. No second prompt.
   * macOS may ask once whether this Hop binary may read the keychain item. After the user allows it, later launches in that account succeed.
   * Linux uses the secret service unlocked by a graphical login. An SSH session, a service account, or a container usually has no secret service. Resolution then depends on `HOP_AES_ENCODER_KEY` or `HOP_AES_ENCODER_KEY_FILE`, which is the path Hop Server, Hop Web, and containers already use.
   * The account that matters is the operating-system user that owns the Hop process. Another OS user on the same machine cannot read the key. A person signing into Hop Web in a browser does not unlock the store. The account that started the JVM does.
   * Any other process running as that same user can read the entry. That is the boundary of Credential Manager, Keychain, and the secret service.
   * Copying `hop-config.json` or a project to another machine does not copy the master key.
   
   **Named secrets**
   
   * Store them in `${HOP_CONFIG_FOLDER}/keystore.json`, readable and writable by the user only. Each entry has a name, an optional username, and an AES2 ciphertext. The username is not secret. The master key is not in this file.
   * Edit the list from a dialog in the configuration perspective: add, replace, and delete. The feature is not file-only.
   * The value field accepts a token. Saving encrypts it with the master key. The dialog never shows the stored ciphertext as something to edit by hand.
   
   **Git**
   
   * Clone, push, and pull look up `git/<host>` from the remote URL before asking for a password. The same entry is used at checkout, before any project or lifecycle environment exists.
   * Remembering a token from the clone dialog or the username/password dialog creates or replaces that entry.
   * The username stored on the entry is sent with the token. For GitHub the existing `x-oauth-basic` form remains valid when the username is empty.
   * The token is not written into the remote URL, `.git/config`, `project-config.json`, or `hop-config.json`.
   * A lifecycle environment can override the user entry for the enabled session, for a deploy token on a build agent. The environment file holds a reference such as `#{hop-keystore:...}` or `#{vault:...}`, or an `AES2` value. Checkout does not require an environment.
   
   **Existing secrets**
   
   * With no explicit AES key variable or key file, the AES2 encoder uses the OS master key. Database passwords and environment secrets already saved as `AES2` then open for the logged-in account.
   * A server that sets `HOP_AES_ENCODER_KEY_FILE` keeps using that file and does not need the OS store.
   
   #### Implementation notes
   
   * `java.security.KeyStore` is a file format such as PKCS12. It is not the operating-system store.
   * Hop already depends on JNA. Put a small Hop interface in front of the native calls so the backend can be replaced. [`com.github.javakeyring:java-keyring`](https://central.sonatype.com/artifact/com.github.javakeyring/java-keyring) 1.0.4 (BSD-3, August 2023) talks to Credential Manager, Keychain, and the secret service and is a candidate behind that interface. Microsoft `credential-secure-storage` was archived in March 2025 and should not be added.
   * Document the desktop first-run and the server key-file fallback in the user manual.
   
   ### Issue Priority
   
   Priority: 2
   
   ### Issue Component
   
   Component: Hop Gui
   Component: API
   Component: Hop Config
   Component: Hop Server
   Component: Hop Web
   Component: Documentation
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org