You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hop.apache.org by "bamaer (via GitHub)" <gi...@apache.org> on 2026/10/09 14:33:28 UTC

[PR] Fixes #8815 : Lint: resolve every field offered for connection, trans… (hop)

bamaer opened a new pull request, #8825:
URL: https://github.com/apache/hop/pull/8825

   …form and action rules
   
   **Please** add a meaningful description for your change here
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   - [x] Run `mvn clean install apache-rat:check` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
   - [x] If you have a group of commits related to the same change, please squash your commits into one and force push your branch using `git rebase -i`.
   - [x] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable.
   
   To make clear that you license your contribution under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
   you have to acknowledge this by using the following check-box.
   
   - [x] I hereby declare this contribution to be licensed under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
   - [ ] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fixes #8815 : Lint: resolve every field offered for connection, trans… (hop)

Posted by "mattcasters (via GitHub)" <gi...@apache.org>.
mattcasters commented on code in PR #8825:
URL: https://github.com/apache/hop/pull/8825#discussion_r4231976231


##########
plugins/misc/lint/src/main/java/org/apache/hop/lint/CustomRuleExecutor.java:
##########
@@ -433,9 +441,18 @@ private static Object extractFieldValue(Object hopObject, String fieldName, Cust
             return dbMeta.getUsername();
           case "password":
             return dbMeta.getPassword();
+          case "servername":
+            return dbMeta.getServername();
+          case "dataTablespace":
+            return dbMeta.getDataTablespace();
+          case "indexTablespace":
+            return dbMeta.getIndexTablespace();
+          case "attributes":
+            // The extra options are attributes too, stored as EXTRA_OPTION_<type>.<option>.
+            return dbMeta.getAttributes();
           default:
-            log.logDetailed("Unknown database field: " + fieldName);
-            return null;
+            // Any other connection property, such as manualUrl, by getter or field.
+            return extractFieldFromObject(dbMeta, fieldName);

Review Comment:
   **[bug]** The new default calls `extractFieldFromObject`, so a rule on `sshTunnelPassphrase` invokes `DatabaseMeta.getSshTunnelPassphrase()`. That property is `@HopMetadataProperty(password = true)` on the nested database interface, but `storedAsPassword` only looks for a field of that name on `DatabaseMeta` itself, and `holdsASecret` only matches names that end with `password`, `secret`, `token`, and the other existing suffixes. `passphrase` matches none of them, so `generateErrorMessage` appends the stored secret as `(current value: ...)`. `sshTunnelPassword` stays hidden only because of the `password` suffix. Attribute maps are redacted; this string is not. A `NO_HARDCODED` rule, which is the rule someone would write for this field, is exactly the case that prints it. That is the CI-log leak DB-001 was closed to stop, on a getter this fallback newly exposes.
   
   **Suggestion:** When the object is a `DatabaseMeta`, also resolve `password = true` on `getIDatabase()`. Treat a name ending in `passphrase` as a secret. Add a test that a finding for `sshTunnelPassphrase` does not contain the value.



##########
plugins/misc/lint/src/main/java/org/apache/hop/lint/CustomRuleExecutor.java:
##########
@@ -586,6 +619,26 @@ private static Object extractFieldFromAction(ActionMeta actionMeta, String field
           return actionMeta.getDescription();
         case "pluginId":
           return actionMeta.getAction().getPluginId();
+        case "errorHandling":
+          // An action handles its errors with a hop followed on failure.
+          return SUBJECT.get() instanceof WorkflowMeta workflow
+              ? workflow.getWorkflowHops().stream()
+                  .anyMatch(
+                      hop ->
+                          hop.isEnabled()
+                              && actionMeta.equals(hop.getFromAction())
+                              && !hop.isUnconditional()
+                              && !hop.isEvaluation())
+              : null;
+        case "targetActions":
+          return SUBJECT.get() instanceof WorkflowMeta workflow
+              ? workflow.getWorkflowHops().stream()
+                  .filter(hop -> hop.isEnabled() && actionMeta.equals(hop.getFromAction()))
+                  .map(hop -> hop.getToAction().getName())

Review Comment:
   **[bug]** `hop.getToAction().getName()` throws when the hop's target was not resolved. `WorkflowHopMeta` still keeps that hop, with `to == null`. The same try also calls `actionMeta.equals(hop.getFromAction())` for `errorHandling` and for this filter. `ActionMeta.equals` returns false for a null argument, but it dereferences `action.getName()` on both sides when the argument is non-null, so a from-action whose inner `IAction` is null throws too. The catch returns null for the entire `targetActions` list and for `errorHandling`. A `NOT_EMPTY_COLLECTION` or `MUST_BE_TRUE` rule then passes, including for the hops that were fine.
   
   **Suggestion:** Keep a hop only when both ends are non-null, and compare those ends without going through `ActionMeta.equals` unless the inner action is present. Build the list from the hops that resolved instead of failing the whole field.



##########
plugins/misc/lint/src/main/java/org/apache/hop/lint/CustomRuleExecutor.java:
##########
@@ -548,6 +571,16 @@ private static Object extractFieldFromTransform(
         case "copies":
           return transformMeta.getCopies(
               org.apache.hop.core.variables.Variables.getADefaultVariableSpace());
+        case "distributes":
+          return transformMeta.isDistributes();
+        case "errorHandling":
+          return transformMeta.getTransform() != null && transformMeta.isDoingErrorHandling();
+        case "targetTransforms":
+          return SUBJECT.get() instanceof PipelineMeta pipeline
+              ? pipeline.findNextTransforms(transformMeta).stream()
+                  .map(TransformMeta::getName)

Review Comment:
   **[bug]** `PipelineMeta.findNextTransforms` puts a hop's `to` transform in the list even when that reference is null. A saved hop whose target name no longer matches a transform is loaded that way (`PipelineHopMeta` keeps the hop). `TransformMeta::getName` then throws. A null `from` throws inside `findNextTransforms` itself (`getFromTransform().equals(...)`). Either exception is caught below, logged at detailed level, and turned into null for the whole field. Null is not `FIELD_NOT_FOUND`: `NOT_EMPTY_COLLECTION` and `MAX_COLLECTION_SIZE` return false, so the rule passes and every other target on that transform is dropped. One broken hop blanks `targetTransforms` for every transform, because `findNextTransforms` walks every hop before it returns.
   
   **Suggestion:** Do not call `findNextTransforms` unchecked. Walk enabled hops, skip those with a null from or to, and map only the names that resolved. An extraction failure should not become a null that the presence checks treat as success.



##########
plugins/misc/lint/src/main/java/org/apache/hop/lint/CustomRuleExecutor.java:
##########
@@ -433,9 +441,18 @@ private static Object extractFieldValue(Object hopObject, String fieldName, Cust
             return dbMeta.getUsername();
           case "password":
             return dbMeta.getPassword();
+          case "servername":
+            return dbMeta.getServername();
+          case "dataTablespace":
+            return dbMeta.getDataTablespace();
+          case "indexTablespace":
+            return dbMeta.getIndexTablespace();
+          case "attributes":
+            // The extra options are attributes too, stored as EXTRA_OPTION_<type>.<option>.
+            return dbMeta.getAttributes();
           default:
-            log.logDetailed("Unknown database field: " + fieldName);
-            return null;
+            // Any other connection property, such as manualUrl, by getter or field.

Review Comment:
   **[suggestion]** `extractFieldFromObject` tries `getX` and a method whose name is the field, not `isX`. Connection flags are `isSshTunnelEnabled`, `isQuoteAllFields`, `isStreamingResults`, and so on, so they come back as `FIELD_NOT_FOUND`. This change reports every unresolved connection field as an error, so a rule on `sshTunnelEnabled` says the field does not exist on every connection, even though the getter is there. `manualUrl` works only because it is `getManualUrl`.
   
   **Suggestion:** Before returning `FIELD_NOT_FOUND`, also try `is` plus the capitalized name. Otherwise the new "any other connection property" path and the "missing connection field is always an error" path disagree for booleans.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fixes #8815 : Lint: resolve every field offered for connection, trans… (hop)

Posted by "mattcasters (via GitHub)" <gi...@apache.org>.
mattcasters merged PR #8825:
URL: https://github.com/apache/hop/pull/8825


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fixes #8815 : Lint: resolve every field offered for connection, trans… (hop)

Posted by "bamaer (via GitHub)" <gi...@apache.org>.
bamaer commented on PR #8825:
URL: https://github.com/apache/hop/pull/8825#issuecomment-6086418790

   Thanks for the review, @mattcasters. All four points were valid; they are fixed in the latest commit.
   
   - **`sshTunnelPassphrase`**: when a rule reads a connection field, the password check now also looks at the database plugin's own settings (`getIDatabase()`), and names ending in `passphrase` count as secrets. A test checks that the value stays out of the finding.
   - **Broken pipeline hops**: `targetTransforms` no longer uses `findNextTransforms`. It walks the enabled hops itself and skips any hop whose `from` or `to` is null, so the valid targets still count.
   - **Broken workflow hops**: `targetActions` and action `errorHandling` skip hops with a null end and compare actions by identity instead of `ActionMeta.equals`.
   - **`isX` getters**: the fallback now also tries `is` + the capitalised name, so `sshTunnelEnabled` and other connection flags resolve.
   
   One more fix in the same area: settings that only one database type has, such as Oracle's `walletPassword`, are now read from the database plugin's settings before `DatabaseMeta`. They were reported as missing before, and fields marked `password = true` there stay hidden.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org