You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@hop.apache.org by "rgus57674 (via GitHub)" <gi...@apache.org> on 2025/02/11 04:25:34 UTC
[D] Need help running embedded Hop pipelines with Springboot (hop)
GitHub user rgus57674 created a discussion: Need help running embedded Hop pipelines with Springboot
We have a large number of Pentaho transformations that are currently deployed using Springboot executable jar files. We plan to convert the existing Pentaho transformations to Hop pipelines and have similar Springboot deployments.
When I try to embed Hop pipelines in a Springboot application and run a pipeline using the Springboot executable jar file uisng the spring-boot-maven-plugin, I get the following error message (also attached the logs).
org.apache.hop.core.exception.HopException:
Unable to find plugin with ID 'Hop'
at org.apache.hop.core.encryption.Encr.init(Encr.java:47)
at org.apache.hop.core.HopClientEnvironment.init(HopClientEnvironment.java:120)
at org.apache.hop.core.HopClientEnvironment.init(HopClientEnvironment.java:76)
at org.apache.hop.core.HopEnvironment.init(HopEnvironment.java:114)
I, however, am able to run the same code in an IDE or using mvn spring-boot:run. It looks like a class loading issue that I am not able to figure out and I am sure someone might be able to provide some guidance. I am attaching the example I used (JDK 17, Hop 2.9.0).
I really appreciate your help.
Regrads
RG
Here is the sample code used - https://github.com/rgus57674/samplehopspringboot
Logs:
[Sprintbootjar_error.txt](https://github.com/user-attachments/files/18745185/Sprintbootjar_error.txt)
[mvn-spring-boot-run-success.txt](https://github.com/user-attachments/files/18745196/mvn-spring-boot-run-success.txt)
GitHub link: https://github.com/apache/hop/discussions/4887
----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org
Re: [D] Need help running embedded Hop pipelines with Springboot (hop)
Posted by "rgus57674 (via GitHub)" <gi...@apache.org>.
GitHub user rgus57674 added a comment to the discussion: Need help running embedded Hop pipelines with Springboot
Included the following Jandex dependency in the pom.xml and it results in same errors.
Again, it works from an IDE or when running using 'mvn spring-boot:run'
<dependency>
<groupId>org.jboss</groupId>
<artifactId>jandex</artifactId>
<version>2.4.5.Final</version>
</dependency>
GitHub link: https://github.com/apache/hop/discussions/4887#discussioncomment-12219058
----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org
Re: [D] Need help running embedded Hop pipelines with Springboot (hop)
Posted by "dvdvardanyan (via GitHub)" <gi...@apache.org>.
GitHub user dvdvardanyan deleted a comment on the discussion: Need help running embedded Hop pipelines with Springboot
For the error "Hop configuration file not found, not serializing: /Users/User/Desktop/SampleHopApp/config/hop-config.json" there is a "hop-config.json" file that comes with the hop binaries download zip, located under "apache-hop-client-2.11.0.zip/hop/config/hop-config.json". It should be copied into the Java project under "samplehopspringboot/config/hop-config.json" or whichever path the error message mentions. Some very limited details about it can be found here https://hop.apache.org/manual/latest/projects/advanced.html.
Also, inside said JSON configuration should be changed under projectsConfig/projectConfigurations path to include the projects that are executed from the Java project.
For "Error loading the pipeline run configuration 'local'", try explicitly creating a new "Pipeline Run Configuration" and using it in every pipeline instead of the "local", as "local" does not have configuration in project metadata.
However, I wasn't able to find any maven repository that would import all the plugins from https://github.com/apache/hop/tree/main/plugins required to run the workflows or pipelines (which, for some reason are not included in hop-core and hop-engine repositories), and need to be imported separately like: "hop-action-pipeline", "hop-transform-sql", "hop-transform-pipelineexecutor", etc, and if any is missed - will get the missing plugin error without properly describing which exact plugin is missing and will have to guess based on the exception which is not always obvious.
So, did I miss somewhere any maven repository which would import all the necessary plugins?
Whatever is described here https://hop.apache.org/dev-manual/latest/porting-kettle-plugins.html doesn't seem to be solving the problem, and I'm not sure if this page is even up to date.
GitHub link: https://github.com/apache/hop/discussions/4887#discussioncomment-12243330
----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org
Re: [D] Need help running embedded Hop pipelines with Springboot (hop)
Posted by "rgus57674 (via GitHub)" <gi...@apache.org>.
GitHub user rgus57674 edited a discussion: Need help running embedded Hop pipelines with Springboot
We have a large number of Pentaho transformations that are currently deployed using Springboot executable jar files. We plan to convert the existing Pentaho transformations to Hop pipelines and have similar Springboot deployments.
When I try to embed Hop pipelines in a Springboot application and run a pipeline using the Springboot executable jar file uisng the spring-boot-maven-plugin, I get the following error message (also attached the logs).
**_org.apache.hop.core.exception.HopException:
Unable to find plugin with ID 'Hop'
at org.apache.hop.core.encryption.Encr.init(Encr.java:47)
at org.apache.hop.core.HopClientEnvironment.init(HopClientEnvironment.java:120)
at org.apache.hop.core.HopClientEnvironment.init(HopClientEnvironment.java:76)
at org.apache.hop.core.HopEnvironment.init(HopEnvironment.java:114)_**
If I create the jar file using maven-shade-plugin, I get the following error.
_**Hop configuration file not found, not serializing: /Users/User/Desktop/SampleHopApp/config/hop-
config.json
org.apache.hop.core.exception.HopException:
Error loading the pipeline run configuration 'local'
Error loading metadata object 'local' from file '/Users/User/Desktop/hop-apache/hop-
2.9.0/config/projects/default/metadata/pipeline-run-configuration/local.json'**_
I, however, am able to run the same code in an IDE or using mvn spring-boot:run. It looks like a class loading issue that I am not able to figure out and I am sure someone might be able to provide some guidance. I am attaching the example I used (JDK 17, Hop 2.9.0).
I really appreciate your help.
Regrads
RG
Here is the sample code used - https://github.com/rgus57674/samplehopspringboot
Logs:
[Sprintbootjar_error.txt](https://github.com/user-attachments/files/18745185/Sprintbootjar_error.txt)
[mvn-spring-boot-run-success.txt](https://github.com/user-attachments/files/18745196/mvn-spring-boot-run-success.txt)
[maven-shade-plugin-error.txt](https://github.com/user-attachments/files/18745252/maven-shade-plugin-error.txt)
GitHub link: https://github.com/apache/hop/discussions/4887
----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org
Re: [D] Need help running embedded Hop pipelines with Springboot (hop)
Posted by "rgus57674 (via GitHub)" <gi...@apache.org>.
GitHub user rgus57674 edited a discussion: Need help running embedded Hop pipelines with Springboot
We have a large number of Pentaho transformations that are currently deployed using Springboot executable jar files. We plan to convert the existing Pentaho transformations to Hop pipelines and have similar Springboot deployments.
When I try to embed Hop pipelines in a Springboot application and run a pipeline using the Springboot executable jar file uisng the spring-boot-maven-plugin, I get the following error message (also attached the logs).
**_org.apache.hop.core.exception.HopException:
Unable to find plugin with ID 'Hop'
at org.apache.hop.core.encryption.Encr.init(Encr.java:47)
at org.apache.hop.core.HopClientEnvironment.init(HopClientEnvironment.java:120)
at org.apache.hop.core.HopClientEnvironment.init(HopClientEnvironment.java:76)
at org.apache.hop.core.HopEnvironment.init(HopEnvironment.java:114)_**
If I create the jar file using maven-shade-plugin, I get the following error.
_**Hop configuration file not found, not serializing: /Users/User/Desktop/SampleHopApp/config/hop-
config.json
org.apache.hop.core.exception.HopException:
Error loading the pipeline run configuration 'local'
Error loading metadata object 'local' from file '/Users/User/Desktop/hop-apache/hop-
2.9.0/config/projects/default/metadata/pipeline-run-configuration/local.json'**_
I, however, am able to run the same code in an IDE or using mvn spring-boot:run. It looks like a class loading issue that I am not able to figure out and I am sure someone might be able to provide some guidance. I am attaching the example I used (JDK 17, Hop 2.9.0).
I really appreciate your help.
Regrads
RG
Here is the sample code used - https://github.com/rgus57674/samplehopspringboot
Logs:
[Sprintbootjar_error.txt](https://github.com/user-attachments/files/18745185/Sprintbootjar_error.txt)
[mvn-spring-boot-run-success.txt](https://github.com/user-attachments/files/18745196/mvn-spring-boot-run-success.txt)
[maven-shade-plugin-error.txt](https://github.com/user-attachments/files/18745252/maven-shade-plugin-error.txt)
GitHub link: https://github.com/apache/hop/discussions/4887
----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org
Re: [D] Need help running embedded Hop pipelines with Springboot (hop)
Posted by "bamaer (via GitHub)" <gi...@apache.org>.
GitHub user bamaer added a comment to the discussion: Need help running embedded Hop pipelines with Springboot
Check the [SDK docs](https://hop.apache.org/dev-manual/latest/sdk/hop-sdk.html), there could be other pages in the developer docs that could help you to get this sorted out. Also, make sure you have all the required dependencies, load your project correctly etc.
GitHub link: https://github.com/apache/hop/discussions/4887#discussioncomment-12135945
----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org
Re: [D] Need help running embedded Hop pipelines with Springboot (hop)
Posted by "rgus57674 (via GitHub)" <gi...@apache.org>.
GitHub user rgus57674 edited a discussion: Need help running embedded Hop pipelines with Springboot
We have a large number of Pentaho transformations that are currently deployed using Springboot executable jar files. We plan to convert the existing Pentaho transformations to Hop pipelines and have similar Springboot deployments.
When I try to embed Hop pipelines in a Springboot application and run a pipeline using the Springboot executable jar file uisng the spring-boot-maven-plugin, I get the following error message (also attached the logs).
**_org.apache.hop.core.exception.HopException:
Unable to find plugin with ID 'Hop'
at org.apache.hop.core.encryption.Encr.init(Encr.java:47)
at org.apache.hop.core.HopClientEnvironment.init(HopClientEnvironment.java:120)
at org.apache.hop.core.HopClientEnvironment.init(HopClientEnvironment.java:76)
at org.apache.hop.core.HopEnvironment.init(HopEnvironment.java:114)_**
If I create the jar file using maven-shade-plugin, I get the following error.
_**Hop configuration file not found, not serializing: /Users/User/Desktop/SampleHopApp/config/hop-
config.json
org.apache.hop.core.exception.HopException:
Error loading the pipeline run configuration 'local'
Error loading metadata object 'local' from file '/Users/User/Desktop/hop-apache/hop-
2.9.0/config/projects/default/metadata/pipeline-run-configuration/local.json'**_
I, however, am able to run the same code in an IDE or using mvn spring-boot:run. It looks like a class loading issue that I am not able to figure out and I am sure someone might be able to provide some guidance. I am attaching the example I used (JDK 17, Hop 2.9.0).
I really appreciate your help.
Regrads
RG
Here is the sample code used - https://github.com/rgus57674/samplehopspringboot
Logs:
[Sprintbootjar_error.txt](https://github.com/user-attachments/files/18745185/Sprintbootjar_error.txt)
[mvn-spring-boot-run-success.txt](https://github.com/user-attachments/files/18745196/mvn-spring-boot-run-success.txt)
[maven-shade-plugin-error.txt](https://github.com/user-attachments/files/18745252/maven-shade-plugin-error.txt)
GitHub link: https://github.com/apache/hop/discussions/4887
----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org
Re: [D] Need help running embedded Hop pipelines with Springboot (hop)
Posted by "nadment (via GitHub)" <gi...@apache.org>.
GitHub user nadment added a comment to the discussion: Need help running embedded Hop pipelines with Springboot
Maybe search about Jandex if it doesn't find the plugins. Jandex is a space efficient Java class file indexer that Hop use to discover annotations plugin.
GitHub link: https://github.com/apache/hop/discussions/4887#discussioncomment-12147503
----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org
Re: [D] Need help running embedded Hop pipelines with Springboot (hop)
Posted by "dvdvardanyan (via GitHub)" <gi...@apache.org>.
GitHub user dvdvardanyan added a comment to the discussion: Need help running embedded Hop pipelines with Springboot
For the error "Hop configuration file not found, not serializing: /Users/User/Desktop/SampleHopApp/config/hop-config.json" there is a "hop-config.json" file that comes with the hop binaries download zip, located under "apache-hop-client-2.11.0.zip/hop/config/hop-config.json". It should be copied into the Java project under "samplehopspringboot/config/hop-config.json" or whichever path the error message mentions. Some very limited details about it can be found here https://hop.apache.org/manual/latest/projects/advanced.html.
Also, inside said JSON configuration should be changed under projectsConfig/projectConfigurations path to include the projects that are executed from the Java project.
For "Error loading the pipeline run configuration 'local'", try explicitly creating a new "Pipeline Run Configuration" and using it in every pipeline instead of the "local", as "local" does not have configuration in project metadata.
However, I wasn't able to find any maven repository that would import all the plugins from https://github.com/apache/hop/tree/main/plugins required to run the workflows or pipelines (which, for some reason are not included in hop-core and hop-engine repositories), and need to be imported separately like: "hop-action-pipeline", "hop-transform-sql", "hop-transform-pipelineexecutor", etc, and if any is missed - will get the missing plugin error without properly describing which exact plugin is missing and will have to guess based on the exception which is not always obvious.
So, did I miss somewhere any maven repository which would import all the necessary plugins?
Whatever is described here https://hop.apache.org/dev-manual/latest/porting-kettle-plugins.html doesn't seem to be solving the problem, and I'm not sure if this page is even up to date.
GitHub link: https://github.com/apache/hop/discussions/4887#discussioncomment-12243330
----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org