You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@hop.apache.org by ho...@comline.de on 2023/03/20 10:51:37 UTC
DB2 JDBC4
Hi,
I am new to HOP, but have been using PDI since about 2005.
We are trying to port our project to HOP 2.3.0 and are running into a problem with DB2 and JDBC 4. With the JDBC4 logic an select id as id_new from table returns id, but I still need it to return id_new. With other programs an "useJDBC4ColumnNameAndLabelSemantics=false" works. I have tried setting it under Options in the connection section, but it seems to be ignoring it. I still get id returned.
I hope somebody can help me with this.
Thanks in advance
Holger Hymmen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Gesch?ftsf?hrer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Re: own libraries and "User defined Java class"
Posted by Thad Guidry <th...@gmail.com>.
PRs and Task for updating documentation to make it easier to find where to
place custom jar libraries.
[Task]: Document folder paths where custom jar libraries should be placed ·
Issue #3369 · apache/hop (github.com)
<https://github.com/apache/hop/issues/3369>
We should document where custom jar libraries should be placed.
Ideally, for
Entire Hop installation - we should document on page
Installation-configuration
For the UDJC transform - we should document on page
docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/userdefinedjavaexpression.adoc
(I think?)
Thad
https://www.linkedin.com/in/thadguidry/
https://calendly.com/thadguidry/
On Thu, Nov 9, 2023 at 8:52 PM Bart Maertens <ba...@apache.org> wrote:
> Hi Holger,
>
> Adding your jars in lib/core in your hop installation folder will make
> those libraries available for your entire hop installation.
> Adding them to plugins/transforms/janino/lib will make them available in
> the User Defined Java Class transform.
>
> Regards,
> Bart
>
>
> On Thu, Nov 9, 2023 at 1:47 PM <ho...@comline.de> wrote:
>
>> Hi,
>>
>>
>>
>> where do I place my own libraries in the hop directory structure, so that
>> I can use them in the “User defined Java class” step? Or do I have to do
>> something else to be able to use them?
>>
>> Regards
>>
>> Holger Hymmen
>>
>>
>>
>> *Dipl.-Inf.** Holger Hymmen*
>> *Berater*
>> Healthcare
>>
>>
>>
>> <https://www.iqvia.com>Learn more about IQVIA
>>
>> Comline GmbH - an IQVIA Business
>> Hauert 8, 44227 Dortmund
>>
>> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>>
>> Geschäftsführer: Dr. Frank Wartenberg
>> Sitz der Gesellschaft: Dortmund
>> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>>
>>
>>
Re: own libraries and "User defined Java class"
Posted by Bart Maertens <ba...@apache.org>.
Hi Holger,
Adding your jars in lib/core in your hop installation folder will make
those libraries available for your entire hop installation.
Adding them to plugins/transforms/janino/lib will make them available in
the User Defined Java Class transform.
Regards,
Bart
On Thu, Nov 9, 2023 at 1:47 PM <ho...@comline.de> wrote:
> Hi,
>
>
>
> where do I place my own libraries in the hop directory structure, so that
> I can use them in the “User defined Java class” step? Or do I have to do
> something else to be able to use them?
>
> Regards
>
> Holger Hymmen
>
>
>
> *Dipl.-Inf.** Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> <https://www.iqvia.com>Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
>
Hop gui on Mac Sonoma
Posted by Diego Mainou <di...@bizcubed.com.au>.
Hi All,
Anyone experiencing issues issues on Mac Sonoma?
I upgraded last week and cant seem to get hop to render properly.
I've played with every Dark mode light mode change I could still no show.
Any feedback that could help me raise a meaningful ticket appreciated.
[ https://www.bizcubed.com.au/ | ] Diego Mainou
Product Manager
M. +61 415 152 091
E. [ mailto:diego.mainou@bizcubed.com.au | diego.mainou@bizcubed.com.au ]
[ https://www.bizcubed.com.au/ | www.bizcubed.com.au ] [ https://www.bizcubed.com.au/ | ]
AW: Error finding child executions for parent ID null
Posted by ho...@comline.de.
Hi Bart,
it’s been a few days, but I believe I may have used the default project and later copied it to another project.
I have just reimported everything into a new project, and I was asked when starting a workflow/pipeline if a “default” run configuration should be created.
The result is the following:
[cid:image005.png@01DA1646.6D14C350]
Regards
Holger
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Von: Bart Maertens <ba...@apache.org>
Gesendet: Montag, 13. November 2023 13:26
An: users@hop.apache.org
Betreff: Re: Error finding child executions for parent ID null
Hi Holger,
Any chance you imported your PDI project into the default or samples hop project?
The "local-audit" and "first-last" execution information and data profiles are not part of the importer, but are defined in the default project.
Regards,
Bart
On Mon, Nov 13, 2023 at 12:28 PM <ho...@comline.de>> wrote:
Hi Bart,
Thank you. That has done the job.
I don’t know if my configuration stems from the port, but it contained the following:
[cid:image001.png@01DA1644.E8DFA8D0]
Regards
Holger
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image002.jpg@01DA1644.E8DFA8D0]
Learn more about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Von: Bart Maertens <ba...@apache.org>>
Gesendet: Montag, 13. November 2023 11:30
An: users@hop.apache.org<ma...@hop.apache.org>
Betreff: Re: Error finding child executions for parent ID null
Hi Holger,
The last part in your error message [1] seems to point to an Execution Information Location.
Even though execution information could provide useful information, let's see if you can get your workflows and pipelines to work as close as possible to the PDI environment.
Can you verify if your pipeline run configuration (metadata perspective) doesn't contain anything for "Execution information location" and "Execution data profile"?
If there are any values there, can you remove those, save your pipeline run configuration and try again (similar for workflow run configuration).
[cid:image003.png@01DA1644.E8DFA8D0]
[1] Caused by: java.lang.NullPointerException: Cannot invoke "String.equals(Object)" because "parentExecutionId" is null
at org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:476)
Regards,
Bart
On Mon, Nov 13, 2023 at 11:07 AM <ho...@comline.de>> wrote:
Hi,
I ported my workflows and pipelines from Pentaho Data Integration 7.1 to Apache HOP 2.6.
When I start my biggest and most complex workflow, it runs fine until it enters a pipeline in one of the sub workflows. It logs 36 times that it has connected to databases and then the log stops for quite a while. That are not all of the needed connections, but it doesn’t log about more and after some time it opens the following error window:
[cid:image004.png@01DA1644.E8DFA8D0]
The details are:
org.apache.hop.core.exception.HopException:
Error finding child executions for parent ID null
Cannot invoke "String.equals(Object)" because "parentExecutionId" is null
at org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:482)
at org.apache.hop.ui.hopgui.file.workflow.HopGuiWorkflowGraph.viewActionExecutionInfo(HopGuiWorkflowGraph.java:4204)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.apache.hop.core.gui.plugin.action.GuiActionLambdaBuilder.lambda$createLambda$0(GuiActionLambdaBuilder.java:96)
at org.apache.hop.ui.hopgui.context.GuiContextUtil.lambda$handleActionSelection$0(GuiContextUtil.java:183)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4029)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3645)
at org.apache.hop.ui.hopgui.HopGui.open(HopGui.java:473)
at org.apache.hop.ui.hopgui.HopGui.main(HopGui.java:351)
Caused by: java.lang.NullPointerException: Cannot invoke "String.equals(Object)" because "parentExecutionId" is null
at org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:476)
... 13 more
Too many connections, too little memory or something completely different?
Regards
Holger Hymmen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image002.jpg@01DA1644.E8DFA8D0]
Learn more about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Re: Error finding child executions for parent ID null
Posted by Bart Maertens <ba...@apache.org>.
Hi Holger,
Any chance you imported your PDI project into the default or samples hop
project?
The "local-audit" and "first-last" execution information and data profiles
are not part of the importer, but are defined in the default project.
Regards,
Bart
On Mon, Nov 13, 2023 at 12:28 PM <ho...@comline.de> wrote:
> Hi Bart,
>
>
>
> Thank you. That has done the job.
>
> I don’t know if my configuration stems from the port, but it contained the
> following:
>
>
>
>
>
> Regards
>
>
>
> Holger
>
>
>
> *Dipl.-Inf.** Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> <https://www.iqvia.com>Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
>
> *Von:* Bart Maertens <ba...@apache.org>
> *Gesendet:* Montag, 13. November 2023 11:30
> *An:* users@hop.apache.org
> *Betreff:* Re: Error finding child executions for parent ID null
>
>
>
> Hi Holger,
>
>
>
> The last part in your error message [1] seems to point to an Execution
> Information Location.
>
> Even though execution information could provide useful information, let's
> see if you can get your workflows and pipelines to work as close as
> possible to the PDI environment.
>
> Can you verify if your pipeline run configuration (metadata perspective)
> doesn't contain anything for "Execution information location" and
> "Execution data profile"?
>
> If there are any values there, can you remove those, save your pipeline
> run configuration and try again (similar for workflow run configuration).
>
>
>
>
>
> [1] Caused by: java.lang.NullPointerException: Cannot invoke
> "String.equals(Object)" because "parentExecutionId" is null
> at
> org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:476)
>
>
>
> Regards,
>
> Bart
>
>
>
> On Mon, Nov 13, 2023 at 11:07 AM <ho...@comline.de> wrote:
>
> Hi,
>
>
>
> I ported my workflows and pipelines from Pentaho Data Integration 7.1 to
> Apache HOP 2.6.
>
> When I start my biggest and most complex workflow, it runs fine until it
> enters a pipeline in one of the sub workflows. It logs 36 times that it has
> connected to databases and then the log stops for quite a while. That are
> not all of the needed connections, but it doesn’t log about more and after
> some time it opens the following error window:
>
>
>
>
>
> The details are:
>
> org.apache.hop.core.exception.HopException:
>
> Error finding child executions for parent ID null
>
> Cannot invoke "String.equals(Object)" because "parentExecutionId" is null
>
>
>
> at
> org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:482)
>
> at
> org.apache.hop.ui.hopgui.file.workflow.HopGuiWorkflowGraph.viewActionExecutionInfo(HopGuiWorkflowGraph.java:4204)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
>
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at
> java.base/java.lang.reflect.Method.invoke(Method.java:568)
>
> at
> org.apache.hop.core.gui.plugin.action.GuiActionLambdaBuilder.lambda$createLambda$0(GuiActionLambdaBuilder.java:96)
>
> at
> org.apache.hop.ui.hopgui.context.GuiContextUtil.lambda$handleActionSelection$0(GuiContextUtil.java:183)
>
> at
> org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
>
> at
> org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)
>
> at
> org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4029)
>
> at
> org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3645)
>
> at org.apache.hop.ui.hopgui.HopGui.open(HopGui.java:473)
>
> at org.apache.hop.ui.hopgui.HopGui.main(HopGui.java:351)
>
> Caused by: java.lang.NullPointerException: Cannot invoke
> "String.equals(Object)" because "parentExecutionId" is null
>
> at
> org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:476)
>
> ... 13 more
>
>
>
> Too many connections, too little memory or something completely different?
>
>
>
> Regards
>
> Holger Hymmen
>
>
>
>
>
> *Dipl.-Inf. Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
>
>
Re: Port and You need to specify a workflow run configuration to execute this workflow
Posted by Matt Casters <ma...@leanwithdata.com>.
Quite annoying indeed. It's a special case for the transforms and actions.
Search uses the properties of the metadata class but somehow rejects those
properties where the value is null.
I filed issue 3389 for this.
Op di 14 nov. 2023 09:39 schreef <ho...@comline.de>:
> Yes, Matt, I have seen the values, but most of my workflows/pipelines are
> missing a runconfiguration after the port. There is no empty value or such.
> They do not contain the words runconfiguration…
>
> So, a search only finds the ones I have set a runconfiguration for yet.
>
>
>
> *Von:* Matt Casters <ma...@leanwithdata.com>
> *Gesendet:* Dienstag, 14. November 2023 09:23
> *An:* users@hop.apache.org
> *Betreff:* Re: Port and You need to specify a workflow run configuration
> to execute this workflow
>
>
>
> The value (or indeed empty) is in a column in the search results if you
> scroll to the right.
>
>
>
> Op di 14 nov. 2023 09:20 schreef <ho...@comline.de>:
>
> Hello Matt,
>
>
>
> when you said do a search, did you mean this one (for some reason my
> search string runconfiguration is gone) :
>
>
>
>
>
> With this, I can find the workflows/pipelines that have been assigned a
> runconfiguration, but I suppose I cannot find the ones, that are still
> missing a runconfiguration with it or am I mistaken?
>
> Regards
>
> Holger
>
> *Dipl.-Inf. Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
>
> *Von:* Matt Casters <ma...@leanwithdata.com>
> *Gesendet:* Dienstag, 14. November 2023 08:58
> *An:* users@hop.apache.org
> *Betreff:* Re: Port and You need to specify a workflow run configuration
> to execute this workflow
>
>
>
> Just do a search on runconfiguration and you'll get a list of all
> occurrences with their value. You can even jump to the locations from there.
>
>
>
> If you feel like there could be improvements to the Kettle importer, feel
> free to raise an issue in the github project. Thanks in advance.
>
>
>
> Good luck,
>
>
>
> Matt
>
>
>
> Op di 14 nov. 2023 08:44 schreef <ho...@comline.de>:
>
> Hi,
>
>
>
> When you port from PDI, you have to check that a run configuration is set
> in every workflow for called sub workflows and pipelines, otherwise you get
> the following error:
>
>
>
> ERROR: Error running action 'workflow' :
>
> ERROR: org.apache.hop.core.exception.HopException:
>
> You need to specify a workflow run configuration to execute this workflow
>
>
>
> It is a simple thing to solve by editing the workflow/pipeling with “edit
> the action” and only pressing OK, because it automatically puts local into
> the run configuration.
>
>
>
> With the number of workflows and pipelines in my port, it is a sisyphean
> task, to start all workflows and only change the missing run configurations
> or to enter all workflows, call “edit the action” on all workflows and
> pipelines and not check before.
>
>
>
> Although the workflow after the port and the workflow after the above
> mentioned, differ quite a bit. The main difference seems to be the sorting
> of the elements, but in some cases, it seems, that the port has not entered
> a run configuration.
>
>
>
> For example:
>
> *Pre-editing
>
> post-editing*
>
>
>
> Is there a bug in the port or why are some of the workflows/pipelines
> overlooked when it comes to assigning a run configuration?
>
>
>
> Mit freundlichen Grüßen
>
>
>
> *Dipl.-Inf. Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
> *Dipl.-Inf.** Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> <https://www.iqvia.com>Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
>
AW: Port and You need to specify a workflow run configuration to execute this workflow
Posted by ho...@comline.de.
Yes, Matt, I have seen the values, but most of my workflows/pipelines are missing a runconfiguration after the port. There is no empty value or such. They do not contain the words runconfiguration…
So, a search only finds the ones I have set a runconfiguration for yet.
Von: Matt Casters <ma...@leanwithdata.com>
Gesendet: Dienstag, 14. November 2023 09:23
An: users@hop.apache.org
Betreff: Re: Port and You need to specify a workflow run configuration to execute this workflow
The value (or indeed empty) is in a column in the search results if you scroll to the right.
Op di 14 nov. 2023 09:20 schreef <ho...@comline.de>>:
Hello Matt,
when you said do a search, did you mean this one (for some reason my search string runconfiguration is gone) :
[cid:image001.png@01DA16DB.55FF1400]
With this, I can find the workflows/pipelines that have been assigned a runconfiguration, but I suppose I cannot find the ones, that are still missing a runconfiguration with it or am I mistaken?
Regards
Holger
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image001.jpg@01DA16DD.7AC4EB00]
Learn more about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Von: Matt Casters <ma...@leanwithdata.com>>
Gesendet: Dienstag, 14. November 2023 08:58
An: users@hop.apache.org<ma...@hop.apache.org>
Betreff: Re: Port and You need to specify a workflow run configuration to execute this workflow
Just do a search on runconfiguration and you'll get a list of all occurrences with their value. You can even jump to the locations from there.
If you feel like there could be improvements to the Kettle importer, feel free to raise an issue in the github project. Thanks in advance.
Good luck,
Matt
Op di 14 nov. 2023 08:44 schreef <ho...@comline.de>>:
Hi,
When you port from PDI, you have to check that a run configuration is set in every workflow for called sub workflows and pipelines, otherwise you get the following error:
ERROR: Error running action 'workflow' :
ERROR: org.apache.hop.core.exception.HopException:
You need to specify a workflow run configuration to execute this workflow
It is a simple thing to solve by editing the workflow/pipeling with “edit the action” and only pressing OK, because it automatically puts local into the run configuration.
With the number of workflows and pipelines in my port, it is a sisyphean task, to start all workflows and only change the missing run configurations or to enter all workflows, call “edit the action” on all workflows and pipelines and not check before.
Although the workflow after the port and the workflow after the above mentioned, differ quite a bit. The main difference seems to be the sorting of the elements, but in some cases, it seems, that the port has not entered a run configuration.
For example:
Pre-editing post-editing
Is there a bug in the port or why are some of the workflows/pipelines overlooked when it comes to assigning a run configuration?
Mit freundlichen Grüßen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image002.jpg@01DA16DB.D5CBE7D0]
Learn more about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Re: Port and You need to specify a workflow run configuration to execute this workflow
Posted by Matt Casters <ma...@leanwithdata.com>.
The value (or indeed empty) is in a column in the search results if you
scroll to the right.
Op di 14 nov. 2023 09:20 schreef <ho...@comline.de>:
> Hello Matt,
>
>
>
> when you said do a search, did you mean this one (for some reason my
> search string runconfiguration is gone) :
>
> [image: cid:image001.png@01DA16DB.55FF1400]
>
>
>
>
>
> With this, I can find the workflows/pipelines that have been assigned a
> runconfiguration, but I suppose I cannot find the ones, that are still
> missing a runconfiguration with it or am I mistaken?
>
> Regards
>
> Holger
>
> *Dipl.-Inf.** Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> <https://www.iqvia.com>Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
>
> *Von:* Matt Casters <ma...@leanwithdata.com>
> *Gesendet:* Dienstag, 14. November 2023 08:58
> *An:* users@hop.apache.org
> *Betreff:* Re: Port and You need to specify a workflow run configuration
> to execute this workflow
>
>
>
> Just do a search on runconfiguration and you'll get a list of all
> occurrences with their value. You can even jump to the locations from there.
>
>
>
> If you feel like there could be improvements to the Kettle importer, feel
> free to raise an issue in the github project. Thanks in advance.
>
>
>
> Good luck,
>
>
>
> Matt
>
>
>
> Op di 14 nov. 2023 08:44 schreef <ho...@comline.de>:
>
> Hi,
>
>
>
> When you port from PDI, you have to check that a run configuration is set
> in every workflow for called sub workflows and pipelines, otherwise you get
> the following error:
>
>
>
> ERROR: Error running action 'workflow' :
>
> ERROR: org.apache.hop.core.exception.HopException:
>
> You need to specify a workflow run configuration to execute this workflow
>
>
>
> It is a simple thing to solve by editing the workflow/pipeling with “edit
> the action” and only pressing OK, because it automatically puts local into
> the run configuration.
>
>
>
> With the number of workflows and pipelines in my port, it is a sisyphean
> task, to start all workflows and only change the missing run configurations
> or to enter all workflows, call “edit the action” on all workflows and
> pipelines and not check before.
>
>
>
> Although the workflow after the port and the workflow after the above
> mentioned, differ quite a bit. The main difference seems to be the sorting
> of the elements, but in some cases, it seems, that the port has not entered
> a run configuration.
>
>
>
> For example:
>
> *Pre-editing
>
> post-editing*
>
> [image: cid:image001.png@01DA1637.557CB720]
>
>
>
> Is there a bug in the port or why are some of the workflows/pipelines
> overlooked when it comes to assigning a run configuration?
>
>
>
> Mit freundlichen Grüßen
>
>
>
> *Dipl.-Inf. Holger Hymmen*
> *Berater*
> Healthcare
>
> [image: cid:cl_logo_footer.jpg]
>
> Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
AW: Port and You need to specify a workflow run configuration to execute this workflow
Posted by ho...@comline.de.
Hello Matt,
when you said do a search, did you mean this one (for some reason my search string runconfiguration is gone) :
[cid:image001.png@01DA16DB.55FF1400]
With this, I can find the workflows/pipelines that have been assigned a runconfiguration, but I suppose I cannot find the ones, that are still missing a runconfiguration with it or am I mistaken?
Regards
Holger
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Von: Matt Casters <ma...@leanwithdata.com>
Gesendet: Dienstag, 14. November 2023 08:58
An: users@hop.apache.org
Betreff: Re: Port and You need to specify a workflow run configuration to execute this workflow
Just do a search on runconfiguration and you'll get a list of all occurrences with their value. You can even jump to the locations from there.
If you feel like there could be improvements to the Kettle importer, feel free to raise an issue in the github project. Thanks in advance.
Good luck,
Matt
Op di 14 nov. 2023 08:44 schreef <ho...@comline.de>>:
Hi,
When you port from PDI, you have to check that a run configuration is set in every workflow for called sub workflows and pipelines, otherwise you get the following error:
ERROR: Error running action 'workflow' :
ERROR: org.apache.hop.core.exception.HopException:
You need to specify a workflow run configuration to execute this workflow
It is a simple thing to solve by editing the workflow/pipeling with “edit the action” and only pressing OK, because it automatically puts local into the run configuration.
With the number of workflows and pipelines in my port, it is a sisyphean task, to start all workflows and only change the missing run configurations or to enter all workflows, call “edit the action” on all workflows and pipelines and not check before.
Although the workflow after the port and the workflow after the above mentioned, differ quite a bit. The main difference seems to be the sorting of the elements, but in some cases, it seems, that the port has not entered a run configuration.
For example:
Pre-editing post-editing
[cid:image001.png@01DA1637.557CB720]
Is there a bug in the port or why are some of the workflows/pipelines overlooked when it comes to assigning a run configuration?
Mit freundlichen Grüßen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Re: Port and You need to specify a workflow run configuration to execute this workflow
Posted by Matt Casters <ma...@leanwithdata.com>.
Just do a search on runconfiguration and you'll get a list of all
occurrences with their value. You can even jump to the locations from there.
If you feel like there could be improvements to the Kettle importer, feel
free to raise an issue in the github project. Thanks in advance.
Good luck,
Matt
Op di 14 nov. 2023 08:44 schreef <ho...@comline.de>:
> Hi,
>
>
>
> When you port from PDI, you have to check that a run configuration is set
> in every workflow for called sub workflows and pipelines, otherwise you get
> the following error:
>
>
>
> ERROR: Error running action 'workflow' :
>
> ERROR: org.apache.hop.core.exception.HopException:
>
> You need to specify a workflow run configuration to execute this workflow
>
>
>
> It is a simple thing to solve by editing the workflow/pipeling with “edit
> the action” and only pressing OK, because it automatically puts local into
> the run configuration.
>
>
>
> With the number of workflows and pipelines in my port, it is a sisyphean
> task, to start all workflows and only change the missing run configurations
> or to enter all workflows, call “edit the action” on all workflows and
> pipelines and not check before.
>
>
>
> Although the workflow after the port and the workflow after the above
> mentioned, differ quite a bit. The main difference seems to be the sorting
> of the elements, but in some cases, it seems, that the port has not entered
> a run configuration.
>
>
>
> For example:
>
> *Pre-editing
>
> post-editing*
>
>
>
> Is there a bug in the port or why are some of the workflows/pipelines
> overlooked when it comes to assigning a run configuration?
>
>
>
> Mit freundlichen Grüßen
>
>
>
> *Dipl.-Inf.** Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> <https://www.iqvia.com>Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
AW: Port and You need to specify a workflow run configuration to execute this workflow
Posted by ho...@comline.de.
Hi Bart,
after redoing parts of the port, I would say its not the pipelines, that are missing the run configuration, but more or less all the sub-workflows. Even though I had some sub workflows at the deepest end that seemed to have the run configuration…
Regards
Holger
Von: Bart Maertens <ba...@apache.org>
Gesendet: Dienstag, 14. November 2023 09:11
An: users@hop.apache.org
Betreff: Re: Port and You need to specify a workflow run configuration to execute this workflow
Hi Holger,
Can you identify a pattern in the workflows/pipelines where the run configuration is not entered correctly?
We've made quite a number of changes and improvements in the importer for the upcoming 2.7.0, but there's always more we can improve.
Most PDI/Kettle projects that are migrated (upgraded) to Apache Hop have had a decade or so to build edge cases. Discovering and implementing all of those cases is an iterative process...
Feel free to create github tickets for the issues you encounter, ideally with XML snippets that we can use to reproduce.
Regards,
Bart
On Tue, Nov 14, 2023 at 8:44 AM <ho...@comline.de>> wrote:
Hi,
When you port from PDI, you have to check that a run configuration is set in every workflow for called sub workflows and pipelines, otherwise you get the following error:
ERROR: Error running action 'workflow' :
ERROR: org.apache.hop.core.exception.HopException:
You need to specify a workflow run configuration to execute this workflow
It is a simple thing to solve by editing the workflow/pipeling with “edit the action” and only pressing OK, because it automatically puts local into the run configuration.
With the number of workflows and pipelines in my port, it is a sisyphean task, to start all workflows and only change the missing run configurations or to enter all workflows, call “edit the action” on all workflows and pipelines and not check before.
Although the workflow after the port and the workflow after the above mentioned, differ quite a bit. The main difference seems to be the sorting of the elements, but in some cases, it seems, that the port has not entered a run configuration.
For example:
Pre-editing post-editing
[cid:image001.png@01DA171C.491866B0]
Is there a bug in the port or why are some of the workflows/pipelines overlooked when it comes to assigning a run configuration?
Mit freundlichen Grüßen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image003.jpg@01DA171C.491866B0]
Learn more about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Re: Port and You need to specify a workflow run configuration to execute this workflow
Posted by Bart Maertens <ba...@apache.org>.
Hi Holger,
Can you identify a pattern in the workflows/pipelines where the run
configuration is not entered correctly?
We've made quite a number of changes and improvements in the importer for
the upcoming 2.7.0, but there's always more we can improve.
Most PDI/Kettle projects that are migrated (upgraded) to Apache Hop have
had a decade or so to build edge cases. Discovering and implementing all of
those cases is an iterative process...
Feel free to create github tickets for the issues you encounter, ideally
with XML snippets that we can use to reproduce.
Regards,
Bart
On Tue, Nov 14, 2023 at 8:44 AM <ho...@comline.de> wrote:
> Hi,
>
>
>
> When you port from PDI, you have to check that a run configuration is set
> in every workflow for called sub workflows and pipelines, otherwise you get
> the following error:
>
>
>
> ERROR: Error running action 'workflow' :
>
> ERROR: org.apache.hop.core.exception.HopException:
>
> You need to specify a workflow run configuration to execute this workflow
>
>
>
> It is a simple thing to solve by editing the workflow/pipeling with “edit
> the action” and only pressing OK, because it automatically puts local into
> the run configuration.
>
>
>
> With the number of workflows and pipelines in my port, it is a sisyphean
> task, to start all workflows and only change the missing run configurations
> or to enter all workflows, call “edit the action” on all workflows and
> pipelines and not check before.
>
>
>
> Although the workflow after the port and the workflow after the above
> mentioned, differ quite a bit. The main difference seems to be the sorting
> of the elements, but in some cases, it seems, that the port has not entered
> a run configuration.
>
>
>
> For example:
>
> *Pre-editing
>
> post-editing*
>
>
>
> Is there a bug in the port or why are some of the workflows/pipelines
> overlooked when it comes to assigning a run configuration?
>
>
>
> Mit freundlichen Grüßen
>
>
>
> *Dipl.-Inf.** Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> <https://www.iqvia.com>Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
Port and You need to specify a workflow run configuration to execute this workflow
Posted by ho...@comline.de.
Hi,
When you port from PDI, you have to check that a run configuration is set in every workflow for called sub workflows and pipelines, otherwise you get the following error:
ERROR: Error running action 'workflow' :
ERROR: org.apache.hop.core.exception.HopException:
You need to specify a workflow run configuration to execute this workflow
It is a simple thing to solve by editing the workflow/pipeling with “edit the action” and only pressing OK, because it automatically puts local into the run configuration.
With the number of workflows and pipelines in my port, it is a sisyphean task, to start all workflows and only change the missing run configurations or to enter all workflows, call “edit the action” on all workflows and pipelines and not check before.
Although the workflow after the port and the workflow after the above mentioned, differ quite a bit. The main difference seems to be the sorting of the elements, but in some cases, it seems, that the port has not entered a run configuration.
For example:
Pre-editing post-editing
[cid:image001.png@01DA1637.557CB720]
Is there a bug in the port or why are some of the workflows/pipelines overlooked when it comes to assigning a run configuration?
Mit freundlichen Grüßen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
AW: Error finding child executions for parent ID null
Posted by ho...@comline.de.
Hi Bart,
Thank you. That has done the job.
I don’t know if my configuration stems from the port, but it contained the following:
[cid:image001.png@01DA162C.C285C650]
Regards
Holger
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Von: Bart Maertens <ba...@apache.org>
Gesendet: Montag, 13. November 2023 11:30
An: users@hop.apache.org
Betreff: Re: Error finding child executions for parent ID null
Hi Holger,
The last part in your error message [1] seems to point to an Execution Information Location.
Even though execution information could provide useful information, let's see if you can get your workflows and pipelines to work as close as possible to the PDI environment.
Can you verify if your pipeline run configuration (metadata perspective) doesn't contain anything for "Execution information location" and "Execution data profile"?
If there are any values there, can you remove those, save your pipeline run configuration and try again (similar for workflow run configuration).
[cid:image002.png@01DA162C.C285C650]
[1] Caused by: java.lang.NullPointerException: Cannot invoke "String.equals(Object)" because "parentExecutionId" is null
at org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:476)
Regards,
Bart
On Mon, Nov 13, 2023 at 11:07 AM <ho...@comline.de>> wrote:
Hi,
I ported my workflows and pipelines from Pentaho Data Integration 7.1 to Apache HOP 2.6.
When I start my biggest and most complex workflow, it runs fine until it enters a pipeline in one of the sub workflows. It logs 36 times that it has connected to databases and then the log stops for quite a while. That are not all of the needed connections, but it doesn’t log about more and after some time it opens the following error window:
[cid:image003.png@01DA162C.C285C650]
The details are:
org.apache.hop.core.exception.HopException:
Error finding child executions for parent ID null
Cannot invoke "String.equals(Object)" because "parentExecutionId" is null
at org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:482)
at org.apache.hop.ui.hopgui.file.workflow.HopGuiWorkflowGraph.viewActionExecutionInfo(HopGuiWorkflowGraph.java:4204)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.apache.hop.core.gui.plugin.action.GuiActionLambdaBuilder.lambda$createLambda$0(GuiActionLambdaBuilder.java:96)
at org.apache.hop.ui.hopgui.context.GuiContextUtil.lambda$handleActionSelection$0(GuiContextUtil.java:183)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4029)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3645)
at org.apache.hop.ui.hopgui.HopGui.open(HopGui.java:473)
at org.apache.hop.ui.hopgui.HopGui.main(HopGui.java:351)
Caused by: java.lang.NullPointerException: Cannot invoke "String.equals(Object)" because "parentExecutionId" is null
at org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:476)
... 13 more
Too many connections, too little memory or something completely different?
Regards
Holger Hymmen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image004.jpg@01DA162C.C285C650]
Learn more about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Re: Error finding child executions for parent ID null
Posted by Bart Maertens <ba...@apache.org>.
Hi Holger,
The last part in your error message [1] seems to point to an Execution
Information Location.
Even though execution information could provide useful information, let's
see if you can get your workflows and pipelines to work as close as
possible to the PDI environment.
Can you verify if your pipeline run configuration (metadata perspective)
doesn't contain anything for "Execution information location" and
"Execution data profile"?
If there are any values there, can you remove those, save your pipeline run
configuration and try again (similar for workflow run configuration).
[image: image.png]
[1] Caused by: java.lang.NullPointerException: Cannot invoke
"String.equals(Object)" because "parentExecutionId" is null
at
org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:476)
Regards,
Bart
On Mon, Nov 13, 2023 at 11:07 AM <ho...@comline.de> wrote:
> Hi,
>
>
>
> I ported my workflows and pipelines from Pentaho Data Integration 7.1 to
> Apache HOP 2.6.
>
> When I start my biggest and most complex workflow, it runs fine until it
> enters a pipeline in one of the sub workflows. It logs 36 times that it has
> connected to databases and then the log stops for quite a while. That are
> not all of the needed connections, but it doesn’t log about more and after
> some time it opens the following error window:
>
>
>
>
>
> The details are:
>
> org.apache.hop.core.exception.HopException:
>
> Error finding child executions for parent ID null
>
> Cannot invoke "String.equals(Object)" because "parentExecutionId" is null
>
>
>
> at
> org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:482)
>
> at
> org.apache.hop.ui.hopgui.file.workflow.HopGuiWorkflowGraph.viewActionExecutionInfo(HopGuiWorkflowGraph.java:4204)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
>
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at
> java.base/java.lang.reflect.Method.invoke(Method.java:568)
>
> at
> org.apache.hop.core.gui.plugin.action.GuiActionLambdaBuilder.lambda$createLambda$0(GuiActionLambdaBuilder.java:96)
>
> at
> org.apache.hop.ui.hopgui.context.GuiContextUtil.lambda$handleActionSelection$0(GuiContextUtil.java:183)
>
> at
> org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
>
> at
> org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)
>
> at
> org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4029)
>
> at
> org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3645)
>
> at org.apache.hop.ui.hopgui.HopGui.open(HopGui.java:473)
>
> at org.apache.hop.ui.hopgui.HopGui.main(HopGui.java:351)
>
> Caused by: java.lang.NullPointerException: Cannot invoke
> "String.equals(Object)" because "parentExecutionId" is null
>
> at
> org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:476)
>
> ... 13 more
>
>
>
> Too many connections, too little memory or something completely different?
>
>
>
> Regards
>
> Holger Hymmen
>
>
>
>
>
> *Dipl.-Inf.** Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> <https://www.iqvia.com>Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
>
Error finding child executions for parent ID null
Posted by ho...@comline.de.
Hi,
I ported my workflows and pipelines from Pentaho Data Integration 7.1 to Apache HOP 2.6.
When I start my biggest and most complex workflow, it runs fine until it enters a pipeline in one of the sub workflows. It logs 36 times that it has connected to databases and then the log stops for quite a while. That are not all of the needed connections, but it doesn’t log about more and after some time it opens the following error window:
[cid:image001.png@01DA1621.959C47F0]
The details are:
org.apache.hop.core.exception.HopException:
Error finding child executions for parent ID null
Cannot invoke "String.equals(Object)" because "parentExecutionId" is null
at org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:482)
at org.apache.hop.ui.hopgui.file.workflow.HopGuiWorkflowGraph.viewActionExecutionInfo(HopGuiWorkflowGraph.java:4204)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.apache.hop.core.gui.plugin.action.GuiActionLambdaBuilder.lambda$createLambda$0(GuiActionLambdaBuilder.java:96)
at org.apache.hop.ui.hopgui.context.GuiContextUtil.lambda$handleActionSelection$0(GuiContextUtil.java:183)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4029)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3645)
at org.apache.hop.ui.hopgui.HopGui.open(HopGui.java:473)
at org.apache.hop.ui.hopgui.HopGui.main(HopGui.java:351)
Caused by: java.lang.NullPointerException: Cannot invoke "String.equals(Object)" because "parentExecutionId" is null
at org.apache.hop.execution.local.FileExecutionInfoLocation.findExecutions(FileExecutionInfoLocation.java:476)
... 13 more
Too many connections, too little memory or something completely different?
Regards
Holger Hymmen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
own libraries and "User defined Java class"
Posted by ho...@comline.de.
Hi,
where do I place my own libraries in the hop directory structure, so that I can use them in the “User defined Java class” step? Or do I have to do something else to be able to use them?
Regards
Holger Hymmen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
AW: DB2 JDBC4
Posted by ho...@comline.de.
For all other columns, that do not use an alias, I do get a type. It’s been ages since I worked on the PDI source code, so I do not remember what was done differently, so that a type can be returned for an alias column.
Von: Bart Maertens <ba...@apache.org>
Gesendet: Montag, 20. März 2023 15:56
An: users@hop.apache.org
Betreff: Re: DB2 JDBC4
From a very distant past, I remember the DB2 JDBC driver just dropped column aliases, looks like that still hasn't changed.
Do you get a type if you leave out the alias?
On Mon, Mar 20, 2023 at 3:20 PM <ho...@comline.de>> wrote:
Ok, so I had a good look at the log file myself and it seems that we misinterpreted it before. It is not a transform that is getting a row with the wrong column name, but it’s the first usage of the with “as” renamed column that is producing the error. If I do “show output fields” on the original transform that is creating the problematic column, I can see that it is not generating a type for the renamed column. As column type we get “None”. Is there anything we can do, without casting every single rename?
Thanks
Holger
Von: holger.hymmen@comline.de<ma...@comline.de> <ho...@comline.de>>
Gesendet: Montag, 20. März 2023 14:39
An: users@hop.apache.org<ma...@hop.apache.org>
Betreff: AW: DB2 JDBC4
No, I do not see it being added to the url. I only see the jdbc:db2://server:port/db part.
I do not remember from which page my info with the url comes from, but I have found this https://www.ibm.com/support/pages/db2-alias-name-behaviour-changes-latest-drivers-versions while searching and it works fine with Squirrel, but I have tried it with a 2 as value too and it does not add the option either.
We are using the fields “server host name”, “port number” and “database name” in tab general to define the connection parameters and not the manual connection url.
If I try the manual connection url, I have to delete the option and I see the url I want to see, but running it does not make a difference with neiter false nor 2. It seems to ignore the setting.
Kind regards
Holger
Von: Hans Van Akelyen <ha...@gmail.com>>
Gesendet: Montag, 20. März 2023 12:31
An: Hymmen, Holger <ho...@comline.de>>; users@hop.apache.org<ma...@hop.apache.org>
Betreff: Re: DB2 JDBC4
After adding the option to the connection and pressing the test button is the parameter correctly added to the url?
I’m also not sure if “false” is a valid value for this setting according to the docs https://www.ibm.com/docs/en/db2/11.5?topic=SSEPGG_11.5.0/com.ibm.db2.luw.apdv.java.doc/src/tpc/imjcc_r0052607.htm its either 0,1,2 in your case it should be set to 2
Kind Regards,
Hans
On 20 March 2023 at 11:52:13, holger.hymmen@comline.de<ma...@comline.de> (holger.hymmen@comline.de<ma...@comline.de>) wrote:
Hi,
I am new to HOP, but have been using PDI since about 2005.
We are trying to port our project to HOP 2.3.0 and are running into a problem with DB2 and JDBC 4. With the JDBC4 logic an select id as id_new from table returns id, but I still need it to return id_new. With other programs an „useJDBC4ColumnNameAndLabelSemantics=false“ works. I have tried setting it under Options in the connection section, but it seems to be ignoring it. I still get id returned.
I hope somebody can help me with this.
Thanks in advance
Holger Hymmen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image001.jpg@01D95B45.5DBE6490]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image002.jpg@01D95B45.5DBE6490]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image004.jpg@01D95B45.5DBE6490]
Learn more about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Re: DB2 JDBC4
Posted by Bart Maertens <ba...@apache.org>.
From a very distant past, I remember the DB2 JDBC driver just dropped
column aliases, looks like that still hasn't changed.
Do you get a type if you leave out the alias?
On Mon, Mar 20, 2023 at 3:20 PM <ho...@comline.de> wrote:
> Ok, so I had a good look at the log file myself and it seems that we
> misinterpreted it before. It is not a transform that is getting a row with
> the wrong column name, but it’s the first usage of the with “as” renamed
> column that is producing the error. If I do “show output fields” on the
> original transform that is creating the problematic column, I can see that
> it is not generating a type for the renamed column. As column type we get
> “None”. Is there anything we can do, without casting every single rename?
>
> Thanks
>
> Holger
>
>
>
> *Von:* holger.hymmen@comline.de <ho...@comline.de>
> *Gesendet:* Montag, 20. März 2023 14:39
> *An:* users@hop.apache.org
> *Betreff:* AW: DB2 JDBC4
>
>
>
> No, I do not see it being added to the url. I only see the
> jdbc:db2://server:port/db part.
>
>
>
> I do not remember from which page my info with the url comes from, but I
> have found this
> https://www.ibm.com/support/pages/db2-alias-name-behaviour-changes-latest-drivers-versions
> while searching and it works fine with Squirrel, but I have tried it with a
> 2 as value too and it does not add the option either.
>
>
>
> We are using the fields “server host name”, “port number” and “database
> name” in tab general to define the connection parameters and not the manual
> connection url.
>
>
>
> If I try the manual connection url, I have to delete the option and I see
> the url I want to see, but running it does not make a difference with
> neiter false nor 2. It seems to ignore the setting.
>
>
>
> Kind regards
>
>
>
> Holger
>
>
>
> *Von:* Hans Van Akelyen <ha...@gmail.com>
> *Gesendet:* Montag, 20. März 2023 12:31
> *An:* Hymmen, Holger <ho...@comline.de>; users@hop.apache.org
> *Betreff:* Re: DB2 JDBC4
>
>
>
> After adding the option to the connection and pressing the test button is
> the parameter correctly added to the url?
>
>
>
> I’m also not sure if “false” is a valid value for this setting according
> to the docs
> https://www.ibm.com/docs/en/db2/11.5?topic=SSEPGG_11.5.0/com.ibm.db2.luw.apdv.java.doc/src/tpc/imjcc_r0052607.htm its
> either 0,1,2 in your case it should be set to 2
>
>
>
> Kind Regards,
>
> Hans
>
>
>
>
>
> On 20 March 2023 at 11:52:13, holger.hymmen@comline.de (
> holger.hymmen@comline.de) wrote:
>
> Hi,
>
>
>
> I am new to HOP, but have been using PDI since about 2005.
>
>
>
> We are trying to port our project to HOP 2.3.0 and are running into a
> problem with DB2 and JDBC 4. With the JDBC4 logic an select id as id_new
> from table returns id, but I still need it to return id_new. With other
> programs an „useJDBC4ColumnNameAndLabelSemantics=false“ works. I have tried
> setting it under Options in the connection section, but it seems to be
> ignoring it. I still get id returned.
>
>
>
> I hope somebody can help me with this.
>
> Thanks in advance
>
> Holger Hymmen
>
>
>
>
> *Dipl.-Inf. Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> Learn more <https://www.iqvia.com> about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
> *Dipl.-Inf. Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> Learn more <https://www.iqvia.com> about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
> *Dipl.-Inf.** Holger Hymmen*
> *Berater*
> Healthcare
>
>
>
> <https://www.iqvia.com>Learn more about IQVIA
>
> Comline GmbH - an IQVIA Business
> Hauert 8, 44227 Dortmund
>
> *T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
>
> Geschäftsführer: Dr. Frank Wartenberg
> Sitz der Gesellschaft: Dortmund
> Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
>
>
>
AW: DB2 JDBC4
Posted by ho...@comline.de.
Ok, so I had a good look at the log file myself and it seems that we misinterpreted it before. It is not a transform that is getting a row with the wrong column name, but it’s the first usage of the with “as” renamed column that is producing the error. If I do “show output fields” on the original transform that is creating the problematic column, I can see that it is not generating a type for the renamed column. As column type we get “None”. Is there anything we can do, without casting every single rename?
Thanks
Holger
Von: holger.hymmen@comline.de <ho...@comline.de>
Gesendet: Montag, 20. März 2023 14:39
An: users@hop.apache.org
Betreff: AW: DB2 JDBC4
No, I do not see it being added to the url. I only see the jdbc:db2://server:port/db part.
I do not remember from which page my info with the url comes from, but I have found this https://www.ibm.com/support/pages/db2-alias-name-behaviour-changes-latest-drivers-versions while searching and it works fine with Squirrel, but I have tried it with a 2 as value too and it does not add the option either.
We are using the fields “server host name”, “port number” and “database name” in tab general to define the connection parameters and not the manual connection url.
If I try the manual connection url, I have to delete the option and I see the url I want to see, but running it does not make a difference with neiter false nor 2. It seems to ignore the setting.
Kind regards
Holger
Von: Hans Van Akelyen <ha...@gmail.com>>
Gesendet: Montag, 20. März 2023 12:31
An: Hymmen, Holger <ho...@comline.de>>; users@hop.apache.org<ma...@hop.apache.org>
Betreff: Re: DB2 JDBC4
After adding the option to the connection and pressing the test button is the parameter correctly added to the url?
I’m also not sure if “false” is a valid value for this setting according to the docs https://www.ibm.com/docs/en/db2/11.5?topic=SSEPGG_11.5.0/com.ibm.db2.luw.apdv.java.doc/src/tpc/imjcc_r0052607.htm its either 0,1,2 in your case it should be set to 2
Kind Regards,
Hans
On 20 March 2023 at 11:52:13, holger.hymmen@comline.de<ma...@comline.de> (holger.hymmen@comline.de<ma...@comline.de>) wrote:
Hi,
I am new to HOP, but have been using PDI since about 2005.
We are trying to port our project to HOP 2.3.0 and are running into a problem with DB2 and JDBC 4. With the JDBC4 logic an select id as id_new from table returns id, but I still need it to return id_new. With other programs an „useJDBC4ColumnNameAndLabelSemantics=false“ works. I have tried setting it under Options in the connection section, but it seems to be ignoring it. I still get id returned.
I hope somebody can help me with this.
Thanks in advance
Holger Hymmen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image001.jpg@01D95B3D.792CEBA0]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image003.jpg@01D95B3D.792CEBA0]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
AW: DB2 JDBC4
Posted by ho...@comline.de.
No, I do not see it being added to the url. I only see the jdbc:db2://server:port/db part.
I do not remember from which page my info with the url comes from, but I have found this https://www.ibm.com/support/pages/db2-alias-name-behaviour-changes-latest-drivers-versions while searching and it works fine with Squirrel, but I have tried it with a 2 as value too and it does not add the option either.
We are using the fields “server host name”, “port number” and “database name” in tab general to define the connection parameters and not the manual connection url.
If I try the manual connection url, I have to delete the option and I see the url I want to see, but running it does not make a difference with neiter false nor 2. It seems to ignore the setting.
Kind regards
Holger
Von: Hans Van Akelyen <ha...@gmail.com>
Gesendet: Montag, 20. März 2023 12:31
An: Hymmen, Holger <ho...@comline.de>; users@hop.apache.org
Betreff: Re: DB2 JDBC4
After adding the option to the connection and pressing the test button is the parameter correctly added to the url?
I’m also not sure if “false” is a valid value for this setting according to the docs https://www.ibm.com/docs/en/db2/11.5?topic=SSEPGG_11.5.0/com.ibm.db2.luw.apdv.java.doc/src/tpc/imjcc_r0052607.htm its either 0,1,2 in your case it should be set to 2
Kind Regards,
Hans
On 20 March 2023 at 11:52:13, holger.hymmen@comline.de<ma...@comline.de> (holger.hymmen@comline.de<ma...@comline.de>) wrote:
Hi,
I am new to HOP, but have been using PDI since about 2005.
We are trying to port our project to HOP 2.3.0 and are running into a problem with DB2 and JDBC 4. With the JDBC4 logic an select id as id_new from table returns id, but I still need it to return id_new. With other programs an „useJDBC4ColumnNameAndLabelSemantics=false“ works. I have tried setting it under Options in the connection section, but it seems to be ignoring it. I still get id returned.
I hope somebody can help me with this.
Thanks in advance
Holger Hymmen
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:image002.jpg@01D95B29.036ECC80]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de<ma...@comline.de>
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Dipl.-Inf. Holger Hymmen
Berater
Healthcare
[cid:cl_logo_footer.jpg]
Learn more<https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
T: +49 231 97575 142 | E: holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636
Re: DB2 JDBC4
Posted by Hans Van Akelyen <ha...@gmail.com>.
After adding the option to the connection and pressing the test button is
the parameter correctly added to the url?
I’m also not sure if “false” is a valid value for this setting according to
the docs
https://www.ibm.com/docs/en/db2/11.5?topic=SSEPGG_11.5.0/com.ibm.db2.luw.apdv.java.doc/src/tpc/imjcc_r0052607.htm
its
either 0,1,2 in your case it should be set to 2
Kind Regards,
Hans
On 20 March 2023 at 11:52:13, holger.hymmen@comline.de (
holger.hymmen@comline.de) wrote:
Hi,
I am new to HOP, but have been using PDI since about 2005.
We are trying to port our project to HOP 2.3.0 and are running into a
problem with DB2 and JDBC 4. With the JDBC4 logic an select id as id_new
from table returns id, but I still need it to return id_new. With other
programs an „useJDBC4ColumnNameAndLabelSemantics=false“ works. I have tried
setting it under Options in the connection section, but it seems to be
ignoring it. I still get id returned.
I hope somebody can help me with this.
Thanks in advance
Holger Hymmen
*Dipl.-Inf.** Holger Hymmen*
*Berater*
Healthcare
<https://www.iqvia.com>Learn more <https://www.iqvia.com> about IQVIA
Comline GmbH - an IQVIA Business
Hauert 8, 44227 Dortmund
*T:* +49 231 97575 142 | *E:* holger.hymmen@comline.de
Geschäftsführer: Dr. Frank Wartenberg
Sitz der Gesellschaft: Dortmund
Registergericht & -nummer: Amtsgericht Dortmund, HRB 32636