You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@hop.apache.org by Dieter Hehn <d....@traviangames.com> on 2022/08/19 11:30:53 UTC
RDBMS Connection not found in remote workflow execution
Heyho,
Not sure if this is a bug or if I'm doing something wrong (probably the latter)...
In Hop gui (2.0, win10), I have defined a database connection (tested with both redshift and mysql) and build a simple Workflow using the "SQL" workflow action. Executing the WF locally works.
From Hop gui, I then want to execute the same workflow on a remote server (running hop 2.0 in docker). When the remote run configuration has "Export linked resources" disabled, this works as well.
However, if I enable "export linked resources", the workflow fails with
2022/08/19 13:15:43 - test - Starting action [SQL]
2022/08/19 13:15:43 - SQL - ERROR: No database connection is defined.
2022/08/19 13:15:43 - test - Finished action [SQL] (result=[false])
2022/08/19 13:15:43 - test - Workflow execution finished
2022/08/19 13:15:43 - test - Workflow duration : 0.008 seconds [ 0.008" ]
2022/08/19 13:16:20 - Hop - Workflow execution has ended
I checked the exported zip file on the server. As expected, it contains the .hwf workflow definition, metadata.json and __workflow_execution_configuration__.xml.
The .hwf file contains the SQL action with the correct connection:
...
<action>
<name>SQL</name>
<description/>
<type>SQL</type>
<attributes/>
<sql>SELECT 1</sql>
<useVariableSubstitution>F</useVariableSubstitution>
<sqlfromfile>F</sqlfromfile>
<sqlfilename/>
<sendOneStatement>F</sendOneStatement>
<connection>MySQL_Test </connection>
<parallel>N</parallel>
<xloc>608</xloc>
<yloc>224</yloc>
<attributes_hac/>
</action>
...
And metadata.json contains the respective connection string:
...{"rdbms":{"MYSQL":{"databaseName":"hop","pluginId":"MYSQL","indexTablespace":null,"dataTablespace":null,"accessType":0,"hostname":"XYZ","password":"Encrypted 2be98afc808d595a049a925feacf88d","port":"3306","pluginName":"MySQL","servername":null,"driverClassName":"Mysql 8+","attributes":{"SUPPORTS_TIMESTAMP_DATA_TYPE":"Y","QUOTE_ALL_FIELDS":"N","STREAM_RESULTS":"Y","SUPPORTS_BOOLEAN_DATA_TYPE":"Y","FORCE_IDENTIFIERS_TO_LOWERCASE":"N","PRESERVE_RESERVED_WORD_CASE":"Y","SQL_CONNECT":"","FORCE_IDENTIFIERS_TO_UPPERCASE":"N","PREFERRED_SCHEMA_NAME":""},"manualUrl":"","username":"hop"}},"name":"MySQL_Test"}....
Shouldn't this be enough for Hop to be able to execute the workflow? Yet I get the error that no database connection is defined. Do I need to specify the rdbms connections explicitly elsewhere on the server?
Cheers,
Dieter Hehn
Re: RDBMS Connection not found in remote workflow execution
Posted by Matt Casters <ma...@neo4j.com>.
Tricky. Perhaps something going on with the extra space after the
connection in the action XML you showed?
Also note that on Windows names are case sensitive. In containers, on other
platforms and with exported metadata Hop is not that forgiving.
Good luck!
Matt
Op vr 19 aug. 2022 13:30 schreef Dieter Hehn <d....@traviangames.com>:
> Heyho,
>
>
>
> Not sure if this is a bug or if I’m doing something wrong (probably the
> latter)…
>
>
>
> In Hop gui (2.0, win10), I have defined a database connection (tested with
> both redshift and mysql) and build a simple Workflow using the “SQL”
> workflow action. Executing the WF locally works.
>
> From Hop gui, I then want to execute the same workflow on a remote server
> (running hop 2.0 in docker). When the remote run configuration has “Export
> linked resources” disabled, this works as well.
>
> However, if I enable “export linked resources”, the workflow fails with
>
>
>
> 2022/08/19 13:15:43 - test - Starting action [SQL]
>
> 2022/08/19 13:15:43 - SQL - ERROR: No database connection is defined.
>
> 2022/08/19 13:15:43 - test - Finished action [SQL] (result=[false])
>
> 2022/08/19 13:15:43 - test - Workflow execution finished
>
> 2022/08/19 13:15:43 - test - Workflow duration : 0.008 seconds [ 0.008" ]
>
> 2022/08/19 13:16:20 - Hop - Workflow execution has ended
>
>
>
> I checked the exported zip file on the server. As expected, it contains
> the .hwf workflow definition, metadata.json and
> __workflow_execution_configuration__.xml.
>
> The .hwf file contains the SQL action with the correct connection:
>
> …
>
> <action>
>
> <name>SQL</name>
>
> <description/>
>
> <type>SQL</type>
>
> <attributes/>
>
> <sql>SELECT 1</sql>
>
> <useVariableSubstitution>F</useVariableSubstitution>
>
> <sqlfromfile>F</sqlfromfile>
>
> <sqlfilename/>
>
> <sendOneStatement>F</sendOneStatement>
>
> <connection>MySQL_Test </connection>
>
> <parallel>N</parallel>
>
> <xloc>608</xloc>
>
> <yloc>224</yloc>
>
> <attributes_hac/>
>
> </action>
>
> …
>
>
>
> And metadata.json contains the respective connection string:
>
>
> …{"rdbms":{"MYSQL":{"databaseName":"hop","pluginId":"MYSQL","indexTablespace":null,"dataTablespace":null,"accessType":0,"hostname":"XYZ","password":"Encrypted
> 2be98afc808d595a049a925feacf88d","port":"3306","pluginName":"MySQL","servername":null,"driverClassName":"Mysql
> 8+","attributes":{"SUPPORTS_TIMESTAMP_DATA_TYPE":"Y","QUOTE_ALL_FIELDS":"N","STREAM_RESULTS":"Y","SUPPORTS_BOOLEAN_DATA_TYPE":"Y","FORCE_IDENTIFIERS_TO_LOWERCASE":"N","PRESERVE_RESERVED_WORD_CASE":"Y","SQL_CONNECT":"","FORCE_IDENTIFIERS_TO_UPPERCASE":"N","PREFERRED_SCHEMA_NAME":""},"manualUrl":"","username":"hop"}},"name":"MySQL_Test"}….
>
>
>
> Shouldn’t this be enough for Hop to be able to execute the workflow? Yet I
> get the error that no database connection is defined. Do I need to specify
> the rdbms connections explicitly elsewhere on the server?
>
>
>
> Cheers,
>
> Dieter Hehn
>
>
>
Re: RDBMS Connection not found in remote workflow execution
Posted by Matt Casters <ma...@neo4j.com>.
Tricky. Perhaps something going on with the extra space after the
connection in the action XML you showed?
Also note that on Windows names are case sensitive. In containers, on other
platforms and with exported metadata Hop is not that forgiving.
Good luck!
Matt
Op vr 19 aug. 2022 13:30 schreef Dieter Hehn <d....@traviangames.com>:
> Heyho,
>
>
>
> Not sure if this is a bug or if I’m doing something wrong (probably the
> latter)…
>
>
>
> In Hop gui (2.0, win10), I have defined a database connection (tested with
> both redshift and mysql) and build a simple Workflow using the “SQL”
> workflow action. Executing the WF locally works.
>
> From Hop gui, I then want to execute the same workflow on a remote server
> (running hop 2.0 in docker). When the remote run configuration has “Export
> linked resources” disabled, this works as well.
>
> However, if I enable “export linked resources”, the workflow fails with
>
>
>
> 2022/08/19 13:15:43 - test - Starting action [SQL]
>
> 2022/08/19 13:15:43 - SQL - ERROR: No database connection is defined.
>
> 2022/08/19 13:15:43 - test - Finished action [SQL] (result=[false])
>
> 2022/08/19 13:15:43 - test - Workflow execution finished
>
> 2022/08/19 13:15:43 - test - Workflow duration : 0.008 seconds [ 0.008" ]
>
> 2022/08/19 13:16:20 - Hop - Workflow execution has ended
>
>
>
> I checked the exported zip file on the server. As expected, it contains
> the .hwf workflow definition, metadata.json and
> __workflow_execution_configuration__.xml.
>
> The .hwf file contains the SQL action with the correct connection:
>
> …
>
> <action>
>
> <name>SQL</name>
>
> <description/>
>
> <type>SQL</type>
>
> <attributes/>
>
> <sql>SELECT 1</sql>
>
> <useVariableSubstitution>F</useVariableSubstitution>
>
> <sqlfromfile>F</sqlfromfile>
>
> <sqlfilename/>
>
> <sendOneStatement>F</sendOneStatement>
>
> <connection>MySQL_Test </connection>
>
> <parallel>N</parallel>
>
> <xloc>608</xloc>
>
> <yloc>224</yloc>
>
> <attributes_hac/>
>
> </action>
>
> …
>
>
>
> And metadata.json contains the respective connection string:
>
>
> …{"rdbms":{"MYSQL":{"databaseName":"hop","pluginId":"MYSQL","indexTablespace":null,"dataTablespace":null,"accessType":0,"hostname":"XYZ","password":"Encrypted
> 2be98afc808d595a049a925feacf88d","port":"3306","pluginName":"MySQL","servername":null,"driverClassName":"Mysql
> 8+","attributes":{"SUPPORTS_TIMESTAMP_DATA_TYPE":"Y","QUOTE_ALL_FIELDS":"N","STREAM_RESULTS":"Y","SUPPORTS_BOOLEAN_DATA_TYPE":"Y","FORCE_IDENTIFIERS_TO_LOWERCASE":"N","PRESERVE_RESERVED_WORD_CASE":"Y","SQL_CONNECT":"","FORCE_IDENTIFIERS_TO_UPPERCASE":"N","PREFERRED_SCHEMA_NAME":""},"manualUrl":"","username":"hop"}},"name":"MySQL_Test"}….
>
>
>
> Shouldn’t this be enough for Hop to be able to execute the workflow? Yet I
> get the error that no database connection is defined. Do I need to specify
> the rdbms connections explicitly elsewhere on the server?
>
>
>
> Cheers,
>
> Dieter Hehn
>
>
>