[D] Pipeline hanged. (hop)

["dikeshmoody (via GitHub)" <gi...@apache.org>]

GitHub user dikeshmoody created a discussion: Pipeline hanged.

I am trying to run a pipeline which has table input(sql server) data being exported to parquet output. "select * from table"

But the pipeline is stuck at below

2025/01/14 22:38:56 - Hop - Projects enabled
2025/01/14 22:38:56 - Hop - Enabling project : 'default'
2025/01/14 22:39:48 - Hop - Pipeline opened.
2025/01/14 22:39:48 - Hop - Launching pipeline [load_links]...
2025/01/14 22:39:48 - Hop - Started the pipeline execution.
2025/01/14 22:39:48 - load_links - nr of transforms to run : 2  , nr of hops : 1
2025/01/14 22:39:48 - load_links - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
2025/01/14 22:39:48 - load_links - Not running a unit test...
2025/01/14 22:39:48 - load_links - Execution started for pipeline [load_links]
2025/01/14 22:39:48 - load_links - I found 2 different transforms to launch.
2025/01/14 22:39:48 - load_links - Allocating rowsets...
2025/01/14 22:39:48 - load_links -  Allocating rowsets for transform 0 --> Table input
2025/01/14 22:39:48 - load_links -   prevcopies = 1, nextcopies=1
2025/01/14 22:39:48 - load_links - Pipeline allocated new rowset [Table input.0 - Parquet File Output.0]
2025/01/14 22:39:48 - load_links -  Allocated 1 rowsets for transform 0 --> Table input
2025/01/14 22:39:48 - load_links -  Allocating rowsets for transform 1 --> Parquet File Output
2025/01/14 22:39:48 - load_links -  Allocated 1 rowsets for transform 1 --> Parquet File Output
2025/01/14 22:39:48 - load_links - Allocating Transforms & TransformData...
2025/01/14 22:39:48 - load_links -  Pipeline is about to allocate transform [Table input] of type [TableInput]
2025/01/14 22:39:48 - load_links -   Transform has nrcopies=1
2025/01/14 22:39:48 - Table input.0 - Starting allocation of buffers & new threads...
2025/01/14 22:39:48 - Table input.0 - Transform info: nrinput=0 nroutput=1
2025/01/14 22:39:48 - Table input.0 - output rel. is 1:1
2025/01/14 22:39:48 - Table input.0 - Found output rowset [Table input.0 - Parquet File Output.0]
2025/01/14 22:39:48 - Table input.0 - Finished dispatching
2025/01/14 22:39:48 - load_links -  Pipeline has allocated a new transform: [Table input].0
2025/01/14 22:39:48 - load_links -  Pipeline is about to allocate transform [Parquet File Output] of type [ParquetFileOutput]
2025/01/14 22:39:48 - load_links -   Transform has nrcopies=1
2025/01/14 22:39:49 - Parquet File Output.0 - Starting allocation of buffers & new threads...
2025/01/14 22:39:49 - Parquet File Output.0 - Transform info: nrinput=1 nroutput=0
2025/01/14 22:39:49 - Parquet File Output.0 - Got previous transform from [Parquet File Output] #0 --> Table input
2025/01/14 22:39:49 - Parquet File Output.0 - input rel is 1:1
2025/01/14 22:39:49 - Parquet File Output.0 - Found input rowset [Table input.0 - Parquet File Output.0]
2025/01/14 22:39:49 - Parquet File Output.0 - Finished dispatching
2025/01/14 22:39:49 - load_links -  Pipeline has allocated a new transform: [Parquet File Output].0
2025/01/14 22:39:49 - load_links - Initialising 2 transforms...
2025/01/14 22:39:49 - obo - New database connection defined
2025/01/14 22:39:49 - obo - Connecting to database using URL: jdbc:jtds:sqlserver://**********************
2025/01/14 22:39:49 - obo - Auto commit off
2025/01/14 22:39:49 - Table input.0 - Connected to database...
2025/01/14 22:39:49 - load_links - Transform [Table input.0] initialized flawlessly.
2025/01/14 22:39:49 - load_links - Transform [Parquet File Output.0] initialized flawlessly.
2025/01/14 22:39:49 - General - From transform: Table input, looking at 0 prev. transforms.
2025/01/14 22:39:49 - General - From transform: Table input, looking at 0 prev. transforms.
2025/01/14 22:39:49 - General - Getting fields from transform: Table input, type=TableInput
2025/01/14 22:39:49 - DbCache - The database cache doesn't exist yet.

GitHub link: https://github.com/apache/hop/discussions/4780

----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org

Re: [D] Pipeline hanged. (hop)

["bamaer (via GitHub)" <gi...@apache.org>]

GitHub user bamaer added a comment to the discussion: Pipeline hanged.

You don't really give us any information to work with. Check if your database returns data without the Parquet output, e.g. write to a dummy or a text file. If that works, check what could be wrong with your Parquet output configuration. If it doesn't, check your database... 

GitHub link: https://github.com/apache/hop/discussions/4780#discussioncomment-11842461

----
This is an automatically sent email for users@hop.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@hop.apache.org