You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "matianhe3 (via GitHub)" <gi...@apache.org> on 2024/01/09 07:50:39 UTC

[I] [Bug] [Seatunnel] Seatunnel work localfile sink, can not find the result file. [dolphinscheduler]

matianhe3 opened a new issue, #15442:
URL: https://github.com/apache/dolphinscheduler/issues/15442

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What happened
   
   ```
   env {
     execution.parallelism = 1
     job.mode = BATCH
     job.name = pro_live
   }
   
   source {
     Jdbc {
       query = """
   
       """
     }
   }
   
   transform {
   }
   
   sink {
     LocalFile {
       source_table_name = [source]
       path = "/tmp/"
       custom_filename = true
       file_name_expression = "name-${now}_${transactionId}"
       filename_time_format = "yyyy-MM-dd"
       file_format_type = "excel"
     }
   }
   ```
   
   ### What you expected to happen
   
   run seatunnel.sh -c my.conf
   it works well. the flie in /tmp/.
   but ds run successed, but no the file.
   
   ### How to reproduce
   
   key 'plugin_name' instead of key 'factory'
   
   	2024-01-09 15:42:25,945 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load SeaTunnelSink Plugin from /opt/apache-seatunnel-2.3.3/connectors/seatunnel
   
   	2024-01-09 15:42:25,945 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: LocalFile at: file:/opt/apache-seatunnel-2.3.3/connectors/seatunnel/connector-file-local-2.3.3.jar
   
   	2024-01-09 15:42:25,949 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load plugin: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='LocalFile'} from classpath
   
   	2024-01-09 15:42:26,027 INFO  org.apache.seatunnel.engine.client.job.ClientJobProxy - Start submit job, job id: 797007288542953482, with plugin jar [file:/opt/apache-seatunnel-2.3.3/connectors/seatunnel/connector-file-local-2.3.3.jar, file:/opt/apache-seatunnel-2.3.3/connectors/seatunnel/connector-jdbc-2.3.3.jar]
   
   	2024-01-09 15:42:26,078 INFO  org.apache.seatunnel.engine.client.job.ClientJobProxy - Submit job finished, job id: 797007288542953482, job name: SeaTunnel
   
   	2024-01-09 15:42:26,084 WARN  org.apache.seatunnel.engine.client.job.JobMetricsRunner - Failed to get job metrics summary, it maybe first-run
   
   [INFO] 2024-01-09 15:42:30.448 +0800 -  -> 
   
   	2024-01-09 15:42:30,077 INFO  org.apache.seatunnel.engine.client.job.ClientJobProxy - Job (797007288542953482) end with state FINISHED
   
   	2024-01-09 15:42:30,080 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - 
   
   	***********************************************
   
   	           Job Statistic Information
   
   	***********************************************
   
   	Start Time                : 2024-01-09 15:42:25
   
   	End Time                  : 2024-01-09 15:42:30
   
   	Total Time(s)             :                   4
   
   	Total Read Count          :                  91
   
   	Total Write Count         :                  91
   
   	Total Failed Count        :                   0
   
   	***********************************************
   
   	
   
   	2024-01-09 15:42:30,081 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
   
   	2024-01-09 15:42:30,084 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [192.168.16.209]:5801:281cc1b0-ce53-436a-a6dc-1ff5eb9000c8, connection: ClientConnection{alive=false, connectionId=2, channel=NioChannel{/192.168.16.213:41287->/192.168.16.209:5801}, remoteAddress=[192.168.16.209]:5801, lastReadTime=2024-01-09 15:42:30.078, lastWriteTime=2024-01-09 15:42:30.077, closedTime=2024-01-09 15:42:30.082, connected server version=5.1}
   
   	2024-01-09 15:42:30,085 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [192.168.16.210]:5801:590bbf42-a3ac-4b16-b662-6e68917e6181, connection: ClientConnection{alive=false, connectionId=3, channel=NioChannel{/192.168.16.213:57856->/192.168.16.210:5801}, remoteAddress=[192.168.16.210]:5801, lastReadTime=2024-01-09 15:42:25.267, lastWriteTime=2024-01-09 15:42:25.267, closedTime=2024-01-09 15:42:30.084, connected server version=5.1}
   
   	2024-01-09 15:42:30,086 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [192.168.16.213]:5801:dc82db18-b751-4369-8643-ede1854f7e45, connection: ClientConnection{alive=false, connectionId=4, channel=NioChannel{/192.168.16.213:39663->/192.168.16.213:5801}, remoteAddress=[192.168.16.213]:5801, lastReadTime=2024-01-09 15:42:25.269, lastWriteTime=2024-01-09 15:42:25.268, closedTime=2024-01-09 15:42:30.085, connected server version=5.1}
   
   	2024-01-09 15:42:30,087 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [192.168.16.212]:5801:481e5204-7ae0-44c2-9958-807516b87620, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/192.168.16.213:52968->/192.168.16.212:5801}, remoteAddress=[192.168.16.212]:5801, lastReadTime=2024-01-09 15:42:27.721, lastWriteTime=2024-01-09 15:42:25.158, closedTime=2024-01-09 15:42:30.086, connected server version=5.1}
   
   	2024-01-09 15:42:30,087 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
   
   	2024-01-09 15:42:30,090 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
   
   	2024-01-09 15:42:30,090 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed SeaTunnel client......
   
   	2024-01-09 15:42:30,091 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed metrics executor service ......
   
   	2024-01-09 15:42:30,092 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - run shutdown hook because get close signal
   
   [INFO] 2024-01-09 15:42:30.449 +0800 - process has exited. execute path:/tmp/dolphinscheduler/exec/process/default/12161586465600/12161685541696_6/31/102, processId:0 ,exitStatusCode:0 ,processWaitForStatus:true ,processExitValue:0
   
   [INFO] 2024-01-09 15:42:30.450 +0800 - ***********************************************************************************************
   
   [INFO] 2024-01-09 15:42:30.450 +0800 - *********************************  Finalize task instance  ************************************
   
   [INFO] 2024-01-09 15:42:30.450 +0800 - ***********************************************************************************************
   
   [INFO] 2024-01-09 15:42:30.451 +0800 - Upload output files: [] successfully
   
   [INFO] 2024-01-09 15:42:30.452 +0800 - Send task execute status: SUCCESS to master : 192.168.16.50:1234
   
   [INFO] 2024-01-09 15:42:30.452 +0800 - Remove the current task execute context from worker cache
   
   [INFO] 2024-01-09 15:42:30.452 +0800 - The current execute mode isn't develop mode, will clear the task execute file: /tmp/dolphinscheduler/exec/process/default/12161586465600/12161685541696_6/31/102
   
   [INFO] 2024-01-09 15:42:30.453 +0800 - Success clear the task execute file: /tmp/dolphinscheduler/exec/process/default/12161586465600/12161685541696_6/31/102
   
   [INFO] 2024-01-09 15:42:30.454 +0800 - FINALIZE_SESSION
   
   
   ### Anything else
   
   _No response_
   
   ### Version
   
   3.2.x
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Seatunnel] Seatunnel work localfile sink, can not find the result file. [dolphinscheduler]

Posted by "matianhe3 (via GitHub)" <gi...@apache.org>.
matianhe3 closed issue #15442: [Bug] [Seatunnel] Seatunnel work localfile sink, can not find the result file.
URL: https://github.com/apache/dolphinscheduler/issues/15442


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org