You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "matianhe3 (via GitHub)" <gi...@apache.org> on 2024/01/09 07:50:39 UTC
[I] [Bug] [Seatunnel] Seatunnel work localfile sink, can not find the result file. [dolphinscheduler]
matianhe3 opened a new issue, #15442:
URL: https://github.com/apache/dolphinscheduler/issues/15442
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
```
env {
execution.parallelism = 1
job.mode = BATCH
job.name = pro_live
}
source {
Jdbc {
query = """
"""
}
}
transform {
}
sink {
LocalFile {
source_table_name = [source]
path = "/tmp/"
custom_filename = true
file_name_expression = "name-${now}_${transactionId}"
filename_time_format = "yyyy-MM-dd"
file_format_type = "excel"
}
}
```
### What you expected to happen
run seatunnel.sh -c my.conf
it works well. the flie in /tmp/.
but ds run successed, but no the file.
### How to reproduce
key 'plugin_name' instead of key 'factory'
2024-01-09 15:42:25,945 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load SeaTunnelSink Plugin from /opt/apache-seatunnel-2.3.3/connectors/seatunnel
2024-01-09 15:42:25,945 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: LocalFile at: file:/opt/apache-seatunnel-2.3.3/connectors/seatunnel/connector-file-local-2.3.3.jar
2024-01-09 15:42:25,949 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load plugin: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='LocalFile'} from classpath
2024-01-09 15:42:26,027 INFO org.apache.seatunnel.engine.client.job.ClientJobProxy - Start submit job, job id: 797007288542953482, with plugin jar [file:/opt/apache-seatunnel-2.3.3/connectors/seatunnel/connector-file-local-2.3.3.jar, file:/opt/apache-seatunnel-2.3.3/connectors/seatunnel/connector-jdbc-2.3.3.jar]
2024-01-09 15:42:26,078 INFO org.apache.seatunnel.engine.client.job.ClientJobProxy - Submit job finished, job id: 797007288542953482, job name: SeaTunnel
2024-01-09 15:42:26,084 WARN org.apache.seatunnel.engine.client.job.JobMetricsRunner - Failed to get job metrics summary, it maybe first-run
[INFO] 2024-01-09 15:42:30.448 +0800 - ->
2024-01-09 15:42:30,077 INFO org.apache.seatunnel.engine.client.job.ClientJobProxy - Job (797007288542953482) end with state FINISHED
2024-01-09 15:42:30,080 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand -
***********************************************
Job Statistic Information
***********************************************
Start Time : 2024-01-09 15:42:25
End Time : 2024-01-09 15:42:30
Total Time(s) : 4
Total Read Count : 91
Total Write Count : 91
Total Failed Count : 0
***********************************************
2024-01-09 15:42:30,081 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2024-01-09 15:42:30,084 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [192.168.16.209]:5801:281cc1b0-ce53-436a-a6dc-1ff5eb9000c8, connection: ClientConnection{alive=false, connectionId=2, channel=NioChannel{/192.168.16.213:41287->/192.168.16.209:5801}, remoteAddress=[192.168.16.209]:5801, lastReadTime=2024-01-09 15:42:30.078, lastWriteTime=2024-01-09 15:42:30.077, closedTime=2024-01-09 15:42:30.082, connected server version=5.1}
2024-01-09 15:42:30,085 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [192.168.16.210]:5801:590bbf42-a3ac-4b16-b662-6e68917e6181, connection: ClientConnection{alive=false, connectionId=3, channel=NioChannel{/192.168.16.213:57856->/192.168.16.210:5801}, remoteAddress=[192.168.16.210]:5801, lastReadTime=2024-01-09 15:42:25.267, lastWriteTime=2024-01-09 15:42:25.267, closedTime=2024-01-09 15:42:30.084, connected server version=5.1}
2024-01-09 15:42:30,086 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [192.168.16.213]:5801:dc82db18-b751-4369-8643-ede1854f7e45, connection: ClientConnection{alive=false, connectionId=4, channel=NioChannel{/192.168.16.213:39663->/192.168.16.213:5801}, remoteAddress=[192.168.16.213]:5801, lastReadTime=2024-01-09 15:42:25.269, lastWriteTime=2024-01-09 15:42:25.268, closedTime=2024-01-09 15:42:30.085, connected server version=5.1}
2024-01-09 15:42:30,087 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [192.168.16.212]:5801:481e5204-7ae0-44c2-9958-807516b87620, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/192.168.16.213:52968->/192.168.16.212:5801}, remoteAddress=[192.168.16.212]:5801, lastReadTime=2024-01-09 15:42:27.721, lastWriteTime=2024-01-09 15:42:25.158, closedTime=2024-01-09 15:42:30.086, connected server version=5.1}
2024-01-09 15:42:30,087 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2024-01-09 15:42:30,090 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2024-01-09 15:42:30,090 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed SeaTunnel client......
2024-01-09 15:42:30,091 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed metrics executor service ......
2024-01-09 15:42:30,092 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - run shutdown hook because get close signal
[INFO] 2024-01-09 15:42:30.449 +0800 - process has exited. execute path:/tmp/dolphinscheduler/exec/process/default/12161586465600/12161685541696_6/31/102, processId:0 ,exitStatusCode:0 ,processWaitForStatus:true ,processExitValue:0
[INFO] 2024-01-09 15:42:30.450 +0800 - ***********************************************************************************************
[INFO] 2024-01-09 15:42:30.450 +0800 - ********************************* Finalize task instance ************************************
[INFO] 2024-01-09 15:42:30.450 +0800 - ***********************************************************************************************
[INFO] 2024-01-09 15:42:30.451 +0800 - Upload output files: [] successfully
[INFO] 2024-01-09 15:42:30.452 +0800 - Send task execute status: SUCCESS to master : 192.168.16.50:1234
[INFO] 2024-01-09 15:42:30.452 +0800 - Remove the current task execute context from worker cache
[INFO] 2024-01-09 15:42:30.452 +0800 - The current execute mode isn't develop mode, will clear the task execute file: /tmp/dolphinscheduler/exec/process/default/12161586465600/12161685541696_6/31/102
[INFO] 2024-01-09 15:42:30.453 +0800 - Success clear the task execute file: /tmp/dolphinscheduler/exec/process/default/12161586465600/12161685541696_6/31/102
[INFO] 2024-01-09 15:42:30.454 +0800 - FINALIZE_SESSION
### Anything else
_No response_
### Version
3.2.x
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Seatunnel] Seatunnel work localfile sink, can not find the result file. [dolphinscheduler]
Posted by "matianhe3 (via GitHub)" <gi...@apache.org>.
matianhe3 closed issue #15442: [Bug] [Seatunnel] Seatunnel work localfile sink, can not find the result file.
URL: https://github.com/apache/dolphinscheduler/issues/15442
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org