You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "SeanZhang2021 (via GitHub)" <gi...@apache.org> on 2023/04/16 13:14:39 UTC
[GitHub] [incubator-seatunnel] SeanZhang2021 opened a new issue, #4584: [Bug] [oss-connector]seatunnel-hadoop3-3.1.4-uber-2.3.1-optional.jar读取/写入oss报类加载器冲突问题
SeanZhang2021 opened a new issue, #4584:
URL: https://github.com/apache/incubator-seatunnel/issues/4584
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
试用本地运行oss功能报错
### SeaTunnel Version
2.3.1
### SeaTunnel Config
```conf
env {
execution.parallelism = 1
job.mode = "BATCH"
}
source {
OssFile {
path = "/seatunnel/sink"
bucket = "xxxxxxxxx"
access_key = "xxxxxxxxx"
access_secret = "xxxxxxxxx"
endpoint = "xxxxxxxxx"
file_format_type = "csv"
delimiter = "#"
schema {
fields {
name1 = string
name2 = string
name3 = string
name4 = string
name5 = string
name6 = string
}
}
}
}
transform {
}
sink {
OssFile {
path = "/seatunnel/sink"
bucket = "xxxxxxx"
access_key = "xxxxxxx"
access_secret = "xxxxxxx"
endpoint = "xxxxxxx"
file_format_type = "csv"
}
}
```
### Running Command
```shell
sh /root/install/apache-seatunnel-incubating-2.3.1/bin/seatunnel.sh --config /root/install/apache-seatunnel-incubating-2.3.1/config/test.conf -e local
```
### Error Exception
```log
2023-04-16 20:31:19,863 ERROR org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (699960036822089729), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-OssFile-default]-SourceTask (1/1)] end with state FAILED and Exception: java.lang.LinkageError: loader constraint violation: when resolving method "org.apache.hadoop.util.SemaphoredDelegatingExecutor.<init>(Lcom/google/common/util/concurrent/ListeningExecutorService;IZ)V" the class loader (instance of org/apache/seatunnel/engine/common/loader/SeaTunnelChildFirstClassLoader) of the current class, org/apache/hadoop/fs/aliyun/oss/AliyunOSSFileSystem, and the class loader (instance of sun/misc/Launcher$AppClassLoader) for the method's defining class, org/apache/hadoop/util/SemaphoredDelegatingExecutor, have different Class objects for the type com/google/common/util/concurrent/ListeningExecutorService used in the signature
at org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem.open(AliyunOSSFileSystem.java:584)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:899)
at org.apache.seatunnel.connectors.seatunnel.file.source.reader.TextReadStrategy.read(TextReadStrategy.java:66)
at org.apache.seatunnel.connectors.seatunnel.file.source.BaseFileSourceReader.pollNext(BaseFileSourceReader.java:66)
at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:135)
at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:84)
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:161)
at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:89)
at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:526)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
```
### Flink or Spark Version
_No response_
### Java or Scala Version
openjdk version "1.8.0_362"
OpenJDK Runtime Environment (build 1.8.0_362-b08)
OpenJDK 64-Bit Server VM (build 25.362-b08, mixed mode)
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] SeanZhang2021 closed issue #4584: [oss-connector]seatunnel-hadoop3-3.1.4-uber-2.3.1-optional.jar读取/写入oss报类加载器冲突问题
Posted by "SeanZhang2021 (via GitHub)" <gi...@apache.org>.
SeanZhang2021 closed issue #4584: [oss-connector]seatunnel-hadoop3-3.1.4-uber-2.3.1-optional.jar读取/写入oss报类加载器冲突问题
URL: https://github.com/apache/incubator-seatunnel/issues/4584
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] SeanZhang2021 commented on issue #4584: [Bug] [oss-connector]seatunnel-hadoop3-3.1.4-uber-2.3.1-optional.jar读取/写入oss报类加载器冲突问题
Posted by "SeanZhang2021 (via GitHub)" <gi...@apache.org>.
SeanZhang2021 commented on issue #4584:
URL: https://github.com/apache/incubator-seatunnel/issues/4584#issuecomment-1510388903
lib下手动放入hadoop-aliyun-2.7.2.jar解决
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org