You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "liushuaihang (via GitHub)" <gi...@apache.org> on 2023/09/14 02:57:07 UTC
[GitHub] [seatunnel] liushuaihang opened a new issue, #5484: seatunnel 抽取 hive 指定分区数据到 MySQL
liushuaihang opened a new issue, #5484:
URL: https://github.com/apache/seatunnel/issues/5484
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
配置文件应该没有问题，但是跑不通
### SeaTunnel Version
seatunnel版本2.3.3
hive版本2.3.9
### SeaTunnel Config
```conf
source {
Hive {
table_name = "default.test3"
metastore_uri = "thrift://xxx:9083"
hdfs_site_path = "/home/hadoop/hadoop-3.2.2/etc/hadoop/hdfs-site.xml"
hive_site_path = "/home/hadoop/hive-2.3.9/conf/hive-site.xml"
read_partitions = ["dt=2023-09-13"]
}
}
```
### Running Command
```shell
./bin/seatunnel.sh --config config/hive_partition_test.config -e local
```
### Error Exception
```log
Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[FILE-04], ErrorDescript
ion:[File list is empty] - The target file list is empty,SeaTunnel will not be able to sync empty table, please check the conf
iguration parameters such as: [file_filter_pattern]
at org.apache.seatunnel.connectors.seatunnel.file.source.reader.AbstractReadStrategy.getFileNamesByPath(AbstractReadSt
rategy.java:158)
at org.apache.seatunnel.connectors.seatunnel.file.source.reader.AbstractReadStrategy.getFileNamesByPath(AbstractReadSt
rategy.java:133)
at org.apache.seatunnel.connectors.seatunnel.file.hdfs.source.BaseHdfsFileSource.prepare(BaseHdfsFileSource.java:76)
at org.apache.seatunnel.connectors.seatunnel.hive.source.HiveSource.prepare(HiveSource.java:176)
at org.apache.seatunnel.engine.core.parse.JobConfigParser.parseSource(JobConfigParser.java:85)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:3
17)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:179)
at org.apache.seatunnel.engine.core.job.AbstractJobEnvironment.getLogicalDag(AbstractJobEnvironment.java:109)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:73)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:143)
```
### Zeta or Flink or Spark Version
_No response_
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] ruanwenjun commented on issue #5484: Cannot read a specific partition data in hive
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on issue #5484:
URL: https://github.com/apache/seatunnel/issues/5484#issuecomment-1720376418
I closed this issue if there is no more thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] liushuaihang commented on issue #5484: Cannot read a specific partition data in hive
Posted by "liushuaihang (via GitHub)" <gi...@apache.org>.
liushuaihang commented on issue #5484:
URL: https://github.com/apache/seatunnel/issues/5484#issuecomment-1723024137
> @liushuaihang you can use seatunnel 2.3.2.
Hi，关于这个BUG，同样的配置，seatunnel2.3.3不能跑成功，seatunnel2.3.1可以跑成功
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] liushuaihang commented on issue #5484: seatunnel 抽取 hive 指定分区数据到 MySQL
Posted by "liushuaihang (via GitHub)" <gi...@apache.org>.
liushuaihang commented on issue #5484:
URL: https://github.com/apache/seatunnel/issues/5484#issuecomment-1718689582
hive的dt是string类型
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] liunaijie commented on issue #5484: Cannot read a specific partition data in hive
Posted by "liunaijie (via GitHub)" <gi...@apache.org>.
liunaijie commented on issue #5484:
URL: https://github.com/apache/seatunnel/issues/5484#issuecomment-1724629581
> > @liushuaihang you can use seatunnel 2.3.2.
>
> Hi，关于这个BUG，同样的配置，seatunnel2.3.3不能跑成功，seatunnel2.3.1可以跑成功
Yes, as i said, this bug was introduced in 2.3.3, and will be fix in next version. So you need use another version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] ruanwenjun closed issue #5484: Cannot read a specific partition data in hive
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun closed issue #5484: Cannot read a specific partition data in hive
URL: https://github.com/apache/seatunnel/issues/5484
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] liushuaihang commented on issue #5484: Cannot read a specific partition data in hive
Posted by "liushuaihang (via GitHub)" <gi...@apache.org>.
liushuaihang commented on issue #5484:
URL: https://github.com/apache/seatunnel/issues/5484#issuecomment-1723022391
Hi，关于这个BUG，同样的配置，seatunnel2.3.3不能跑成功，seatunnel2.3.1可以跑成功
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] liunaijie commented on issue #5484: Cannot read a specific partition data in hive
Posted by "liunaijie (via GitHub)" <gi...@apache.org>.
liunaijie commented on issue #5484:
URL: https://github.com/apache/seatunnel/issues/5484#issuecomment-1719043230
this is an issue, if some path,subpath doesn't have any file will get this error.
fixed at https://github.com/apache/seatunnel/pull/5427
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] ruanwenjun commented on issue #5484: seatunnel 抽取 hive 指定分区数据到 MySQL
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on issue #5484:
URL: https://github.com/apache/seatunnel/issues/5484#issuecomment-1718706373
Could you please check if your table exist and hive config is correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] liunaijie commented on issue #5484: Cannot read a specific partition data in hive
Posted by "liunaijie (via GitHub)" <gi...@apache.org>.
liunaijie commented on issue #5484:
URL: https://github.com/apache/seatunnel/issues/5484#issuecomment-1719064401
@liushuaihang you can use seatunnel 2.3.2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org