You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "ffayr (via GitHub)" <gi...@apache.org> on 2023/04/04 06:55:59 UTC
[GitHub] [incubator-seatunnel] ffayr opened a new issue, #4497: [Bug] [Source Http] HTTP Source read data "params" setting is invalid
ffayr opened a new issue, #4497:
URL: https://github.com/apache/incubator-seatunnel/issues/4497
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
接口传入时间参数，去数据库查询日期大于参数的数据返回，参数类型String。
脚本运行后seatunnel发送的查询请求没有参数信息，导致查不到数据，且一直重试。
The time parameter is passed in to the interface, and the data returned by querying the database with a date greater than the parameter, the parameter type is String. After the script runs, the query request sent by seatunnel has no parameter information, so the data cannot be found, and it keeps retrying
### SeaTunnel Version
2.3.1
### SeaTunnel Config
```conf
env {
execution.parallelism = 1
job.mode = "STREAMING"
}
source {
Http {
result_table_name = "mq"
url = "http://192.168.10.181:9000/test/"
method = "POST"
format = "json"
params = {sync_time="2023-03-23"}
poll_interval_ms = 3000
retry_backoff_multiplier_ms = 1000
retry_backoff_max_ms = 10000
json_field = {
id_card = "$.data[*].idCard"
json_data = "$.data[*]"
}
schema = {
fields{
id_card = string
json_data = string
}
}
}
}
sink {
Clickhouse {
source_table_name = "mq"
host = "xx:8123"
username = "xxx"
password = "xxx"
database = "db2"
table = "xx"
# cdc options
primary_key = "id_card"
support_upsert = true
allow_experimental_lightweight_delete = true
}
}
```
### Running Command
```shell
seatunnel.sh --config soctest.conf -e local
```
### Error Exception
```log
2023-04-04 14:44:31,116 ERROR org.apache.seatunnel.connectors.seatunnel.http.source.HttpSourceReader - http client execute exception, http response status code:[422], content:[{"detail":[{"loc":["query","sync_time"],"msg":"field required","type":"value_error.missing"}]}]
2023-04-04 14:44:31,118 ERROR org.apache.seatunnel.connectors.seatunnel.http.source.HttpSourceReader - http client execute exception, http response status code:[422], content:[{"detail":[{"loc":["query","sync_time"],"msg":"field required","type":"value_error.missing"}]}]
2023-04-04 14:44:31,121 ERROR org.apache.seatunnel.connectors.seatunnel.http.source.HttpSourceReader - http client execute exception, http response status code:[422], content:[{"detail":[{"loc":["query","sync_time"],"msg":"field required","type":"value_error.missing"}]}]
2023-04-04 14:44:31,123 ERROR org.apache.seatunnel.connectors.seatunnel.http.source.HttpSourceReader - http client execute exception, http response status code:[422], content:[{"detail":[{"loc":["query","sync_time"],"msg":"field required","type":"value_error.missing"}]}]
2023-04-04 14:44:31,126 ERROR org.apache.seatunnel.connectors.seatunnel.http.source.HttpSourceReader - http client execute exception, http response status code:[422], content:[{"detail":[{"loc":["query","sync_time"],"msg":"field required","type":"value_error.missing"}]}]
2023-04-04 14:44:31,129 ERROR org.apache.seatunnel.connectors.seatunnel.http.source.HttpSourceReader - http client execute exception, http response status code:[422], content:[{"detail":[{"loc":["query","sync_time"],"msg":"field required","type":"value_error.missing"}]}]
2023-04-04 14:44:31,132 ERROR org.apache.seatunnel.connectors.seatunnel.http.source.HttpSourceReader - http client execute exception, http response status code:[422], content:[{"detail":[{"loc":["query","sync_time"],"msg":"field required","type":"value_error.missing"}]}]
```
### Flink or Spark Version
### Java or Scala Version
jdk 1.8
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4497: [Bug] [Source Http] HTTP Source read data "params" setting is invalid
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4497:
URL: https://github.com/apache/incubator-seatunnel/issues/4497#issuecomment-1537251911
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] MonsterChenzhuo commented on issue #4497: [Bug] [Source Http] HTTP Source read data "params" setting is invalid
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo commented on issue #4497:
URL: https://github.com/apache/incubator-seatunnel/issues/4497#issuecomment-1498422680
please assign to me; thx。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] ocean-zhc commented on issue #4497: [Bug] [Source Http] HTTP Source read data "params" setting is invalid
Posted by "ocean-zhc (via GitHub)" <gi...@apache.org>.
ocean-zhc commented on issue #4497:
URL: https://github.com/apache/seatunnel/issues/4497#issuecomment-1663435592
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org