You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/03/02 02:05:07 UTC
[GitHub] [incubator-seatunnel] BenJFan opened a new issue #1368: [Bug] [Connector] Clickhouse sink doesn't not handle sql output data types correctly
BenJFan opened a new issue #1368:
URL: https://github.com/apache/incubator-seatunnel/issues/1368
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
When use sql transform, like : " select 1.1 as number from table". The number 1.1 data type will be de decimal. But Clickhouse sink need number field treat as double/float. Decimal can't cast to double/float directly
Clickhouse DDL:
CREATE TABLE default.rowdata_all
(
`id` Int32,
`message` String,
`name` String,
`money` Nullable(Float32),
`datetime` DateTime
)
ENGINE = Distributed('cluster1', 'default', 'rowdata', xxHash64(name))
### SeaTunnel Version
2.0.5
### SeaTunnel Config
```conf
env {
# You can set spark configuration here
spark.app.name = "SeaTunnel"
spark.executor.instances = 2
spark.executor.cores = 1
spark.executor.memory = "1g"
spark.master = local
}
source {
Fake {
result_table_name = "my_dataset"
}
}
transform {
sql {
sql = "select 2 as id, 'data' as message, 'name' as name, 1.1 as money, '2011-11-11 11:11:22' as datetime from my_dataset"
}
}
sink {
# choose stdout output plugin to output data to console
ClickHouse {
host = "localhost:8123"
database = "default"
table = "rowdata_all"
username = "default"
password = "9999"
bulk_size = 20000
retry_codes = [209, 210]
retry = 3
}
}
```
### Running Command
```shell
default
```
### Error Exception
```log
Caused by: java.lang.ClassCastException: java.math.BigDecimal cannot be cast to java.lang.Float
at scala.runtime.BoxesRunTime.unboxToFloat(BoxesRunTime.java:109)
at org.apache.seatunnel.spark.sink.Clickhouse.org$apache$seatunnel$spark$sink$Clickhouse$$renderBaseTypeStatement(Clickhouse.scala:361)
at org.apache.seatunnel.spark.sink.Clickhouse$$anonfun$org$apache$seatunnel$spark$sink$Clickhouse$$renderStatement$1.apply$mcVI$sp(Clickhouse.scala:394)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
at org.apache.seatunnel.spark.sink.Clickhouse.org$apache$seatunnel$spark$sink$Clickhouse$$renderStatement(Clickhouse.scala:376)
at org.apache.seatunnel.spark.sink.Clickhouse$$anonfun$output$1$$anonfun$apply$1.apply(Clickhouse.scala:94)
at org.apache.seatunnel.spark.sink.Clickhouse$$anonfun$output$1$$anonfun$apply$1.apply(Clickhouse.scala:91)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at org.apache.seatunnel.spark.sink.Clickhouse$$anonfun$output$1.apply(Clickhouse.scala:91)
at org.apache.seatunnel.spark.sink.Clickhouse$$anonfun$output$1.apply(Clickhouse.scala:79)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2101)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2101)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:121)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:402)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:408)
```
### Flink or Spark Version
spark local
### Java or Scala Version
openjdk8
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] wuchunfu closed issue #1368: [Bug] [Connector] Clickhouse sink doesn't not handle sql output data types correctly
Posted by GitBox <gi...@apache.org>.
wuchunfu closed issue #1368:
URL: https://github.com/apache/incubator-seatunnel/issues/1368
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org