You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/06/30 02:13:35 UTC
[GitHub] [pulsar] xjfeng306602097 opened a new issue, #16291: Exception occurs when runs clickhouse sink
xjfeng306602097 opened a new issue, #16291:
URL: https://github.com/apache/pulsar/issues/16291
**Describe the bug**
I use standalone mode to start pulsar and plan to localrun sink to clickhouse, but when after running command './bin/pulsar-admin sinks localrun --archive ./connectors/pulsar-io-jdbc-clickhouse-2.10.0.nar --inputs public/default/mail_stat_goods_click_sink --name pulsar-clickhouse-goods-click-jdbc-sink --sink-config-file ./connectors/pulsar-clickhouse-goods-click-jdbc-sink.yaml --parallelism 1',
occurs an exception of 'java.sql.SQLFeatureNotSupportedException: Transactions are not supported'
**To Reproduce**
Steps to reproduce the behavior:
1.before reproduce, you should add the schema
```shell
./bin/pulsar-admin schemas upload mail_stat_goods_click_sink -f ./connectors/goods-click.json
```
2.schema
```json
{
"type": "AVRO",
"schema": "{\"type\":\"record\",\"name\":\"GoodClickRecord\",\"namespace\":\"com.makro.mall.stat.pojo.entity\",\"fields\":[{\"name\":\"channel\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"goodsCode\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"isNew\",\"type\":[\"null\",\"int\"],\"default\":null},{\"name\":\"memberNo\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"memberType\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"mmCode\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"pageNo\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"publishType\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"storeCode\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"bizId\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"browser\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"browserVersion\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\
":\"engine\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"engineVersion\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"event\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"eventDate\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"Date\",\"namespace\":\"java.util\",\"fields\":[]}],\"default\":null},{\"name\":\"ip\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"mobile\",\"type\":[\"null\",\"int\"],\"default\":null},{\"name\":\"os\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"osVersion\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"platform\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"referer\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"totalCount\",\"type\":[\"null\",\"long\"],\"default\":null},{\"name\":\"ts\",\"type\":[\"null\",\"long\"],\"default\":null},{\"name\":\"userAgent\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"uuid\",\"
type\":[\"null\",\"string\"],\"default\":null}]}",
"properties": {}
}
```
3.localrun the sink to clickhouse
```shell
./bin/pulsar-admin sinks localrun --archive ./connectors/pulsar-io-jdbc-clickhouse-2.10.0.nar --inputs public/default/mail_stat_goods_click_sink --name pulsar-clickhouse-goods-click-jdbc-sink --sink-config-file ./connectors/pulsar-clickhouse-goods-click-jdbc-sink.yaml --parallelism 1
```
4. See error
**Expected behavior**
It can successfully run the sink to clickhouse, I think the problem is **JdbcAbstractSink** setAutoCommit(false) leads to the exception, cause clickhouse doesn't support transactions, in the jdbc connection class you can see it throws **'java.sql.SQLFeatureNotSupportedException: Transactions are not supported'** while setting autocommit false, I wonder if we can set this option in config file to avoid this error? Such as settting autocommit:true
**Screenshots**
**Desktop (please complete the following information):**
- OS: Centos 8, Docker
**Additional context**
Add any other context about the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #16291: Exception occurs when runs clickhouse sink
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #16291: Exception occurs when runs clickhouse sink
URL: https://github.com/apache/pulsar/issues/16291
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org