You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/06/07 00:03:32 UTC
[GitHub] [hudi] ykPrograWorld opened a new issue, #5774: Using flink sql to read kafka data to hudi fails
ykPrograWorld opened a new issue, #5774:
URL: https://github.com/apache/hudi/issues/5774
**_Tips before filing an issue_**
- Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)?
- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.
- If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.
**Describe the problem you faced**
Using flink sql to read kafka data to hudi fails
**To Reproduce**
Steps to reproduce the behavior:
1.kafka table sql:
CREATE TABLE t3_kafka_sour(
uuid STRING,
name STRING
) WITH (
'connector' = 'kafka',
'topic' = 't3',
'properties.bootstrap.servers' = 'localhost:9092',
'properties.group.id' = 't3',
'scan.startup.mode' = 'earliest-offset',
'format' = 'json',
'json.fail-on-missing-field' = 'false',
'json.ignore-parse-errors' = 'true'
);
2.hudi table sql:
CREATE TABLE test6(
uuid STRING PRIMARY KEY NOT ENFORCED,
name STRING,
part STRING)
PARTITIONED BY(part)
WITH (
'connector' = 'hudi',
'path'='hdfs:///user/data/hudi/test6',
'table.typ'='COPY_ON_WRITE',
'hoodie.datasource.write.recordkey.field'='uuid',
'write.tasks'='1',
'write.rate.limit'='2000',
'compaction.tasks'='1',
'compaction.async.enabled'='true',
'compaction.trigger.strategy'='time_elapsed',
'compaction.delta_commits'='1',
'changelog.enabled'= 'true',
'compaction.timeout.seconds'='1200',
'hoodie.compact.inline.max.delta.seconds'='1'
);
3.
INSERT INTO test6 SELECT *,CAST(CURRENT_DATE AS STRING) AS part FROM t3_kafka_sour;
**Expected behavior**
A clear and concise description of what you expected to happen.
**Environment Description**
* Hudi version :
* Spark version :
* Hive version :
* Hadoop version :
* Storage (HDFS/S3/GCS..) :
* Running on Docker? (yes/no) :
**Additional context**
Add any other context about the problem here.
**Stacktrace**
```Add the stacktrace of the error.```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LinMingQiang commented on issue #5774: [SUPPORT]Using flink sql to read kafka data to hudi fails
Posted by GitBox <gi...@apache.org>.
LinMingQiang commented on issue #5774:
URL: https://github.com/apache/hudi/issues/5774#issuecomment-1148100504
Did you open checkpoint?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #5774: [SUPPORT]Using flink sql to read kafka data to hudi fails
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #5774:
URL: https://github.com/apache/hudi/issues/5774#issuecomment-1149119047
@wangxianghu @danny0405 : can you loop in someone to assist here please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] sagarkoo commented on issue #5774: [SUPPORT]Using flink sql to read kafka data to hudi fails
Posted by GitBox <gi...@apache.org>.
sagarkoo commented on issue #5774:
URL: https://github.com/apache/hudi/issues/5774#issuecomment-1171532625
Hi,
I am getting the same problem. How is the solved??
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] ykPrograWorld commented on issue #5774: [SUPPORT]Using flink sql to read kafka data to hudi fails
Posted by GitBox <gi...@apache.org>.
ykPrograWorld commented on issue #5774:
URL: https://github.com/apache/hudi/issues/5774#issuecomment-1149330777
thks it wroks when i open checkponiht
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] ykPrograWorld commented on issue #5774: [SUPPORT]Using flink sql to read kafka data to hudi fails
Posted by GitBox <gi...@apache.org>.
ykPrograWorld commented on issue #5774:
URL: https://github.com/apache/hudi/issues/5774#issuecomment-1171901228
> Hi,
>
> I am getting the same problem. How is the solved??
open checkpoint
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] ykPrograWorld closed issue #5774: [SUPPORT]Using flink sql to read kafka data to hudi fails
Posted by GitBox <gi...@apache.org>.
ykPrograWorld closed issue #5774: [SUPPORT]Using flink sql to read kafka data to hudi fails
URL: https://github.com/apache/hudi/issues/5774
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org