You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/09/20 09:26:12 UTC
[GitHub] [iceberg] zixiao144 opened a new issue, #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
zixiao144 opened a new issue, #5803:
URL: https://github.com/apache/iceberg/issues/5803
### Apache Iceberg version
0.14.0 (latest release)
### Query engine
Flink
### Please describe the bug 🐞
SET execution.runtime-mode = streaming;
select * from test /*+ OPTIONS('streaming'='true', 'monitor-interval'='1s')*/
no result,it is always pending, bath is normal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 closed issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 closed issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
URL: https://github.com/apache/iceberg/issues/5803
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1258830760
2022-09-27 08:19:21,864 INFO org.apache.flink.api.java.typeutils.TypeExtractor [] - class org.apache.iceberg.flink.source.FlinkInputSplit does not contain a getter for field task
2022-09-27 08:19:21,864 INFO org.apache.flink.api.java.typeutils.TypeExtractor [] - class org.apache.iceberg.flink.source.FlinkInputSplit does not contain a setter for field task
2022-09-27 08:19:21,864 INFO org.apache.flink.api.java.typeutils.TypeExtractor [] - Class class org.apache.iceberg.flink.source.FlinkInputSplit cannot be used as a POJO type because not all fields are valid POJO fields, and must be processed as GenericType. Please read the Flink documentation on "Data Types & Serialization" for details of the effect on performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1253094839
I must be use sql，I use pyflink。How to do deal with it. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1253180973
begin ：
and
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1260249202
thanks. cdc doesn't supprot upsert.I disable iceberg's option . it can work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1504306707
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1252565391
BTW, the FLIP-27 Flink Iceberg source is also merged in the master branch. it is available in 0.14.0 (except for the SQL connector commit and a metrics fix commit). If you are interested in giving it a try, you can refer to the usage in the [unit test code](https://github.com/apache/iceberg/blob/master/flink/v1.15/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceContinuous.java).
The benefit of FLIP-27 source for streaming is that it uses pull-based split assignment, which is good for dynamic split assignment and work sharing. Current FlinkSource would assign all discovered splits eagerly to all reader subtasks in a round-robin fashion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1259759074
Ah. You are doing CDC write to Iceberg. For the streaming read out of Iceberg CDC table, you want to read rows out with proper row type (like insert, delete, etc.). Then this is expected as the current Flink streaming read doesn't support upsert/row-level deletes.
Anton has been working on supporting CDC reads from Iceberg core and Spark. iceberg-flink can leverage the new APIs in Iceberg-api/core. don't think there is anyone picking it up from Flink side yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1252518941
@zixiao144 maybe a dumb question. is there any snapshots after the startSnapshotId?
also can you enable DEBUG level logging for `StreamingMonitorFunction` class? then we can see the split discovery logs. e.g.
```
LOG.debug(
"Start discovering splits from {} (exclusive) to {} (inclusive)",
lastSnapshotId,
snapshotId);
long start = System.currentTimeMillis();
FlinkInputSplit[] splits =
FlinkSplitPlanner.planInputSplits(table, newScanContext, workerPool);
LOG.debug(
"Discovered {} splits, time elapsed {}ms",
splits.length,
System.currentTimeMillis() - start);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1253153847
@zixiao144 stream read will discover new snapshots with data files appended. if you launch a Flink SQL job with streaming read, you would need to insert new rows to the stable for the streaming SQL jobs to discover. By default, the streaming read starts from the latest snapshot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1486028382
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1257568234
[jobmanager_log.txt](https://github.com/apache/iceberg/files/9643722/jobmanager_log.txt)
[flink-root-taskexecutor-0-violet.log](https://github.com/apache/iceberg/files/9643725/flink-root-taskexecutor-0-violet.log)
[flink--sql-client-violet.log](https://github.com/apache/iceberg/files/9643728/flink--sql-client-violet.log)
[taskmanager_localhost_44203-b19935_log.txt](https://github.com/apache/iceberg/files/9643731/taskmanager_localhost_44203-b19935_log.txt)
I use flink1.15.2 ,iceberg 0.14.1 Flink 1.15 runtime Jar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1258358115
the jobmanager log doesn't contain any log lines from `org.apache.iceberg`. is the logging level set properly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1259027115
create table by the way
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1252087194
it is always pending, no result
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1257489258
Can you provide the log file for Flink jobmanager with DEBUG enabled for `org.apache.iceberg.flink.source.StreamingMonitorFunction`? Then we can check the periodical split discovery for the streaming execution.
For FLIP-27 source, SQL is not available in 0.14.1. Did you try it with build from master branch? If yes, we also need jobmanager log file (with INFO level) to investigate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1252346901
2022-09-20 21:15:43,205 INFO org.apache.flink.api.java.typeutils.TypeExtractor [] - class org.apache.iceberg.flink.source.FlinkInputSplit does not contain a getter for field task 2022-09-20 21:15:43,206 INFO org.apache.flink.api.java.typeutil
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1253054979
@stevenzwu
I use flink1.15.2's sqlclient.sh .Do simple test.
SET execution.runtime-mode = streaming;
SET table.dynamic-table-options.enabled=true;
or (SET table.exec.iceberg.use-flip27-source = true;)
select * from test /*+ OPTIONS('streaming'='true', 'monitor-interval'='1s')*/ ;
streaming result:
SET execution.runtime-mode = batch;
batch result:
batch is normal ,but streaming is not normal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1258817527
[flink-root-sql-client-violet.log](https://github.com/apache/iceberg/files/9650453/flink-root-sql-client-violet.log)
[jobmanager_log (1).txt](https://github.com/apache/iceberg/files/9650457/jobmanager_log.1.txt)
[flink-root-taskexecutor-0-violet (1).log](https://github.com/apache/iceberg/files/9650458/flink-root-taskexecutor-0-violet.1.log)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1258984740
@zixiao144 yes, saw the logs in taskmanager log file. I forgot the the `StreamingMonitorFunction` runs on taskmanagers. Flink streaming read only discovers `append` snapshot. How did you append data to the table? it seems that Flink source see the snapshots but didn't find any splits from snapshots with `append` operation.
Can you run java API to check the snapshot history and the `operation` type of the `Snapshot`? Alternatively, you can run Spark SQL to query the metadata table: https://iceberg.apache.org/docs/latest/spark-queries/#snapshots.
```
2022-09-27 00:19:28,682 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from -1 (exclusive) to 6496244702617183271 (inclusive)
2022-09-27 00:19:40,565 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 11883ms
2022-09-27 00:19:40,566 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 1ms
2022-09-27 00:19:57,703 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 6496244702617183271 (exclusive) to 2877339576078753525 (inclusive)
2022-09-27 00:19:57,704 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 1ms
2022-09-27 00:19:57,705 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms
2022-09-27 00:20:02,759 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 2877339576078753525 (exclusive) to 9205452674806088860 (inclusive)
2022-09-27 00:20:02,759 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 0ms
2022-09-27 00:20:02,759 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms
2022-09-27 00:20:52,053 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 9205452674806088860 (exclusive) to 85589348609116719 (inclusive)
2022-09-27 00:20:52,070 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 17ms
2022-09-27 00:20:52,070 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms
2022-09-27 00:21:42,325 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 85589348609116719 (exclusive) to 519180963154444509 (inclusive)
2022-09-27 00:21:42,345 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 20ms
2022-09-27 00:21:42,345 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms
2022-09-27 00:22:32,575 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 519180963154444509 (exclusive) to 2413530983076443232 (inclusive)
2022-09-27 00:22:32,595 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 20ms
2022-09-27 00:22:32,595 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms
2022-09-27 00:23:21,804 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Start discovering splits from 2413530983076443232 (exclusive) to 5161337709432232128 (inclusive)
2022-09-27 00:23:21,819 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 15ms
2022-09-27 00:23:21,819 DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Forwarded 0 splits, time elapsed 0ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] github-actions[bot] closed issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
URL: https://github.com/apache/iceberg/issues/5803
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1258820165
in [flink-root-taskexecutor](https://github.com/apache/iceberg/files/9650458/flink-root-taskexecutor-0-violet.1.log)
log lines from org.apache.iceberg .
2022-09-27 00:19:28,670 DEBUG org.apache.flink.streaming.runtime.tasks.StreamTask [] - Legacy source Source: Iceberg table (iotCatalog.iotdw.test) monitor (1/1)#0 (d77ec0c3c3c1493cb4a6a506b8648be3) skip execution since the task is finished on restore
DEBUG org.apache.iceberg.flink.source.StreamingMonitorFunction [] - Discovered 0 splits, time elapsed 11883ms
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1258827219
I opened debug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] kingwind94 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
kingwind94 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1258862631
Is it working well in flink 1.15.0 instead of 1.15.2? The default flink version for iceberg 0.14.0 is 1.15.0. And I have another wired issue with 1.14.3 in iceberg 0.13.2. The small version fix might cause a problem. Can some developer answer the question?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1259025578
I use flink cdc to iceberg ,I test it .before insert, snapshots is 19.
and I insert one row ,it is 19.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] zixiao144 commented on issue #5803: flink1.15.2 or 1.15.1 stream read ,it is pending
Posted by GitBox <gi...@apache.org>.
zixiao144 commented on issue #5803:
URL: https://github.com/apache/iceberg/issues/5803#issuecomment-1253179440
thanks @stevenzwu I use hadoop catelog .
step1 stream read
step2 appended data
not work,no result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org