You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "liming30 (via GitHub)" <gi...@apache.org> on 2023/03/17 09:33:12 UTC
[GitHub] [incubator-paimon] liming30 opened a new pull request, #624: [FLINK-31462] Supports full calculation from the specified snapshot in streaming mode
liming30 opened a new pull request, #624:
URL: https://github.com/apache/incubator-paimon/pull/624
### Purpose
When consuming multiple tables, it may be necessary to align data versions in different tables to ensure version consistency. However, the current stream mode only supports incremental processing from the specified snapshot, and does not support full calculation of the specified snapshot first, and then incremental calculation.
### Brief change
A new startup mode **FROM_SNAPSHOT_FULL** is added to support full calculation of the specified snapshot of data in streaming mode, and then incrementally process the newly added data to ensure the consistency of the data version when processing multiple tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on pull request #624: [FLINK-31462] Supports full calculation from the specified snapshot in streaming mode
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #624:
URL: https://github.com/apache/incubator-paimon/pull/624#issuecomment-1486248382
Hi @liming30 , test failed, you can find action in your repo: https://github.com/liming30/flink-table-store/actions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi merged pull request #624: [FLINK-31462] Supports full calculation from the specified snapshot in streaming mode
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #624:
URL: https://github.com/apache/incubator-paimon/pull/624
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] liming30 commented on pull request #624: [FLINK-31462] Supports full calculation from the specified snapshot in streaming mode
Posted by "liming30 (via GitHub)" <gi...@apache.org>.
liming30 commented on PR #624:
URL: https://github.com/apache/incubator-paimon/pull/624#issuecomment-1486516696
@JingsongLi hi, I have fixed the test, can you please help to take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] liming30 commented on pull request #624: [FLINK-31462] Supports full calculation from the specified snapshot in streaming mode
Posted by "liming30 (via GitHub)" <gi...@apache.org>.
liming30 commented on PR #624:
URL: https://github.com/apache/incubator-paimon/pull/624#issuecomment-1482427191
@JingsongLi Sorry for replying now, I added a `StartupModeTest` to verify that the plan generated by each `StartupMode` is correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org