You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/08/10 18:01:10 UTC
[GitHub] [iceberg] hililiwei opened a new pull request, #5493: Flink: Refactoring StreamingReaderOperator to read data nonblocking
hililiwei opened a new pull request, #5493:
URL: https://github.com/apache/iceberg/pull/5493
Co-authored-by: xuwei <xu...@huawei.com>
We currently have tens of billions of data in some of our tables, the old one needed to get all the data at once during one checkpoint, which caused job to get stuck for long time. Finally, a series of error such as checkpoint timeout occur.
<img width="1536" alt="image" src="https://user-images.githubusercontent.com/59213263/183982266-215e9858-927f-4dbb-9541-85686eb3a6f8.png">
The new one solves this problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Refactoring StreamingReaderOperator to read data nonblocking [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5493:
URL: https://github.com/apache/iceberg/pull/5493#issuecomment-2294473785
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Refactoring StreamingReaderOperator to read data nonblocking [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #5493: Flink: Refactoring StreamingReaderOperator to read data nonblocking
URL: https://github.com/apache/iceberg/pull/5493
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] hililiwei commented on pull request #5493: Flink: Refactoring StreamingReaderOperator to read data nonblocking
Posted by GitBox <gi...@apache.org>.
hililiwei commented on PR #5493:
URL: https://github.com/apache/iceberg/pull/5493#issuecomment-1219450439
> * can you elaborate what is exactly blocked in readers?
I'll add some screenshots to illustrate.
> * can you try the new FLIP-27 source?
We don't have Flink 1.14 in our pro-environment, but we plan to upgrade it at the end of the month and I'll try the FLIP-27 as soon as I can.
Could you please review it? Thank you. @stevenzwu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on pull request #5493: Flink: Refactoring StreamingReaderOperator to read data nonblocking
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on PR #5493:
URL: https://github.com/apache/iceberg/pull/5493#issuecomment-1221416065
@hililiwei can you explain the problem and solution with more details? It is not easy to understand what exactly is the issue here based on the screenshots?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on pull request #5493: Flink: Refactoring StreamingReaderOperator to read data nonblocking
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on PR #5493:
URL: https://github.com/apache/iceberg/pull/5493#issuecomment-1212232061
@hililiwei I have a couple of high-level questions
* can you elaborate what is exactly blocked in readers?
* can you try the new FLIP-27 source?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #5493: Flink: Refactoring StreamingReaderOperator to read data nonblocking
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #5493:
URL: https://github.com/apache/iceberg/pull/5493#issuecomment-1211379285
@stevenzwu and @openinx, can you review this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Flink: Refactoring StreamingReaderOperator to read data nonblocking [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5493:
URL: https://github.com/apache/iceberg/pull/5493#issuecomment-2309064301
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org