You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "hailin0 (via GitHub)" <gi...@apache.org> on 2023/04/18 03:49:45 UTC
[GitHub] [incubator-seatunnel] hailin0 opened a new issue, #4604: [Feature][CDC] Support spark running cdc job
hailin0 opened a new issue, #4604:
URL: https://github.com/apache/incubator-seatunnel/issues/4604
### Search before asking
- [X] I had searched in the [feature](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
### Description
Support spark submit cdc job
### Usage Scenario
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] laglangyue commented on issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "laglangyue (via GitHub)" <gi...@apache.org>.
laglangyue commented on issue #4604:
URL: https://github.com/apache/incubator-seatunnel/issues/4604#issuecomment-1512455990
I am willing to try it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] hailin0 commented on issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 commented on issue #4604:
URL: https://github.com/apache/seatunnel/issues/4604#issuecomment-1723353057
hello, is there any progress?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 commented on issue #4604:
URL: https://github.com/apache/incubator-seatunnel/issues/4604#issuecomment-1512395651
@laglangyue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] laglangyue commented on issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "laglangyue (via GitHub)" <gi...@apache.org>.
laglangyue commented on issue #4604:
URL: https://github.com/apache/seatunnel/issues/4604#issuecomment-1724746078
> hello, is there any progress?
sorry，I encountered a problem while processing the state recovery of spark struct streaming. Personally, I believe that the current spark state processing logic cannot recover the streaming task, including CDC.
I hope this PR can give you some help #4713
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] github-actions[bot] closed issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #4604: [Feature][CDC] Support spark running cdc job
URL: https://github.com/apache/seatunnel/issues/4604
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] Carl-Zhou-CN commented on issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "Carl-Zhou-CN (via GitHub)" <gi...@apache.org>.
Carl-Zhou-CN commented on issue #4604:
URL: https://github.com/apache/seatunnel/issues/4604#issuecomment-1725035703
> > hello, is there any progress?
>
> sorry，I encountered a problem while processing the state recovery of spark struct streaming. Personally, I believe that the current spark state processing logic cannot recover the streaming task, including CDC. I hope this PR can give you some help #4713 @Carl-Zhou-CN
Okay, thank you very much
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] laglangyue commented on issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "laglangyue (via GitHub)" <gi...@apache.org>.
laglangyue commented on issue #4604:
URL: https://github.com/apache/seatunnel/issues/4604#issuecomment-1724740395
> @hailin0 I am also very willing to participate i
welcome for you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] github-actions[bot] commented on issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4604:
URL: https://github.com/apache/seatunnel/issues/4604#issuecomment-1583664165
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] Carl-Zhou-CN commented on issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "Carl-Zhou-CN (via GitHub)" <gi...@apache.org>.
Carl-Zhou-CN commented on issue #4604:
URL: https://github.com/apache/seatunnel/issues/4604#issuecomment-1723395111
@hailin0 I am also very willing to participate in it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4604:
URL: https://github.com/apache/incubator-seatunnel/issues/4604#issuecomment-1553844639
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] laglangyue commented on issue #4604: [Feature][CDC] Support spark running cdc job
Posted by "laglangyue (via GitHub)" <gi...@apache.org>.
laglangyue commented on issue #4604:
URL: https://github.com/apache/seatunnel/issues/4604#issuecomment-1724748650
I once tried to read the source code of Spark/Flink and migrate the streaming state processing logic. But the progress is slow due to some life changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org