You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uniffle.apache.org by "jerqi (via GitHub)" <gi...@apache.org> on 2023/04/08 03:21:56 UTC
[GitHub] [incubator-uniffle] jerqi opened a new issue, #802: [Improvement] Implement the interface ShuffleDataIO
jerqi opened a new issue, #802:
URL: https://github.com/apache/incubator-uniffle/issues/802
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
We can implement ShuffleDataIO
https://github.com/apache/spark/blob/master/core/src/main/java/org/apache/spark/shuffle/api/ShuffleDataIO.java
https://github.com/apache/spark/blob/master/core/src/main/java/org/apache/spark/shuffle/api/ShuffleDriverComponents.java
Dynamic Allocation don't need patch any more.
### How should we improve?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi commented on issue #802: [Improvement] Implement the interface ShuffleDataIO
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on issue #802:
URL: https://github.com/apache/incubator-uniffle/issues/802#issuecomment-1501704527
Yes, you can build a spark client with master branch to test it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi commented on issue #802: [Improvement] Implement the interface ShuffleDataIO
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on issue #802:
URL: https://github.com/apache/incubator-uniffle/issues/802#issuecomment-1659517633
@Kwafoor If you're busy recently, let @summaryzb finish this issue first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [I] [Improvement] Implement the interface ShuffleDataIO [incubator-uniffle]
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi closed issue #802: [Improvement] Implement the interface ShuffleDataIO
URL: https://github.com/apache/incubator-uniffle/issues/802
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] Kwafoor commented on issue #802: [Improvement] Implement the interface ShuffleDataIO
Posted by "Kwafoor (via GitHub)" <gi...@apache.org>.
Kwafoor commented on issue #802:
URL: https://github.com/apache/incubator-uniffle/issues/802#issuecomment-1501528897
I'm Interested in this, could you assign it to me？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] summaryzb commented on issue #802: [Improvement] Implement the interface ShuffleDataIO
Posted by "summaryzb (via GitHub)" <gi...@apache.org>.
summaryzb commented on issue #802:
URL: https://github.com/apache/incubator-uniffle/issues/802#issuecomment-1659516007
volunteering
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] Kwafoor commented on issue #802: [Improvement] Implement the interface ShuffleDataIO
Posted by "Kwafoor (via GitHub)" <gi...@apache.org>.
Kwafoor commented on issue #802:
URL: https://github.com/apache/incubator-uniffle/issues/802#issuecomment-1501692541
The patch [SPARK-42689] that allows ShuffleDriverComponent to declare if shuffle data is reliably stored has been fixed in Spark 3.5, which has not yet been released. Do I need to test it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] Kwafoor commented on issue #802: [Improvement] Implement the interface ShuffleDataIO
Posted by "Kwafoor (via GitHub)" <gi...@apache.org>.
Kwafoor commented on issue #802:
URL: https://github.com/apache/incubator-uniffle/issues/802#issuecomment-1501715607
> Yes, you can build a spark client with master branch to test it.
Okay, I will do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi commented on issue #802: [Improvement] Implement the interface ShuffleDataIO
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on issue #802:
URL: https://github.com/apache/incubator-uniffle/issues/802#issuecomment-1501529962
> I'm Interested in this, could you assign it to me？
OK, I have assigned to you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org