You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "hasnain-db (via GitHub)" <gi...@apache.org> on 2023/09/28 05:20:33 UTC
[GitHub] [spark] hasnain-db opened a new pull request, #43162: [SPARK-44937][CORE] Mark connection as timedOut in TransportClient.close
hasnain-db opened a new pull request, #43162:
URL: https://github.com/apache/spark/pull/43162
### What changes were proposed in this pull request?
This PR avoids a race condition where a connection which is in the process of being closed could be returned by the TransportClientFactory only to be immediately closed and cause errors upon use.
This race condition is rare and not easily triggered, but with the upcoming changes to introduce SSL connection support, connection closing can take just a slight bit longer and it's much easier to trigger this issue.
Looking at the history of the code I believe this was an oversight in https://github.com/apache/spark/pull/9853.
### Why are the changes needed?
Without this change, some of the new tests added in https://github.com/apache/spark/pull/42685 would fail
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing tests were run in CI.
Without this change, some of the new tests added in
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] hasnain-db commented on pull request #43162: [SPARK-44937][CORE] Mark connection as timedOut in TransportClient.close
Posted by "hasnain-db (via GitHub)" <gi...@apache.org>.
hasnain-db commented on PR #43162:
URL: https://github.com/apache/spark/pull/43162#issuecomment-1739770687
@srowen I don't know about spark OSS guidelines for when something should be backported or not. It's been in the code this way for many years and no one's complained so I think we can be fine with a backport. It's only with SSL support that this starts to become a problem, and since that functionality isn't being backported I assume we are fine?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #43162: [SPARK-44937][CORE] Mark connection as timedOut in TransportClient.close
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on PR #43162:
URL: https://github.com/apache/spark/pull/43162#issuecomment-1740115347
Merged to master, 3.5, 3.4, 3.3
Thanks for fixing this @hasnain-db !
Thanks for the reviews @JoshRosen, @srowen :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm closed pull request #43162: [SPARK-44937][CORE] Mark connection as timedOut in TransportClient.close
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm closed pull request #43162: [SPARK-44937][CORE] Mark connection as timedOut in TransportClient.close
URL: https://github.com/apache/spark/pull/43162
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org