You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "wbo4958 (via GitHub)" <gi...@apache.org> on 2024/02/18 23:33:29 UTC
[PR] Using BigDecimal to do the resource calculation [spark]
wbo4958 opened a new pull request, #45157:
URL: https://github.com/apache/spark/pull/45157
### What changes were proposed in this pull request?
This PR utilizes BigDecimal to calculate the GPU resources.
### Why are the changes needed?
To prevent precision errors, the current method of calculating GPU resources involves multiplying by 1E16 to convert doubles to Longs. If needed, it will also convert Longs back to doubles. This approach introduces redundancy in the code, especially for test code.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass the CIs
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1963177180
I think I agree with it. I don't think there's a 'real' fix without changing the API and not sure it's worth it for the corner case here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "tgravescs (via GitHub)" <gi...@apache.org>.
tgravescs commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1954881564
Sorry, I'm not following what this pr changes from the previous implementation. the method toInternalResource handles 1.0/9.0 as well, correct? That is perhaps what I'm missing, what case does that not work for.. maybe 1/9.0 works out but you are saying others won't?
say 1/9.0 comes out a big bigger like 0.111111111111111112:
```
scala> toInternalResource(0.111111111111111112)
res17: Long = 1111111111111111
```
In this pr, the key line seems to be (the round down):
> var taskAmount = BigDecimal(taskReqs.amount).setScale(ResourceAmountUtils.SCALE, RoundingMode.DOWN)
but as we have discussed the api takes the double so if someone is programmatically passing something in its going to be computed on their side not by Spark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "tgravescs (via GitHub)" <gi...@apache.org>.
tgravescs commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1954460818
sorry, I've been ooo and busy with other things. I didn't fully follow the discussion in https://github.com/apache/spark/pull/44690.
Can you two please summarize exactly why are we changing this? Is there actually some case the previous method did not handle? Yes I understand there could be some precision loss when dealing with floats/double but I think that is going to come from the API.
Is this purely for readability at this point?
I think this goes from storing longs to storing BigDecimals, which is not going to be more memory efficient and I thought BigDecimal was fairly slow at operations, but maybe Java has made that better. There obviously was some conversion going on before with toInternalResource so some overhead there if this makes things more efficient. Doing some very poor many testing in a scala shell creating bigdecimal takes around 30,000ns and doing the conversions is around 6,000ns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "wbo4958 (via GitHub)" <gi...@apache.org>.
wbo4958 commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1963192025
Thx @srowen, Let me close this PR. And how about https://github.com/apache/spark/pull/44690 for spark 3.5 branch? we think it's a real bug, so we hope it can be merged into spark 3.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "wbo4958 (via GitHub)" <gi...@apache.org>.
wbo4958 commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1952138200
Hi @srowen, Could you help review this PR? Thx very much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "wbo4958 (via GitHub)" <gi...@apache.org>.
wbo4958 commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1953312076
Hi @srowen, The public API [def resource(resourceName: String, amount: Double): this.type = {](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/resource/TaskResourceRequests.scala#L71) in TaskResourceRequests require double, and the [TaskResourceRequest](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/resource/TaskResourceRequest.scala#L37) also require double. So if we would like to go from the string, that probably breaks the APIs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "tgravescs (via GitHub)" <gi...@apache.org>.
tgravescs commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1957867014
ok so we should just close this then, if it becomes an issue we could revisit the API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "wbo4958 (via GitHub)" <gi...@apache.org>.
wbo4958 closed pull request #45157: [SPARK-47088] Using BigDecimal to do the resource calculation
URL: https://github.com/apache/spark/pull/45157
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "tgravescs (via GitHub)" <gi...@apache.org>.
tgravescs commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1954835426
> so you request resource amount 1.0/9.0
How does a user specify "1.0/9.0" in the configs?
I tend to agree with your second statement that I don't see this as a problem in practice so my question is why are we changing it to be slower and user more memory if its not solving a problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1954623553
The issue is roughly: let's say you want to schedule a task that needs 1/9th of a GPU, and so you request resource amount `1.0/9.0`. Let's say that float is just a tiny bit bigger than 1/9th. After 8 of those resource requests have gone through, the amount of resource left is less than 1/9th, so the 9th can't schedule.
I think trying to do integer math after floating-point still has the issue; the value is already 'wrong' when it comes in as a double in the API.
It may in practice not come up a lot, as often these values are set as a Spark config, as a string, and a user is going to write "0.111" or something; some prefix of what is notionally a big repeating decimal, and so they will ask for less than 1/9th (say) of a GPU and schedule 9 as desired, leaving Spark thinking there's a tiny bit of GPU available when there isn't, but certainly nothing like 1/9th.
It's still a bit ugly and yeah the API kind of needs to take a different input type to _really_ address this but maybe not worth breakage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1954900085
I think the discussion on this got lost after a force-push, and referred to an earlier implementation, but my assertion is: if you start from double, then no matter how you represent that with integers or BigDecimal downstream, there will be _some_ problem of this form, yes. In some cases your double representation is too small and it works, in some cases it isn't. You should be able to find counterexamples, indeed. The representation would have to be exact from start to finish to really solve this and that seems to need API change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1954842973
It would have to be programmatically. Imagine, for example, that I generate a config with value 1.0/9.0 that works out to the string "0.1111111111111111" and when parsed as double, that's slightly bigger than 1/9. I think there is a real case here, just possibly rare; that's not unimaginable though.
(Performance is not the issue here, these calculations are trivial even in BigDecimal)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "wbo4958 (via GitHub)" <gi...@apache.org>.
wbo4958 commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1963168936
Hi @srowen, May I ask what's your opinion on @tgravescs [comment](https://github.com/apache/spark/pull/45157#issuecomment-1957867014)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47088] Using BigDecimal to do the resource calculation [spark]
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen commented on PR #45157:
URL: https://github.com/apache/spark/pull/45157#issuecomment-1953315425
Darn. Yeah I don't think there is a way to do better without changing the API and that's too much change for what isn't typically an issue in practice. I can't think of a way that keeps the double API but doesn't have a round off error in some case. Let's leave this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org