You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by viirya <gi...@git.apache.org> on 2015/10/02 17:01:37 UTC
[GitHub] spark pull request: [SPARK-10909][SQL] Fix precision overflow prob...
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/8963
[SPARK-10909][SQL] Fix precision overflow problem for Oracle NUMBER column
JIRA: https://issues.apache.org/jira/browse/SPARK-10909
When connecting with Oracle DB through JDBC, the precision and scale of BigDecimal object returned by ResultSet.getBigDecimal may not be correctly matched to the precision and the scale of the reported schema. It may cause a problem like: a BigDecimal object representing 19999 with precision 5 and scale 0 for a column with schema of precision 5 and scale 2. If we use the reported precision and scale and create Decimal object later, Decimal class will raise "Overflowed precision" error because actually we create a Decimal(7, 2) for 19999.99. To solve that, this patch adds scale to the precision here.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 fix-oracle-jdbc-number-overflow
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8963.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8963
----
commit a79836019316bab3090973aae9c3fdb76c6ae76f
Author: Liang-Chi Hsieh <vi...@appier.com>
Date: 2015-10-02T14:58:54Z
Fix precision overflow problem for Oracle NUMBER column.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10909][SQL] Fix precision overflow prob...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8963#issuecomment-145086285
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10909][SQL] Fix precision overflow prob...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8963#issuecomment-145052343
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10909][SQL] Fix precision overflow prob...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/8963#issuecomment-156102537
This issue is solved by other commit. Close it now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10909][SQL] Fix precision overflow prob...
Posted by viirya <gi...@git.apache.org>.
Github user viirya closed the pull request at:
https://github.com/apache/spark/pull/8963
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10909][SQL] Fix precision overflow prob...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8963#issuecomment-145052408
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10909][SQL] Fix precision overflow prob...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8963#issuecomment-145086112
[Test build #43179 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43179/console) for PR 8963 at commit [`a798360`](https://github.com/apache/spark/commit/a79836019316bab3090973aae9c3fdb76c6ae76f).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10909][SQL] Fix precision overflow prob...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8963#issuecomment-145054869
[Test build #43179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43179/consoleFull) for PR 8963 at commit [`a798360`](https://github.com/apache/spark/commit/a79836019316bab3090973aae9c3fdb76c6ae76f).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10909][SQL] Fix precision overflow prob...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8963#issuecomment-145086286
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43179/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10909][SQL] Fix precision overflow prob...
Posted by rick-ibm <gi...@git.apache.org>.
Github user rick-ibm commented on the pull request:
https://github.com/apache/spark/pull/8963#issuecomment-154125587
A decimal value with precision 7 and scale 0 must incur an out-of-range error when put into a decimal column which has precision 7 and scale 2. The number has 7 digits to the left of the decimal point but the column datatype only accepts 5 digits to the left of the decimal point. That is my reading of the behavior defined by the 2011 SQL Standard, part 2, section 9.2 (store assignment), general rule 3.b.xv. If Oracle allows some non-Standard treatment when storing decimal values and Spark must support that behavior, then I think that an Oracle-specific JdbcDialect should be created. The solution in this pull request does not look correct to me for other SQL dialects. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org