You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by wangyum <gi...@git.apache.org> on 2016/07/27 09:35:14 UTC

[GitHub] spark pull request #14377: [SPARK-16625][SQL] General data types to be mappe...

GitHub user wangyum opened a pull request:

    https://github.com/apache/spark/pull/14377

    [SPARK-16625][SQL] General data types to be mapped to Oracle

    ## What changes were proposed in this pull request?
    
    
    Spark will convert **BooleanType** to **BIT(1)**, **LongType** to **BIGINT**, **ByteType**  to **BYTE** when saving DataFrame to Oracle, but Oracle does not support BIT, BIGINT and BYTE types.
    
    This PR is convert following _Spark Types_ to _Oracle types_ refer to [Oracle Developer's Guide](https://docs.oracle.com/cd/E19501-01/819-3659/gcmaz/)
    
    Spark Type | Oracle  
    ----|----
    BooleanType | NUMBER(1) 
    IntegerType | NUMBER(10)  
    LongType | NUMBER(19)  
    FloatType | NUMBER(19, 4)  
    DoubleType | NUMBER(19, 4)  
    ByteType | NUMBER(3)  
    ShortType | NUMBER(5)  
    
    ## How was this patch tested?
    
    Add new tests in [JDBCSuite.scala](https://github.com/wangyum/spark/commit/22b0c2a4228cb8b5098ad741ddf4d1904e745ff6#diff-dc4b58851b084b274df6fe6b189db84d) and [OracleDialect.scala](https://github.com/wangyum/spark/commit/22b0c2a4228cb8b5098ad741ddf4d1904e745ff6#diff-5e0cadf526662f9281aa26315b3750ad)
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/wangyum/spark SPARK-16625

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/14377.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #14377
    
----
commit 22b0c2a4228cb8b5098ad741ddf4d1904e745ff6
Author: Yuming Wang <wg...@gmail.com>
Date:   2016-07-27T06:33:10Z

    General data types to be mapped to Oracle

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    **[Test build #63263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63263/consoleFull)** for PR 14377 at commit [`bfadf77`](https://github.com/apache/spark/commit/bfadf774b52d696be73ec69b771e651ed365a3aa).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by vivekdixit05 <gi...@git.apache.org>.
Github user vivekdixit05 commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    This change has broken number with size to be converted to boolean. I have one table which has number column with size 1, in spark 2.1 its get converted to boolean, instead of integer. If this is expected, from the code is there a way to avoid this conversion?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #14377: [SPARK-16625][SQL] General data types to be mappe...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/14377


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    @vivekdixit05 Which direction? Reading or writing?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    **[Test build #62919 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62919/consoleFull)** for PR 14377 at commit [`22b0c2a`](https://github.com/apache/spark/commit/22b0c2a4228cb8b5098ad741ddf4d1904e745ff6).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    **[Test build #62919 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62919/consoleFull)** for PR 14377 at commit [`22b0c2a`](https://github.com/apache/spark/commit/22b0c2a4228cb8b5098ad741ddf4d1904e745ff6).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    @poolis @thomastechs do you have an opinion on this? I see you suggested changes to the Oracle dialect previously so might be good reviewers


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Jenkins add to whitelist


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by pokkie <gi...@git.apache.org>.
Github user pokkie commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    `NUMBER(10)` shouldn't automatically map to Integer. In our case, we have values stored like `4001826725` in that column and thus we get `java.sql.SQLException:` Numeric Overflow`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by vivekdixit05 <gi...@git.apache.org>.
Github user vivekdixit05 commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    While reading. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    **[Test build #63170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63170/consoleFull)** for PR 14377 at commit [`88e5d2e`](https://github.com/apache/spark/commit/88e5d2eaf497d131b355ed545793ac9de43e2110).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Jenkins test this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    **[Test build #63170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63170/consoleFull)** for PR 14377 at commit [`88e5d2e`](https://github.com/apache/spark/commit/88e5d2eaf497d131b355ed545793ac9de43e2110).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63263/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    **[Test build #63263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63263/consoleFull)** for PR 14377 at commit [`bfadf77`](https://github.com/apache/spark/commit/bfadf774b52d696be73ec69b771e651ed365a3aa).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Merged to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63170/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    @vivekdixit05 Does this PR https://github.com/apache/spark/pull/17830 resolve your issue?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #14377: [SPARK-16625][SQL] General data types to be mappe...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:

    https://github.com/apache/spark/pull/14377#discussion_r121545683
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala ---
    @@ -28,28 +28,42 @@ private case object OracleDialect extends JdbcDialect {
     
       override def getCatalystType(
           sqlType: Int, typeName: String, size: Int, md: MetadataBuilder): Option[DataType] = {
    -    // Handle NUMBER fields that have no precision/scale in special way
    -    // because JDBC ResultSetMetaData converts this to 0 precision and -127 scale
    -    // For more details, please see
    -    // https://github.com/apache/spark/pull/8780#issuecomment-145598968
    -    // and
    -    // https://github.com/apache/spark/pull/8780#issuecomment-144541760
    -    if (sqlType == Types.NUMERIC && size == 0) {
    -      // This is sub-optimal as we have to pick a precision/scale in advance whereas the data
    -      //  in Oracle is allowed to have different precision/scale for each value.
    -      Option(DecimalType(DecimalType.MAX_PRECISION, 10))
    -    } else if (sqlType == Types.NUMERIC && md.build().getLong("scale") == -127) {
    -      // Handle FLOAT fields in a special way because JDBC ResultSetMetaData converts
    -      // this to NUMERIC with -127 scale
    -      // Not sure if there is a more robust way to identify the field as a float (or other
    -      // numeric types that do not specify a scale.
    -      Option(DecimalType(DecimalType.MAX_PRECISION, 10))
    +    if (sqlType == Types.NUMERIC) {
    +      val scale = if (null != md) md.build().getLong("scale") else 0L
    +      size match {
    +        // Handle NUMBER fields that have no precision/scale in special way
    +        // because JDBC ResultSetMetaData converts this to 0 precision and -127 scale
    +        // For more details, please see
    +        // https://github.com/apache/spark/pull/8780#issuecomment-145598968
    +        // and
    +        // https://github.com/apache/spark/pull/8780#issuecomment-144541760
    +        case 0 => Option(DecimalType(DecimalType.MAX_PRECISION, 10))
    +        // Handle FLOAT fields in a special way because JDBC ResultSetMetaData converts
    +        // this to NUMERIC with -127 scale
    +        // Not sure if there is a more robust way to identify the field as a float (or other
    +        // numeric types that do not specify a scale.
    +        case _ if scale == -127L => Option(DecimalType(DecimalType.MAX_PRECISION, 10))
    +        case 1 => Option(BooleanType)
    +        case 3 | 5 | 10 => Option(IntegerType)
    +        case 19 if scale == 0L => Option(LongType)
    +        case 19 if scale == 4L => Option(FloatType)
    --- End diff --
    
    We should not change the read path. The bug we need to resolve is just the write path. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Jenkins retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Jenkins test this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #14377: [SPARK-16625][SQL] General data types to be mappe...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/14377#discussion_r73189845
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala ---
    @@ -26,30 +26,38 @@ private case object OracleDialect extends JdbcDialect {
     
       override def canHandle(url: String): Boolean = url.startsWith("jdbc:oracle")
     
    -  override def getCatalystType(
    -      sqlType: Int, typeName: String, size: Int, md: MetadataBuilder): Option[DataType] = {
    +  override def getCatalystType(sqlType: Int, typeName: String, size: Int, md: MetadataBuilder):
    +  Option[DataType] = sqlType match {
         // Handle NUMBER fields that have no precision/scale in special way
         // because JDBC ResultSetMetaData converts this to 0 precision and -127 scale
         // For more details, please see
         // https://github.com/apache/spark/pull/8780#issuecomment-145598968
         // and
         // https://github.com/apache/spark/pull/8780#issuecomment-144541760
    -    if (sqlType == Types.NUMERIC && size == 0) {
    -      // This is sub-optimal as we have to pick a precision/scale in advance whereas the data
    -      //  in Oracle is allowed to have different precision/scale for each value.
    +    case Types.NUMERIC if size == 0 => Option(DecimalType(DecimalType.MAX_PRECISION, 10))
    +    // Handle FLOAT fields in a special way because JDBC ResultSetMetaData converts
    +    // this to NUMERIC with -127 scale
    +    // Not sure if there is a more robust way to identify the field as a float (or other
    +    // numeric types that do not specify a scale.
    +    case Types.NUMERIC if md.build().getLong("scale") == -127 =>
    --- End diff --
    
    @wangyum I can merge this, since it looks reasonable, even though I'm not an Oracle expert. Can we streamline this a bit by only computing `md.build().getLong("scale")` once? The repeated matching of `Types.NUMERIC` could be done once in an `if` statement and be clearer.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62919/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by wangyum <gi...@git.apache.org>.
Github user wangyum commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    @gatorsmile We can consider merging this PR: https://github.com/apache/spark/pull/18266.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14377: [SPARK-16625][SQL] General data types to be mapped to Or...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/14377
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org