You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/28 13:53:35 UTC
[GitHub] [spark] gengliangwang commented on a change in pull request #29837: [SPARK-32463][SQL][DOCS] Add "Type Conversion" section in "Supported Data Types" of SQL docs
gengliangwang commented on a change in pull request #29837:
URL: https://github.com/apache/spark/pull/29837#discussion_r513460544
##########
File path: docs/sql-ref-datatypes.md
##########
@@ -314,3 +314,200 @@ SELECT COUNT(*), c2 FROM test GROUP BY c2;
| 3| Infinity|
+---------+---------+
```
+
+### Type Conversion
+
+In general, an expression can contain different data types and type conversion is the transformation of some data types into others in order to resolve type mismatches.
+Spark supports both implicit conversions by type coercion and explicit conversions by explicit casting and store assignment casting.
+
+#### Type Coercion in Operations between Different Types
+
+Type Coercion refers to the automatic or implicit conversion of values from one type to another when you need to to resolve type mismatches.
+The following matrix shows the resulting type to which they are implicitly converted to resolve an expression involving different data types.
+
+**Numeric Expressions**:
+
+| |ByteType |ShortType |IntegerType |LongType |FloatType |DoubleType |DecimalType |
+|---------------|-----------|-----------|------------|-----------|----------------------|----------------------|-----------------------------|
+|**ByteType** |-- |ShortType |IntegerType |LongType |FloatType |DoubleType |DecimalType(3,0)<sup>1</sup> |
+|**ShortType** |ShortType |-- |IntegerType |LongType |FloatType |DoubleType |DecimalType(5,0)<sup>1</sup> |
+|**IntegerType**|IntegerType|IntegerType|-- |LongType |FloatType |DoubleType |DecimalType(10,0)<sup>1</sup>|
+|**LongType** |LongType |LongType |LongType |-- |FloatType |DoubleType |DecimalType(20,0)<sup>1</sup>|
+|**FloatType** |FloatType |FloatType |FloatType |FloatType |-- |DoubleType |DoubleType |
+|**DoubleType** |DoubleType |DoubleType |DoubleType |DoubleType |DoubleType |-- |DoubleType |
+|**DecimalType**|DecimalType|DecimalType|DecimalType |DecimalType|DoubleType<sup>2</sup>|DoubleType<sup>2</sup>|-- |
+
+**Note 1**: DecimalType(precision,scale)
+**Note 2**: In these cases DecimalType can lose precision, there is no common type for decimal and double because double's range is larger than decimal, and yet decimal is more precise than double so when we cast Decimaltype into DobleType it could lose precision.
+
+**StringType Behavior**
+* Arithmetic Expressions:
+
+ | |ByteType |ShortType |IntegerType |LongType |FloatType |DoubleType |
+ |---------------|-----------|-----------|------------|-----------|-------------|------------|
+ |**StringType** |DoubleType |DoubleType |DoubleType |DoubleType |DoubleType |DoubleType |
Review comment:
I think we need to make it clear that the String type will be implicit cast as Double type.
##########
File path: docs/sql-ref-datatypes.md
##########
@@ -314,3 +314,200 @@ SELECT COUNT(*), c2 FROM test GROUP BY c2;
| 3| Infinity|
+---------+---------+
```
+
+### Type Conversion
+
+In general, an expression can contain different data types and type conversion is the transformation of some data types into others in order to resolve type mismatches.
+Spark supports both implicit conversions by type coercion and explicit conversions by explicit casting and store assignment casting.
+
+#### Type Coercion in Operations between Different Types
+
+Type Coercion refers to the automatic or implicit conversion of values from one type to another when you need to to resolve type mismatches.
+The following matrix shows the resulting type to which they are implicitly converted to resolve an expression involving different data types.
+
+**Numeric Expressions**:
+
+| |ByteType |ShortType |IntegerType |LongType |FloatType |DoubleType |DecimalType |
+|---------------|-----------|-----------|------------|-----------|----------------------|----------------------|-----------------------------|
+|**ByteType** |-- |ShortType |IntegerType |LongType |FloatType |DoubleType |DecimalType(3,0)<sup>1</sup> |
+|**ShortType** |ShortType |-- |IntegerType |LongType |FloatType |DoubleType |DecimalType(5,0)<sup>1</sup> |
+|**IntegerType**|IntegerType|IntegerType|-- |LongType |FloatType |DoubleType |DecimalType(10,0)<sup>1</sup>|
+|**LongType** |LongType |LongType |LongType |-- |FloatType |DoubleType |DecimalType(20,0)<sup>1</sup>|
+|**FloatType** |FloatType |FloatType |FloatType |FloatType |-- |DoubleType |DoubleType |
+|**DoubleType** |DoubleType |DoubleType |DoubleType |DoubleType |DoubleType |-- |DoubleType |
+|**DecimalType**|DecimalType|DecimalType|DecimalType |DecimalType|DoubleType<sup>2</sup>|DoubleType<sup>2</sup>|-- |
+
+**Note 1**: DecimalType(precision,scale)
+**Note 2**: In these cases DecimalType can lose precision, there is no common type for decimal and double because double's range is larger than decimal, and yet decimal is more precise than double so when we cast Decimaltype into DobleType it could lose precision.
+
+**StringType Behavior**
+* Arithmetic Expressions:
+
+ | |ByteType |ShortType |IntegerType |LongType |FloatType |DoubleType |
+ |---------------|-----------|-----------|------------|-----------|-------------|------------|
+ |**StringType** |DoubleType |DoubleType |DoubleType |DoubleType |DoubleType |DoubleType |
+
+* Comparison:
+
+ | |ByteType |ShortType |IntegerType |LongType |FloatType |DoubleType |DecimalType |DateType |TimestampType |
+ |---------------|-----------|-----------|------------|-----------|-------------|------------|------------|---------------------|--------------------------|
+ |**StringType** |ByteType |ShortType |IntegerType |LongType |FloatType |DoubleType |DoubleType |DateType<sup>1</sup> |TimestampType<sup>1</sup> |
Review comment:
ditto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org