[jira] [Created] (LIVY-509) livy SQL drops leading zeros

["Glenn McCall (JIRA)" <ji...@apache.org>]

Glenn McCall created LIVY-509:
---------------------------------

             Summary: livy SQL drops leading zeros
                 Key: LIVY-509
                 URL: https://issues.apache.org/jira/browse/LIVY-509
             Project: Livy
          Issue Type: Bug
          Components: Interpreter
    Affects Versions: 0.5.0, 0.4.0
            Reporter: Glenn McCall
         Attachments: Screen Shot 2018-09-13 at 4.21.37 pm.png, Screen Shot 2018-09-13 at 4.21.37 pm.png

The %sql drops leading zeros from columns with *String* datatype.

Consider the following test data:

 
{code:java}
0123,zero one two three
1230,one two three zero
1010,one zero one zero

{code}
Created as an external table in Hive using:

 

 
{code:java}
create external table lz_test (
  id String,
  description String
) row format delimited
fields terminated by ','
location '/pathTo/leadingZero_test'
;
{code}
and accessed using the following scala (%livy) code:

 

 
{code:java}
val lzDF = sql("select * from lz_Test")
lzDF.createOrReplaceTempView("LZT")
lzDF.printSchema
lzDF.show(false)
{code}
and the following sql in the same notebook:

 

 
{code:java}
%sql
select *
from LZT
{code}
The result is the following (note the missing zero on the first record):

 

!Screen Shot 2018-09-13 at 4.21.37 pm.png!

The output of the scala code does, however, display the leading zero.Also note the data types from the print schema: ID is a String.

 
{code:java}
lzDF: org.apache.spark.sql.DataFrame = [id: string, description: string]
root
|-- id: string (nullable = true)
|-- description: string (nullable = true)

+----+------------------+
|id |description |
+----+------------------+
|0123|zero one two three|
|1230|one two three zero|
|1010|one zero one zero |
+----+------------------+

{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)