[jira] [Created] (ORC-502) Hive ORC read INT, BIGINT as NULL for Data created by spark

["Oleksiy Sayankin (JIRA)" <ji...@apache.org>]

Oleksiy Sayankin created ORC-502:
------------------------------------

             Summary: Hive ORC read INT, BIGINT as NULL for Data created by spark
                 Key: ORC-502
                 URL: https://issues.apache.org/jira/browse/ORC-502
             Project: ORC
          Issue Type: Bug
            Reporter: Oleksiy Sayankin


*Preconditions*

Create file {{ratings.csv}} and put it to HDFS {{/user/test/rating/ratings.csv}}.

{code}
userId,movieId,rating,timestamp
1,2,4.5,1784325658
{code}

*STR:*


1. Using spark (tested on version 2.2.1 and 2.3.1) created {{dataframe(df)}} of using {{interSchema}} from a CSV file

{code}
val df =spark.read.format("csv").option("header","true").option("inferSchema","true").load("/user/test/rating/ratings.csv")
{code}

2. Now save the df into ORC format file.

{code}
df.write.format("orc").save("/user/test/spark_rating_orc_typesafe")
{code}

3. Using hive 2.3. Try creating hive external table respective.

{code}
create external table rating_orc_hive_type_1(userId int,movieId int,rating double, `timestamp` int) stored as ORC location "/user/test/spark_orc_rating_typesafe/";
{code}

4. Do query
{code}
select * from rating_orc_hive_type_1;
{code}

Only double value is printed. Null for integer and even for BIGINT.
{code}
OK
NULL    NULL    4.5     1784325658
{code}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)