You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by linxi zeng <li...@gmail.com> on 2016/06/23 03:10:59 UTC

why did spark2.0 Disallow ROW FORMAT and STORED AS (parquet | orc | avro etc.)

Hi All,
    I have tried the spark sql of Spark branch-2.0 and countered an
unexpected problem:

Operation not allowed: ROW FORMAT DELIMITED is only compatible with
'textfile', not 'orc'(line 1, pos 0)

the sql is like:

CREATE TABLE IF NOT EXISTS test.test_orc
(
 ...
)
PARTITIONED BY (xxx)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n'
stored as orc

I found this JIRA: https://issues.apache.org/jira/browse/SPARK-15279,
but still can't understand why?

and by the way, this sql is work fine on spark1.4.