You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Andre Schumacher (JIRA)" <ji...@apache.org> on 2014/04/01 16:44:26 UTC

[jira] [Created] (SPARK-1383) Spark-SQL: ParquetRelation improvements

Andre Schumacher created SPARK-1383:
---------------------------------------

             Summary: Spark-SQL: ParquetRelation improvements
                 Key: SPARK-1383
                 URL: https://issues.apache.org/jira/browse/SPARK-1383
             Project: Spark
          Issue Type: Improvement
    Affects Versions: 1.0.0
            Reporter: Andre Schumacher


Improve Spark-SQL's ParquetRelation as follows:
- Instead of files a ParquetRelation is should be backed by a directory, which simplifies importing data from other sources
- InsertIntoParquetTable operation should supports switching between overwriting or appending (at least in HiveQL)
- tests should use the new API
- Parquet logging should be forwarded to Log4J
- It should be possible to enable compression (default compression for Parquet files: GZIP, as in parquet-mr)
- OverwriteCatalog should support dropping of tables





--
This message was sent by Atlassian JIRA
(v6.2#6252)