[jira] [Commented] (SPARK-12417) Orc bloom filter options are not propagated during file write in spark

["Apache Spark (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-12417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108085#comment-15108085 ] 

Apache Spark commented on SPARK-12417:
--------------------------------------

User 'rajeshbalamohan' has created a pull request for this issue:
https://github.com/apache/spark/pull/10842

> Orc bloom filter options are not propagated during file write in spark
> ----------------------------------------------------------------------
>
>                 Key: SPARK-12417
>                 URL: https://issues.apache.org/jira/browse/SPARK-12417
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Rajesh Balamohan
>            Assignee: Apache Spark
>         Attachments: SPARK-12417.1.patch
>
>
> ORC bloom filter is supported by the version of hive used in Spark 1.5.2. However, when trying to create orc file with bloom filter option, it does not make use of it.
> E.g, following orc output does not create the bloom filter even though the options are specified.
> {noformat}
>     Map<String, String> orcOption = new HashMap<String, String>();
>     orcOption.put("orc.bloom.filter.columns", "*");
>     hiveContext.sql("select * from accounts where effective_date='2015-12-30'").write().
>         format("orc").options(orcOption).save("/tmp/accounts");
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org