You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@orc.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2021/08/07 16:06:00 UTC

[jira] [Resolved] (ORC-913) Support data/format/compress options in Spark benchmark

     [ https://issues.apache.org/jira/browse/ORC-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dongjoon Hyun resolved ORC-913.
-------------------------------
    Fix Version/s: 1.8.0
                   1.7.0
         Assignee: William Hyun
       Resolution: Fixed

This is resolved via https://github.com/apache/orc/pull/825

> Support data/format/compress options in Spark benchmark
> -------------------------------------------------------
>
>                 Key: ORC-913
>                 URL: https://issues.apache.org/jira/browse/ORC-913
>             Project: ORC
>          Issue Type: Improvement
>          Components: Java
>    Affects Versions: 1.7.0, 1.8.0
>            Reporter: Dongjoon Hyun
>            Assignee: William Hyun
>            Priority: Major
>              Labels: starter
>             Fix For: 1.7.0, 1.8.0
>
>
> Currently, Spark benchmark jar supports the JMH options.
> {code}
> $ java -jar bench/spark/target/orc-benchmarks-spark-1.8.0-SNAPSHOT.jar spark -h
> usage: java -jar <jar> <command> <options> <data>
>  -f,--fork <arg>                How many forks to use
>  -g,--gc                        Should GC be profiled
>  -h,--help                      Provide help
>  -i,--iterations <arg>          Number of iterations
>  -I,--warmup-iterations <arg>   Number of warmup iterations
>  -m,--min-memory <arg>          The minimum size of each JVM
>  -M,--max-memory <arg>          The maximum size of each JVM
>  -p,--stack                     Should enable stack profiler in JMH
>  -t,--time <arg>                How long each iteration is in seconds
> {code}
> `core` benchmark jar file supports `format`/`data`/`compress` options. It would be great if Spark benchmark jar also supports its parameter options too.
> {code}
> $ java -jar bench/core/target/orc-benchmarks-core-*-uber.jar generate -h
> usage: convert <root>
>  -c,--compress <arg>   List of compression
>  -d,--data <arg>       List of data sets
>  -f,--format <arg>     List of formats
>  -h,--help             Provide help
>  -s,--sales <arg>      Number of records for sales
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)