[orc] branch main updated: ORC-1108: Use `RawLocalFileSystem` to skip checksum files during benchmark data generation (#1034)

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/orc.git


The following commit(s) were added to refs/heads/main by this push:
     new 9dfafce  ORC-1108: Use `RawLocalFileSystem` to skip checksum files during benchmark data generation (#1034)
9dfafce is described below

commit 9dfafce8a416ffa21f170aa9f3da9c3d0d84a832
Author: William Hyun <wi...@apache.org>
AuthorDate: Sun Jan 30 19:51:35 2022 -0800

    ORC-1108: Use `RawLocalFileSystem` to skip checksum files during benchmark data generation (#1034)
    
    ### What changes were proposed in this pull request?
    
    This PR aims to use `RawLocalFileSystem` to skip checksum files during benchmark data generation.
    
    ### Why are the changes needed?
    
    There is no need to check for checksums.
    
    ### How was this patch tested?
    
    Manually.
---
 .../src/java/org/apache/orc/bench/core/convert/GenerateVariants.java    | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/java/bench/core/src/java/org/apache/orc/bench/core/convert/GenerateVariants.java b/java/bench/core/src/java/org/apache/orc/bench/core/convert/GenerateVariants.java
index efa332d..0fc3683 100644
--- a/java/bench/core/src/java/org/apache/orc/bench/core/convert/GenerateVariants.java
+++ b/java/bench/core/src/java/org/apache/orc/bench/core/convert/GenerateVariants.java
@@ -120,6 +120,8 @@ public class GenerateVariants implements OrcBenchmark {
         cli.getOptionValue("format", "avro,json,orc,parquet").split(",");
     long records = Long.parseLong(cli.getOptionValue("sales", "25000000"));
     Configuration conf = new Configuration();
+    // Disable Hadoop checksums
+    conf.set("fs.file.impl", "org.apache.hadoop.fs.RawLocalFileSystem");
     Path root = new Path(cli.getArgs()[0]);
 
     for (final String data: dataList) {