[jira] [Created] (DRILL-5128) Issue with unable to create parquet file on dfs workspace

["David Smith (JIRA)" <ji...@apache.org>]

David Smith created DRILL-5128:
----------------------------------

             Summary: Issue with unable to create parquet file on dfs workspace
                 Key: DRILL-5128
                 URL: https://issues.apache.org/jira/browse/DRILL-5128
             Project: Apache Drill
          Issue Type: Bug
          Components: Storage - Parquet
    Affects Versions: 1.9.0
         Environment: docker instance built on harisekhon/apache-drill
            Reporter: David Smith


Unable to write a parquet file to dfs workspace with CTAS from an Oracle query.

Oracle querying works fine using ojdbc6.jar in /jars/3rdparty and configured via enabled storage plugin

Created dfs "data" folder and storage workspace:
{quote}
drwxr-xr-x    3 root     root          4096 Dec 13 16:28 data
{quote}
{quote}
"data": {
      "location": "/data",
      "writable": true,
      "defaultInputFormat": null
    },
{
{quote}
and set up to use it:
{quote}
use dfs.data;
alter session set `store.format` = 'parquet'
{quote}
and then used CTAS to query Oracle for a test sample to write out:
{quote}
create table dfs.data.tsdtest1 as select * from enviroi.D_EF_FAC.FRS_INTEREST where INTEREST_TYPE = 'TSD' LIMIT 10;
{quote}
Which then throws the error
{quote}
Exception in thread "drill-executor-28" java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy
        at org.apache.parquet.hadoop.DirectCodecFactory$SnappyCompressor.compress(DirectCodecFactory.java:290)
        at org.apache.parquet.hadoop.ColumnChunkPageWriteStore$ColumnChunkPageWriter.writePage(ColumnChunkPageWriteStore.java:94)
        at org.apache.parquet.column.impl.ColumnWriterV1.writePage(ColumnWriterV1.java:154)
        at org.apache.parquet.column.impl.ColumnWriterV1.flush(ColumnWriterV1.ja
va:242)
        at org.apache.parquet.column.impl.ColumnWriteStoreV1.flush(ColumnWriteStoreV1.java:129)
        at org.apache.drill.exec.store.parquet.ParquetRecordWriter.flush(Parquet
RecordWriter.java:253)
        at org.apache.drill.exec.store.parquet.ParquetRecordWriter.cleanup(Parqu
etRecordWriter.java:381)
        at org.apache.drill.exec.physical.impl.WriterRecordBatch.closeWriter(Wri
terRecordBatch.java:180)
        at org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(Write
rRecordBatch.java:128)
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordB
atch.java:162)
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordB
atch.java:119)
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordB
atch.java:109)
        at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(Abst
ractSingleRecordBatch.java:51)
        at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerN
ext(ProjectRecordBatch.java:135)
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordB
atch.java:162)
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.ja
va:104)
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNex
t(ScreenCreator.java:81)
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.ja
va:94)
        at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentEx
ecutor.java:232)
        at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:226)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:226)
        at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:617)
        at java.lang.Thread.run(Thread.java:745)
Error: SYSTEM ERROR: IOException: The file being written is in an invalid state. Probably caused by an error thrown previously. Current state: BLOCK

Fragment 0:0

[Error Id: 23f78b7e-f15c-461a-8179-2738ebfcf4a7 on 8637c49541d:31010] (state=,code=0)
{quote}
The process did create \data\tsdtest1 with a file, 0_0_0.parquet:
{quote}
bash-4.3# ls -l tsdtest1
total 0
-rw-r--r--    1 root     root             0 Dec 13 16:28 0_0_0.parquet
{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)