You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "Hyunsik Choi (JIRA)" <ji...@apache.org> on 2015/10/15 06:36:05 UTC
[jira] [Resolved] (TAJO-1720) When converting type error in Parquet
occurs, query execution doesn't stop.
[ https://issues.apache.org/jira/browse/TAJO-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyunsik Choi resolved TAJO-1720.
--------------------------------
Resolution: Won't Fix
It was already fixed in another jira.
> When converting type error in Parquet occurs, query execution doesn't stop.
> ---------------------------------------------------------------------------
>
> Key: TAJO-1720
> URL: https://issues.apache.org/jira/browse/TAJO-1720
> Project: Tajo
> Issue Type: Bug
> Components: Storage
> Affects Versions: 0.10.1
> Reporter: Jongyoung Park
> Labels: parquet
> Attachments: tajo-eminency-worker-Jongyoungui-MBP.log
>
>
> I am testing Tajo with TPC-H dataset.
> The scheme of Lineitem table is like following :
> {code:sql}
> create external table lineitem_parquet (
> L_ORDERKEY bigint,
> L_PARTKEY bigint,
> L_SUPPKEY bigint,
> L_LINENUMBER bigint,
> L_QUANTITY double,
> L_EXTENDEDPRICE double,
> L_DISCOUNT double,
> L_TAX double,
> L_RETURNFLAG text,
> L_LINESTATUS text,
> L_SHIPDATE date,
> L_COMMITDATE date,
> L_RECEIPTDATE date,
> L_SHIPINSTRUCT text,
> L_SHIPMODE text,
> L_COMMENT text)
> using parquet location 'hdfs://localhost:8020/tpch/parquet/lineitem';
> {code}
> After that, I did 'insert overwrite' into the table from lineitem as text file.
> Because Tajo Parquet module doesn't support DATE type currently, it is clear that converting error occurs.
> {quote}
> 2015-07-29 16:18:13,826 ERROR org.apache.tajo.worker.TaskImpl: Cannot convert Tajo type: DATE
> java.lang.RuntimeException: Cannot convert Tajo type: DATE
> at org.apache.tajo.storage.parquet.TajoSchemaConverter.convertColumn(TajoSchemaConverter.java:191)
> at org.apache.tajo.storage.parquet.TajoSchemaConverter.convert(TajoSchemaConverter.java:150)
> at org.apache.tajo.storage.parquet.TajoWriteSupport.<init>(TajoWriteSupport.java:53)
> at org.apache.tajo.storage.parquet.TajoParquetWriter.<init>(TajoParquetWriter.java:80)
> at org.apache.tajo.storage.parquet.ParquetAppender.init(ParquetAppender.java:78)
> at org.apache.tajo.engine.planner.physical.StoreTableExec.openNewFile(StoreTableExec.java:119)
> at org.apache.tajo.engine.planner.physical.StoreTableExec.init(StoreTableExec.java:79)
> at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:398)
> at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2015-07-29 16:18:13,827 DEBUG org.apache.tajo.storage.text.DelimitedTextFile: DelimitedTextFileScanner processed record:0
> 2015-07-29 16:18:13,828 ERROR org.apache.tajo.worker.TaskContainer:
> java.lang.NullPointerException
> at org.apache.tajo.storage.parquet.ParquetAppender.getStats(ParquetAppender.java:147)
> at org.apache.tajo.engine.planner.physical.StoreTableExec.close(StoreTableExec.java:156)
> at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:411)
> at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2015-07-29 16:18:13,828 INFO org.apache.tajo.worker.TaskAttemptContext: Query status of ta_1438154071434_0001_000001_000000_00 is changed to TA_FAILED
> 2015-07-29 16:18:13,830 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.tajo.master.event.TaskFatalErrorEvent.EventType: TA_FATAL_ERROR
> 2015-07-29 16:18:13,830 DEBUG org.apache.tajo.querymaster.DefaultTaskScheduler: Try to schedule task resources: 2
> {quote}
> But query execution didn't stop and progress 0% was being displayed endlessly.
> It looks it should display corresponding error and stop.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)