You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@orc.apache.org by "Eugene Koifman (JIRA)" <ji...@apache.org> on 2017/05/18 19:41:04 UTC
[jira] [Updated] (ORC-195) FileFormatException should include file
name in the message
[ https://issues.apache.org/jira/browse/ORC-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koifman updated ORC-195:
-------------------------------
Description:
Here is 1 example:
{noformat}
ReaderImpl.extractFileTail(FileSystem fs, Path path, long maxFileLength) throws IOException
{noformat}
has
{noformat}
if (size <= OrcFile.MAGIC.length()) {
throw new FileFormatException("Not a valid ORC file");
}
{noformat}
which in the logs looks like
{noformat}
2017-05-18T12:08:23,572 WARN [Thread-360] mapred.LocalJobRunner: job_local150767050_0007
java.lang.Exception: org.apache.orc.FileFormatException: Not a valid ORC file
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489) ~[hadoop-mapreduce-client-common-2.8.0.jar:?]
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549) [hadoop-mapreduce-client-common-2.8.0.jar:?]
Caused by: org.apache.orc.FileFormatException: Not a valid ORC file
at org.apache.orc.impl.ReaderImpl.extractFileTail(ReaderImpl.java:511) ~[orc-core-1.3.3.jar:1.3.3]
at org.apache.orc.impl.ReaderImpl.<init>(ReaderImpl.java:378) ~[orc-core-1.3.3.jar:1.3.3]
at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.<init>(ReaderImpl.java:63) ~[classes/:?]
at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:90) ~[classes/:?]
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRawReader(OrcInputFormat.java:2279) ~[classes/:?]
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorMap.map(CompactorMR.java:665) ~[classes/:?]
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorMap.map(CompactorMR.java:642) ~[classes/:?]
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) ~[hadoop-mapreduce-client-core-2.8.0.jar:?]
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) ~[hadoop-mapreduce-client-core-2.8.0.jar:?]
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) ~[hadoop-mapreduce-client-core-2.8.0.jar:?]
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270) ~[hadoop-mapreduce-client-common-2.8.0.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_25]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_25]
at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_25]
{noformat}
was:
Here is 1 example:
{noformat}
ReaderImpl.extractFileTail(FileSystem fs, Path path, long maxFileLength) throws IOException
has
if (size <= OrcFile.MAGIC.length()) {
throw new FileFormatException("Not a valid ORC file");
}
{noformat}
which in the logs looks like
{noformat}
2017-05-18T12:08:23,572 WARN [Thread-360] mapred.LocalJobRunner: job_local150767050_0007
java.lang.Exception: org.apache.orc.FileFormatException: Not a valid ORC file
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489) ~[hadoop-mapreduce-client-common-2.8.0.jar:?]
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549) [hadoop-mapreduce-client-common-2.8.0.jar:?]
Caused by: org.apache.orc.FileFormatException: Not a valid ORC file
at org.apache.orc.impl.ReaderImpl.extractFileTail(ReaderImpl.java:511) ~[orc-core-1.3.3.jar:1.3.3]
at org.apache.orc.impl.ReaderImpl.<init>(ReaderImpl.java:378) ~[orc-core-1.3.3.jar:1.3.3]
at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.<init>(ReaderImpl.java:63) ~[classes/:?]
at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:90) ~[classes/:?]
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRawReader(OrcInputFormat.java:2279) ~[classes/:?]
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorMap.map(CompactorMR.java:665) ~[classes/:?]
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorMap.map(CompactorMR.java:642) ~[classes/:?]
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) ~[hadoop-mapreduce-client-core-2.8.0.jar:?]
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) ~[hadoop-mapreduce-client-core-2.8.0.jar:?]
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) ~[hadoop-mapreduce-client-core-2.8.0.jar:?]
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270) ~[hadoop-mapreduce-client-common-2.8.0.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_25]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_25]
at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_25]
{noformat}
> FileFormatException should include file name in the message
> -----------------------------------------------------------
>
> Key: ORC-195
> URL: https://issues.apache.org/jira/browse/ORC-195
> Project: ORC
> Issue Type: Bug
> Affects Versions: 1.3.3
> Reporter: Eugene Koifman
>
> Here is 1 example:
> {noformat}
> ReaderImpl.extractFileTail(FileSystem fs, Path path, long maxFileLength) throws IOException
> {noformat}
> has
> {noformat}
> if (size <= OrcFile.MAGIC.length()) {
> throw new FileFormatException("Not a valid ORC file");
> }
> {noformat}
> which in the logs looks like
> {noformat}
> 2017-05-18T12:08:23,572 WARN [Thread-360] mapred.LocalJobRunner: job_local150767050_0007
> java.lang.Exception: org.apache.orc.FileFormatException: Not a valid ORC file
> at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489) ~[hadoop-mapreduce-client-common-2.8.0.jar:?]
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549) [hadoop-mapreduce-client-common-2.8.0.jar:?]
> Caused by: org.apache.orc.FileFormatException: Not a valid ORC file
> at org.apache.orc.impl.ReaderImpl.extractFileTail(ReaderImpl.java:511) ~[orc-core-1.3.3.jar:1.3.3]
> at org.apache.orc.impl.ReaderImpl.<init>(ReaderImpl.java:378) ~[orc-core-1.3.3.jar:1.3.3]
> at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.<init>(ReaderImpl.java:63) ~[classes/:?]
> at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:90) ~[classes/:?]
> at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRawReader(OrcInputFormat.java:2279) ~[classes/:?]
> at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorMap.map(CompactorMR.java:665) ~[classes/:?]
> at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorMap.map(CompactorMR.java:642) ~[classes/:?]
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) ~[hadoop-mapreduce-client-core-2.8.0.jar:?]
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) ~[hadoop-mapreduce-client-core-2.8.0.jar:?]
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) ~[hadoop-mapreduce-client-core-2.8.0.jar:?]
> at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270) ~[hadoop-mapreduce-client-common-2.8.0.jar:?]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_25]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_25]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_25]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_25]
> at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_25]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)