[jira] [Updated] (DRILL-3827) Empty metadata file causes queries on the table to fail

["Zelaine Fong (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DRILL-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zelaine Fong updated DRILL-3827:
--------------------------------
    Fix Version/s: 1.3.0

> Empty metadata file causes queries on the table to fail
> -------------------------------------------------------
>
>                 Key: DRILL-3827
>                 URL: https://issues.apache.org/jira/browse/DRILL-3827
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 1.2.0
>            Reporter: Victoria Markman
>            Assignee: Parth Chandra
>            Priority: Critical
>             Fix For: 1.3.0
>
>
> I ran into a situation where drill created an empty metadata file (which is a separate issue and I will try to narrow it down. Suspicion is that this happens when "refresh table metada x" fails with "permission denied" error).
> However, we need to guard against situation where metadata file is empty or corrupted. We probably should skip reading it if we encounter unexpected result and continue with query planning without that information. In the same fashion as partition pruning failure. It's also important to log this information somewhere, drillbit.log as a start. It would be really nice to have a flag in the query profile that tells a user if we used metadata file for planning or not. Will help in debugging performance issues.
> Very confusing exception is thrown if you have zero length meta data file in the directory:
> {code}
> [Wed Sep 23 07:45:28] # ls -la
> total 2
> drwxr-xr-x  2 root root   2 Sep 10 14:55 .
> drwxr-xr-x 16 root root  35 Sep 15 12:54 ..
> -rwxr-xr-x  1 root root 483 Jul  1 11:29 0_0_0.parquet
> -rwxr-xr-x  1 root root   0 Sep 10 14:55 .drill.parquet_metadata
> 0: jdbc:drill:schema=dfs> select * from t1;
> Error: SYSTEM ERROR: JsonMappingException: No content to map due to end-of-input
>  at [Source: com.mapr.fs.MapRFsDataInputStream@342bd88d; line: 1, column: 1]
> [Error Id: c97574f6-b3e8-4183-8557-c30df6ca675f on atsqa4-133.qa.lab:31010] (state=,code=0)
> {code}
> Workaround is trivial, remove the file. Marking it as critical, since we don't have any concurrency control in place and this file can get corrupted as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)