You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by "Arina Ielchiieva (JIRA)" <ji...@apache.org> on 2017/07/04 15:41:00 UTC
[jira] [Resolved] (DRILL-4356) Subsequent refresh table metadata
command returns in-correct message to user.
[ https://issues.apache.org/jira/browse/DRILL-4356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arina Ielchiieva resolved DRILL-4356.
-------------------------------------
Resolution: Fixed
> Subsequent refresh table metadata command returns in-correct message to user.
> -----------------------------------------------------------------------------
>
> Key: DRILL-4356
> URL: https://issues.apache.org/jira/browse/DRILL-4356
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
> Affects Versions: 1.5.0
> Environment: 4 node cluster CentOS
> Reporter: Khurram Faraaz
>
> Subsequent refresh table metadata command returns in-correct message to user.
> git commit ID: ca53c244
> MapR Drill 1.5.0
> JDK8
> MapR FS 5.0.0 GA
>
> {noformat}
> CREATE TABLE drill4349 AS SELECT CAST(columns[0] AS INT) AS id, CAST(NULLIF(columns[1], '') AS DOUBLE) AS val from `manyNullsInColum.csv`;
> 0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
> +-------+-----------------------------------------------------+
> | ok | summary |
> +-------+-----------------------------------------------------+
> | true | Successfully updated metadata for table drill4349. |
> +-------+-----------------------------------------------------+
> 1 row selected (0.336 seconds)
> [root@centos-01 ~]# hadoop fs -ls /tmp/drill4349
> Found 2 items
> -rwxr-xr-x 3 mapr mapr 764 2016-02-04 09:00 /tmp/drill4349/.drill.parquet_metadata
> -rwxr-xr-x 3 mapr mapr 201205 2016-02-04 08:44 /tmp/drill4349/0_0_0.parquet
> Delete the parquet_metadata cache file.
> [root@centos-01 ~]# hadoop fs -rm -r /tmp/drill4349/.drill.parquet_metadata
> 16/02/05 06:11:49 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
> 16/02/05 06:11:49 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
> Deleted /tmp/drill4349/.drill.parquet_metadata
> Re-run refresh table metadata command, this one succeeds as expected.
> 0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
> +-------+-----------------------------------------------------+
> | ok | summary |
> +-------+-----------------------------------------------------+
> | true | Successfully updated metadata for table drill4349. |
> +-------+-----------------------------------------------------+
> 1 row selected (0.336 seconds)
> However, a subsequent refresh table metadata command fails with message that says, Support is currently limited to single-directory-based Parquet tables.
> 0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
> +----+---------+
> | ok | summary |
> +----+---------+
> | false | Table drill4349 does not support metadata refresh. Support is currently limited to single-directory-based Parquet tables. |
> +----+---------+
> 1 row selected (0.299 seconds)
> {noformat}
> Interesting to note that in drillbit.log, in the entry corresponding to the second failed refresh table metadata command, is also reported as FINISHED and no errors/message is logged to indicate that it failed.
> {noformat}
> 2016-02-05 06:11:59,494 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id 294bc2cf-8a84-2f98-68a2-2aefe3d27b43: refresh table metadata drill4349
> 2016-02-05 06:11:59,728 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1 using 1 threads. Time: 3ms total, 3.565858ms avg, 3ms max.
> 2016-02-05 06:11:59,728 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1 using 1 threads. Earliest start: 1.093000 μs, Latest start: 1.093000 μs, Average start: 1.093000 μs .
> 2016-02-05 06:11:59,756 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State change requested AWAITING_ALLOCATION --> RUNNING
> 2016-02-05 06:11:59,756 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State to report: RUNNING
> 2016-02-05 06:11:59,761 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State change requested RUNNING --> FINISHED
> 2016-02-05 06:11:59,761 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State to report: FINISHED
> 2016-02-05 06:12:07,564 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id 294bc2c7-9f94-16b1-442f-eb6dc74d707a: refresh table metadata drill4349
> 2016-02-05 06:12:07,778 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:foreman] INFO o.a.d.exec.store.parquet.Metadata - Took 13 ms to read metadata from cache file
> 2016-02-05 06:12:07,791 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State change requested AWAITING_ALLOCATION --> RUNNING
> 2016-02-05 06:12:07,791 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State to report: RUNNING
> 2016-02-05 06:12:07,796 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State change requested RUNNING --> FINISHED
> 2016-02-05 06:12:07,796 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State to report: FINISHED
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)