You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by "Khurram Faraaz (JIRA)" <ji...@apache.org> on 2016/02/05 07:43:39 UTC
[jira] [Created] (DRILL-4356) Subsequent refresh table metadata
command returns in-correct message to user.
Khurram Faraaz created DRILL-4356:
-------------------------------------
Summary: Subsequent refresh table metadata command returns in-correct message to user.
Key: DRILL-4356
URL: https://issues.apache.org/jira/browse/DRILL-4356
Project: Apache Drill
Issue Type: Bug
Components: Storage - Parquet
Affects Versions: 1.5.0
Environment: 4 node cluster CentOS
Reporter: Khurram Faraaz
Subsequent refresh table metadata command returns in-correct message to user.
git commit ID: ca53c244
MapR Drill 1.5.0
JDK8
MapR FS 5.0.0 GA
{noformat}
CREATE TABLE drill4349 AS SELECT CAST(columns[0] AS INT) AS id, CAST(NULLIF(columns[1], '') AS DOUBLE) AS val from `manyNullsInColum.csv`;
0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
+-------+-----------------------------------------------------+
| ok | summary |
+-------+-----------------------------------------------------+
| true | Successfully updated metadata for table drill4349. |
+-------+-----------------------------------------------------+
1 row selected (0.336 seconds)
[root@centos-01 ~]# hadoop fs -ls /tmp/drill4349
Found 2 items
-rwxr-xr-x 3 mapr mapr 764 2016-02-04 09:00 /tmp/drill4349/.drill.parquet_metadata
-rwxr-xr-x 3 mapr mapr 201205 2016-02-04 08:44 /tmp/drill4349/0_0_0.parquet
Delete the parquet_metadata cache file.
[root@centos-01 ~]# hadoop fs -rm -r /tmp/drill4349/.drill.parquet_metadata
16/02/05 06:11:49 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
16/02/05 06:11:49 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
Deleted /tmp/drill4349/.drill.parquet_metadata
Re-run refresh table metadata command, this one succeeds as expected.
0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
+-------+-----------------------------------------------------+
| ok | summary |
+-------+-----------------------------------------------------+
| true | Successfully updated metadata for table drill4349. |
+-------+-----------------------------------------------------+
1 row selected (0.336 seconds)
However, a subsequent refresh table metadata command fails with message that says, Support is currently limited to single-directory-based Parquet tables.
0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
+----+---------+
| ok | summary |
+----+---------+
| false | Table drill4349 does not support metadata refresh. Support is currently limited to single-directory-based Parquet tables. |
+----+---------+
1 row selected (0.299 seconds)
{noformat}
Interesting to note that in drillbit.log, in the entry corresponding to the second failed refresh table metadata command, is also reported as FINISHED and no errors/message is logged to indicate that it failed.
{noformat}
2016-02-05 06:11:59,494 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id 294bc2cf-8a84-2f98-68a2-2aefe3d27b43: refresh table metadata drill4349
2016-02-05 06:11:59,728 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1 using 1 threads. Time: 3ms total, 3.565858ms avg, 3ms max.
2016-02-05 06:11:59,728 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1 using 1 threads. Earliest start: 1.093000 μs, Latest start: 1.093000 μs, Average start: 1.093000 μs .
2016-02-05 06:11:59,756 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State change requested AWAITING_ALLOCATION --> RUNNING
2016-02-05 06:11:59,756 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State to report: RUNNING
2016-02-05 06:11:59,761 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State change requested RUNNING --> FINISHED
2016-02-05 06:11:59,761 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State to report: FINISHED
2016-02-05 06:12:07,564 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id 294bc2c7-9f94-16b1-442f-eb6dc74d707a: refresh table metadata drill4349
2016-02-05 06:12:07,778 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:foreman] INFO o.a.d.exec.store.parquet.Metadata - Took 13 ms to read metadata from cache file
2016-02-05 06:12:07,791 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State change requested AWAITING_ALLOCATION --> RUNNING
2016-02-05 06:12:07,791 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State to report: RUNNING
2016-02-05 06:12:07,796 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State change requested RUNNING --> FINISHED
2016-02-05 06:12:07,796 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State to report: FINISHED
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)