You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by StevenMPhillips <gi...@git.apache.org> on 2015/10/02 07:45:13 UTC
[GitHub] drill pull request: Parquet meta
GitHub user StevenMPhillips opened a pull request:
https://github.com/apache/drill/pull/186
Parquet meta
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StevenMPhillips/drill parquet_meta
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/186.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #186
----
commit c3cf974db0fe437f893fa9c1fa907620d93c994f
Author: Steven Phillips <sm...@apache.org>
Date: 2015-10-01T23:54:18Z
DRILL-3887: Fix bug where metadata file not being used
commit 96534e3a5502524171df569373cc47e9860de466
Author: Steven Phillips <sm...@apache.org>
Date: 2015-10-02T03:36:15Z
DRILL-3867: Store relative paths in metadata file
commit 728d6e6b12eec11d52e9f106deecaf311ebd7958
Author: Steven Phillips <sm...@apache.org>
Date: 2015-10-02T03:51:29Z
DRILL-3820: Set 700 permission on metadata file
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] drill pull request: Parquet meta
Posted by StevenMPhillips <gi...@git.apache.org>.
Github user StevenMPhillips commented on a diff in the pull request:
https://github.com/apache/drill/pull/186#discussion_r41051093
--- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java ---
@@ -716,6 +726,7 @@ public String toString() {
return "ParquetGroupScan [entries=" + entries
+ ", selectionRoot=" + selectionRoot
+ ", numFiles=" + getEntries().size()
+ + ", usedCache=" + usedMetadataCache
--- End diff --
We only use the cache if there is a cache file for the root directory of the query.
I can change the name to useMetadataFile
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] drill pull request: Parquet meta
Posted by amansinha100 <gi...@git.apache.org>.
Github user amansinha100 commented on a diff in the pull request:
https://github.com/apache/drill/pull/186#discussion_r41035620
--- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java ---
@@ -716,6 +726,7 @@ public String toString() {
return "ParquetGroupScan [entries=" + entries
+ ", selectionRoot=" + selectionRoot
+ ", numFiles=" + getEntries().size()
+ + ", usedCache=" + usedMetadataCache
--- End diff --
This is a useful attribute; however usedCache might be misinterpreted to mean data cache. usedMetadataCache is more explicit (although the term 'cache' itself is misleading - here we are referring to file cache on disk, not in-memory). Another thing: is a boolean value sufficient ? if there are 2 subdirectories and only one of them has the metadata cache, and I query the parent directory: would the cache get used for one of the subdirectories and if so, what would this flag indicate ? It sounds like if the parent directory has the cache, then it will be true, otherwise false.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] drill pull request: Parquet meta
Posted by amansinha100 <gi...@git.apache.org>.
Github user amansinha100 commented on a diff in the pull request:
https://github.com/apache/drill/pull/186#discussion_r41053297
--- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java ---
@@ -716,6 +726,7 @@ public String toString() {
return "ParquetGroupScan [entries=" + entries
+ ", selectionRoot=" + selectionRoot
+ ", numFiles=" + getEntries().size()
+ + ", usedCache=" + usedMetadataCache
--- End diff --
Sounds good. I'll make sure the docs are consistent with the behavior.
Could you change the name to 'usedMetadataFile' (instead of 'use').
Rest of 3887 looks good to me. +1 . I have not reviewed the other changes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---