You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Sergey Shelukhin <se...@hortonworks.com> on 2016/02/12 03:58:58 UTC
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/
-----------------------------------------------------------
(Updated Feb. 12, 2016, 2:58 a.m.)
Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
Repository: hive-git
Description
-------
see jira
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cd17a84
metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 564fca4
metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 07c20bf
metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 520ae74
ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java f36f707
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java a92c002
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 9b1d7ae
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
Diff: https://reviews.apache.org/r/40415/diff/
Testing
-------
Thanks,
Sergey Shelukhin
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by Sergey Shelukhin <se...@hortonworks.com>.
> On Feb. 16, 2016, 7:38 p.m., Prasanth_J wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java, line 1523
> > <https://reviews.apache.org/r/40415/diff/3/?file=1240578#file1240578line1523>
> >
> > Then why provide this interface? Is it client's responsibility as to when to use this interface?
Yes :)
> On Feb. 16, 2016, 7:38 p.m., Prasanth_J wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java, line 484
> > <https://reviews.apache.org/r/40415/diff/3/?file=1240581#file1240581line484>
> >
> > Is it intentional? Stacktrace will be lost.
Yeah, this exception is not very useful, it's usually just the fact that the FS doesn't support file IDs.
> On Feb. 16, 2016, 7:38 p.m., Prasanth_J wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java, line 41
> > <https://reviews.apache.org/r/40415/diff/3/?file=1240583#file1240583line41>
> >
> > what is the byteBuffer arg contain? What is the return type? It's unclear from the function signature. I think it will be better to do the conversion from SplitInfo to ByteBuffer just right before we send it across the wire. SplitInfo can be generic across file formats.
changed the return type; the arg is file metadata, documented on the interface, I renamed it for clarity
> On Feb. 16, 2016, 7:38 p.m., Prasanth_J wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java, line 625
> > <https://reviews.apache.org/r/40415/diff/3/?file=1240584#file1240584line625>
> >
> > nit. are there multiple caches?
yes, by conf. Usually there would only be one. Renamed for clarity.
> On Feb. 16, 2016, 7:38 p.m., Prasanth_J wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java, line 1002
> > <https://reviews.apache.org/r/40415/diff/3/?file=1240584#file1240584line1002>
> >
> > same as above. is it required to drop the stacktrace and just print the message?
Yes, stacktrace is annoying and pointless :) In case if there's some FS issue, the other (non-ID) API will probably also fail and produce a proper stack trace
- Sergey
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/#review119116
-----------------------------------------------------------
On Feb. 26, 2016, 5:04 a.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40415/
> -----------------------------------------------------------
>
> (Updated Feb. 26, 2016, 5:04 a.m.)
>
>
> Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see jira
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e868500
> metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java b5c4d1d
> metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java cb092d1
> metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
> orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
> ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 520ae74
> ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java d175d2d
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
> ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java aa1c925
> ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 6f84708
> ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
>
> Diff: https://reviews.apache.org/r/40415/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by j....@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/#review119116
-----------------------------------------------------------
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java (line 2230)
<https://reviews.apache.org/r/40415/#comment180526>
nit: use single arg ctor instead?
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java (line 2255)
<https://reviews.apache.org/r/40415/#comment180527>
nit: same as above. use 2 arg ctor instead?
metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java (line 1523)
<https://reviews.apache.org/r/40415/#comment180528>
Then why provide this interface? Is it client's responsibility as to when to use this interface?
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java (line 484)
<https://reviews.apache.org/r/40415/#comment180529>
Is it intentional? Stacktrace will be lost.
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java (line 41)
<https://reviews.apache.org/r/40415/#comment180533>
what is the byteBuffer arg contain? What is the return type? It's unclear from the function signature. I think it will be better to do the conversion from SplitInfo to ByteBuffer just right before we send it across the wire. SplitInfo can be generic across file formats.
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java (line 625)
<https://reviews.apache.org/r/40415/#comment180530>
nit. are there multiple caches?
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java (line 998)
<https://reviews.apache.org/r/40415/#comment180531>
same as above. is it required to drop the stacktrace and just print the message?
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java (line 2002)
<https://reviews.apache.org/r/40415/#comment180657>
OrcInputFormat is becoming the next SemanticAnalyzer. Too many inner classes/interfaces. :)
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java (line 2312)
<https://reviews.apache.org/r/40415/#comment180380>
In general, ORC PPD is controlled by hive.optimizer.index.filter config. If this config is disabled, we should disable metastore footer cache ppd as well.
- Prasanth_J
On Feb. 12, 2016, 2:58 a.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40415/
> -----------------------------------------------------------
>
> (Updated Feb. 12, 2016, 2:58 a.m.)
>
>
> Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see jira
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java cd17a84
> metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 564fca4
> metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 07c20bf
> metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
> orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
> ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 520ae74
> ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java f36f707
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
> ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java a92c002
> ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 9b1d7ae
> ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
>
> Diff: https://reviews.apache.org/r/40415/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by Sergey Shelukhin <se...@hortonworks.com>.
> On March 7, 2016, 7:27 a.m., Prasanth_J wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java, line 2223
> > <https://reviews.apache.org/r/40415/diff/6/?file=1276199#file1276199line2223>
> >
> > use ThreadLocal?
Not sure that helps. Also, it's a threadlocal... Remembering the result for some time could be a good solution in a follow-up, but we assume by default that if the user enables it they know what they are doing, or pay a small perf penalty
- Sergey
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/#review122261
-----------------------------------------------------------
On March 2, 2016, 2:30 a.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40415/
> -----------------------------------------------------------
>
> (Updated March 2, 2016, 2:30 a.m.)
>
>
> Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see jira
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2723dad
> metastore/src/java/org/apache/hadoop/hive/metastore/FileFormatProxy.java ec0be2b
> metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java b5c4d1d
> metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java cb092d1
> metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
> orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
> ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 9bf9377
> ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/ExternalCache.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/LocalCache.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/MetastoreExternalCachesByConf.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
> ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java fdc7956
> ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 4fafe8c
> ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
>
> Diff: https://reviews.apache.org/r/40415/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by j....@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/#review122261
-----------------------------------------------------------
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java (line 2222)
<https://reviews.apache.org/r/40415/#comment184177>
use ThreadLocal?
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java (line 2251)
<https://reviews.apache.org/r/40415/#comment184178>
Can this be set here or provide a separate API for not getting footer? We would otherwise transfer more data through wire.
orc/src/java/org/apache/orc/impl/InStream.java
<https://reviews.apache.org/r/40415/#comment184179>
nit: Unrelated changes.
- Prasanth_J
On March 2, 2016, 2:30 a.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40415/
> -----------------------------------------------------------
>
> (Updated March 2, 2016, 2:30 a.m.)
>
>
> Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see jira
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2723dad
> metastore/src/java/org/apache/hadoop/hive/metastore/FileFormatProxy.java ec0be2b
> metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java b5c4d1d
> metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java cb092d1
> metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
> orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
> ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 9bf9377
> ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/ExternalCache.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/LocalCache.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/MetastoreExternalCachesByConf.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
> ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java fdc7956
> ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 4fafe8c
> ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
>
> Diff: https://reviews.apache.org/r/40415/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/
-----------------------------------------------------------
(Updated March 9, 2016, 2:07 a.m.)
Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
Repository: hive-git
Description
-------
see jira
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5098851
metastore/src/java/org/apache/hadoop/hive/metastore/FileFormatProxy.java ec0be2b
metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 0e8a157
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 9048d45
metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 62677d1
metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 9bf9377
ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
ql/src/java/org/apache/hadoop/hive/ql/io/orc/ExternalCache.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/LocalCache.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/MetastoreExternalCachesByConf.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java fdc7956
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 4fafe8c
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
Diff: https://reviews.apache.org/r/40415/diff/
Testing
-------
Thanks,
Sergey Shelukhin
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by j....@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/#review122384
-----------------------------------------------------------
Ship it!
Ship It!
- Prasanth_J
On March 7, 2016, 9:36 p.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40415/
> -----------------------------------------------------------
>
> (Updated March 7, 2016, 9:36 p.m.)
>
>
> Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see jira
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5098851
> metastore/src/java/org/apache/hadoop/hive/metastore/FileFormatProxy.java ec0be2b
> metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java b5c4d1d
> metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java cb092d1
> metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
> orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
> ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 9bf9377
> ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/ExternalCache.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/LocalCache.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/MetastoreExternalCachesByConf.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
> ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java ad17096
> ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 4fafe8c
> ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
> storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
>
> Diff: https://reviews.apache.org/r/40415/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/
-----------------------------------------------------------
(Updated March 7, 2016, 9:36 p.m.)
Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
Repository: hive-git
Description
-------
see jira
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5098851
metastore/src/java/org/apache/hadoop/hive/metastore/FileFormatProxy.java ec0be2b
metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java b5c4d1d
metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java cb092d1
metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 9bf9377
ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
ql/src/java/org/apache/hadoop/hive/ql/io/orc/ExternalCache.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/LocalCache.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/MetastoreExternalCachesByConf.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java ad17096
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 4fafe8c
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
Diff: https://reviews.apache.org/r/40415/diff/
Testing
-------
Thanks,
Sergey Shelukhin
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/
-----------------------------------------------------------
(Updated March 7, 2016, 9:32 p.m.)
Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
Repository: hive-git
Description
-------
see jira
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5098851
metastore/src/java/org/apache/hadoop/hive/metastore/FileFormatProxy.java ec0be2b
metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java b5c4d1d
metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java cb092d1
metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 9bf9377
ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
ql/src/java/org/apache/hadoop/hive/ql/io/orc/ExternalCache.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/LocalCache.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/MetastoreExternalCachesByConf.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java ad17096
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 4fafe8c
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
Diff: https://reviews.apache.org/r/40415/diff/
Testing
-------
Thanks,
Sergey Shelukhin
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/
-----------------------------------------------------------
(Updated March 2, 2016, 2:30 a.m.)
Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
Repository: hive-git
Description
-------
see jira
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2723dad
metastore/src/java/org/apache/hadoop/hive/metastore/FileFormatProxy.java ec0be2b
metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java b5c4d1d
metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java cb092d1
metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 9bf9377
ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
ql/src/java/org/apache/hadoop/hive/ql/io/orc/ExternalCache.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/LocalCache.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/MetastoreExternalCachesByConf.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 0ebcd2a
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java fdc7956
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 4fafe8c
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
Diff: https://reviews.apache.org/r/40415/diff/
Testing
-------
Thanks,
Sergey Shelukhin
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/
-----------------------------------------------------------
(Updated Feb. 26, 2016, 8:12 p.m.)
Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
Repository: hive-git
Description
-------
see jira
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e868500
metastore/src/java/org/apache/hadoop/hive/metastore/FileFormatProxy.java ec0be2b
metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java b5c4d1d
metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java cb092d1
metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 520ae74
ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
ql/src/java/org/apache/hadoop/hive/ql/io/orc/ExternalCache.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/LocalCache.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/MetastoreExternalCachesByConf.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java d175d2d
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java aa1c925
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 6f84708
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
Diff: https://reviews.apache.org/r/40415/diff/
Testing
-------
Thanks,
Sergey Shelukhin
Re: Review Request 40415: HIVE-11675 make use of file footer PPD API
in ETL strategy or separate strategy
Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40415/
-----------------------------------------------------------
(Updated Feb. 26, 2016, 5:04 a.m.)
Review request for hive, Gopal V, Prasanth_J, and Vikram Dixit Kumaraswamy.
Repository: hive-git
Description
-------
see jira
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e868500
metastore/src/java/org/apache/hadoop/hive/metastore/FileMetadataHandler.java bd4e188
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java b5c4d1d
metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java cb092d1
metastore/src/java/org/apache/hadoop/hive/metastore/filemeta/OrcFileMetadataHandler.java 1b388aa
orc/src/java/org/apache/orc/impl/InStream.java b1c6de5
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 520ae74
ql/src/java/org/apache/hadoop/hive/ql/io/HdfsUtils.java af64fc8
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcFileFormatProxy.java ef76723
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java d175d2d
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcNewInputFormat.java 2782d7e
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java aa1c925
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 6f84708
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSplitElimination.java 7a93b54
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/PredicateLeaf.java dc71db4
storage-api/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgumentImpl.java be5e67b
Diff: https://reviews.apache.org/r/40415/diff/
Testing
-------
Thanks,
Sergey Shelukhin