You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Amareshwari Sriramadasu <am...@apache.org> on 2016/01/04 06:47:32 UTC
Re: Review Request 41794: LENS-518: Move Hive dependency to Apache
Hive
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41794/#review112495
-----------------------------------------------------------
lens-cube/src/main/java/org/apache/lens/cube/parse/CubeSemanticAnalyzer.java (line 55)
<https://reviews.apache.org/r/41794/#comment172946>
What does false mean here ?
lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java (line 527)
<https://reviews.apache.org/r/41794/#comment172950>
See if we can add check KW_LOCAL for adding local here.
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/RemoteThriftConnection.java (line 89)
<https://reviews.apache.org/r/41794/#comment172951>
RetryingThriftClient should be available in Hive with HIVE-11482.
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java (line 139)
<https://reviews.apache.org/r/41794/#comment172952>
change the comment accordingly.
lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java (line 381)
<https://reviews.apache.org/r/41794/#comment172953>
whats the behavior is restoreSession is removed ?
pom.xml (line 849)
<https://reviews.apache.org/r/41794/#comment172945>
Move the version number to a property.
- Amareshwari Sriramadasu
On Dec. 31, 2015, 8:33 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41794/
> -----------------------------------------------------------
>
> (Updated Dec. 31, 2015, 8:33 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-518
> https://issues.apache.org/jira/browse/LENS-518
>
>
> Repository: lens
>
>
> Description
> -------
>
> Creating wish for moving to Apache hive dependency instead of forked hive dependency. We will create follow up issues in lens and hive make them link here.
>
>
> Diffs
> -----
>
> lens-api/pom.xml 8aa2f67a92d22cc6a807b44bdc7e451d09959768
> lens-cli/pom.xml 90824179e231d9cd82d8c3a067ea1815d9f3ff56
> lens-client/pom.xml 5f2f3a983cca87c8c8055f245cde6295f56381b3
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java e7550ca2c8307cba086e848e8b476a670366f7a5
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 1fd1d17bbc05b2ca07d88237b4143ea3cd904d0e
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeSemanticAnalyzer.java ebbe404d3a4a6b7e46adc94c0cec35e26cb17acd
> lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 6c3d4c3b0235f88c16aad148cbbbc30eca89eb7f
> lens-driver-es/pom.xml 5705cd2c291e2c0e4879bc8acc390f150881520e
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java c7ef8f124807be41ad9b36629d66ee8f16bbefb3
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveInMemoryResultSet.java f8abd782518cb08ed1607970f4c1fb4075ca4263
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/RemoteThriftConnection.java 741294cac27f610b94099c24d59adaa791776487
> lens-driver-jdbc/pom.xml f84513bb7641629b2758245f75a52a3ff1032185
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/ColumnarSQLRewriter.java 561452747da67feeab1f08fddbc8615324d2737f
> lens-examples/pom.xml 0a5a744d1e376c22914c6fe73e80f9c069ed5ee6
> lens-query-lib/pom.xml 2eec781034a8830aaf2c846dab899ac8fd8d22a2
> lens-server-api/pom.xml d836c162c9920e8608d60600427506a0d75f08e3
> lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java 81160f5b84e1b17a459681f82aa5cc9ba5714db0
> lens-server/src/main/java/org/apache/lens/server/BaseLensService.java 0821fe7bb12e3628f3a800059e0c3e813baca6d7
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e002a75b8ea54c9b8672d50bfc8af25f7483
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java cf49a136a5b540c515c5be6f976b1c21733b6b06
> lens-server/src/main/java/org/apache/lens/server/metastore/JAXBUtils.java 817c84c665c59bf8c7a75fd9c12355bfc1109663
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java abec2b338e0da6b56934a02d16c48a6a7917845a
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java 2745b272de0d8ce1501c5e3dbde1cb41ec72e8ef
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java cc62d92347c9308e80046e83125e3eb823c3cc6c
> lens-server/src/main/java/org/apache/lens/server/ui/StaticFileResource.java 7e0758975cc4d991c24a1df963095d38189f5622
> lens-server/src/test/java/org/apache/lens/server/session/TestSessionClassLoaders.java 5feab715b9a283e0951570a407b464b3bd7007a6
> lens-server/src/test/java/org/apache/lens/server/session/TestSessionExpiry.java dbdaaecb58e2c8dcad7b8ce0157f9f74cbb38484
> pom.xml 746204e87d046454fbd0e3f4690e3b220487418c
>
> Diff: https://reviews.apache.org/r/41794/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 41794: LENS-518: Move Hive dependency to Apache
Hive
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On Jan. 4, 2016, 11:17 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java, line 389
> > <https://reviews.apache.org/r/41794/diff/2/?file=1180115#file1180115line389>
> >
> > whats the behavior is restoreSession is removed ?
Added back restore functionality.
> On Jan. 4, 2016, 11:17 a.m., Amareshwari Sriramadasu wrote:
> > pom.xml, line 849
> > <https://reviews.apache.org/r/41794/diff/2/?file=1180120#file1180120line849>
> >
> > Move the version number to a property.
Yes, WIP patch.
> On Jan. 4, 2016, 11:17 a.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/CubeSemanticAnalyzer.java, line 55
> > <https://reviews.apache.org/r/41794/diff/2/?file=1180098#file1180098line55>
> >
> > What does false mean here ?
In the super class, reset is:
```
@Override
protected void reset(boolean clearPartsCache) {
super.reset(true);
if(clearPartsCache) {
prunedPartitions.clear();
}
loadTableWork.clear();
loadFileWork.clear();
topOps.clear();
destTableId = 1;
idToTableNameMap.clear();
qb = null;
ast = null;
uCtx = null;
joinContext.clear();
smbMapJoinContext.clear();
opParseCtx.clear();
groupOpToInputTables.clear();
disableJoinMerge = false;
aliasToCTEs.clear();
topToTable.clear();
opToPartPruner.clear();
opToPartList.clear();
opToPartToSkewedPruner.clear();
opToSamplePruner.clear();
nameToSplitSample.clear();
fsopToTable.clear();
resultSchema = null;
createVwDesc = null;
viewsExpanded = null;
viewSelect = null;
ctesExpanded = null;
globalLimitCtx.disableOpt();
viewAliasToInput.clear();
reduceSinkOperatorsAddedByEnforceBucketingSorting.clear();
topToTableProps.clear();
listMapJoinOpsNoReducer.clear();
unparseTranslator.clear();
queryProperties.clear();
outputs.clear();
}
```
And in inmobi-hive, reset doesn't take an argument but executes `prunedPartitions.clear();` without any condition. Hence passing true here.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41794/#review112495
-----------------------------------------------------------
On March 8, 2016, 1:35 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41794/
> -----------------------------------------------------------
>
> (Updated March 8, 2016, 1:35 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-518
> https://issues.apache.org/jira/browse/LENS-518
>
>
> Repository: lens
>
>
> Description
> -------
>
> Creating wish for moving to Apache hive dependency instead of forked hive dependency. We will create follow up issues in lens and hive make them link here.
>
>
> Diffs
> -----
>
> lens-cube/pom.xml 0533f44219d2828792738c8e9fedc7e354820e60
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 02bbbbd01372e2d8c76d98e83989d952e3dc0779
> lens-cube/src/main/java/org/apache/lens/cube/metadata/FactPartition.java 355a1f0d37d27beec207b4db9b6cce94d0d02268
> lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java 9318603f2ccb979644c49f863e4e841cd5ad3bd8
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java b8b6db94ccbad408415f23fa9aba6711c3d4e291
> lens-cube/src/main/java/org/apache/lens/cube/parse/CubeSemanticAnalyzer.java ebbe404d3a4a6b7e46adc94c0cec35e26cb17acd
> lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 8d6105f7a0917d9b625b4573807fdafa9fbeea6e
> lens-cube/src/main/java/org/apache/lens/cube/query/cost/FactPartitionBasedQueryCostCalculator.java 9fecdbccbf188efd39d770662df3606cbf7d766e
> lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java 65387c68d751358460814b64214675595b812e8f
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestHQLParser.java f9d7457facbf5fe42050fdf76448d6344f4eda67
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestQuery.java cd20fef09468891af5c18aac2a20f71be30afd77
> lens-cube/src/test/resources/hive-site.xml 8456f8b6090b953ddbe51604b9e20fbf664cbdc9
> lens-driver-es/pom.xml 89736ab8e82f147970dcbc85590a7113bd58597f
> lens-driver-es/src/test/java/org/apache/lens/driver/es/ESDriverTest.java ab6f22b34035849275074986466f8015b995cf42
> lens-driver-es/src/test/resources/hive-site.xml PRE-CREATION
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/EmbeddedThriftConnection.java 0b46485baf69551eb03ba89cfac980ead0d019eb
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java f4225439ed50b1bb10d9512423e19749f6383664
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveInMemoryResultSet.java 4d52e224a036bea128d2ec0f89d937966d8d1a5c
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveQueryPlan.java f53e2b65580fa01bf5049bcdd511f71cf598f4e7
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/RemoteThriftConnection.java 741294cac27f610b94099c24d59adaa791776487
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java ba3f3d59ec929ecc6b04b4b84e0dd6f68ad06fc0
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 4f18c2446e804b4127c1382359d7f38353ec9104
> lens-driver-hive/src/test/resources/hive-site.xml de5af502ca8347a962aef34756bfb8a22d67f6f8
> lens-driver-jdbc/pom.xml 0dd0fb5dd777021c45d9364e9f1ba1dd5d01fd4f
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/ColumnarSQLRewriter.java a004de950eaeb6fb4ac5a9663e3ddacac1f7c5ec
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestColumnarSQLRewriter.java 1aa74914ae524c5f5b7f20fbd0c8df74bb0fc162
> lens-driver-jdbc/src/test/resources/hive-site.xml 2cb1d8d47fc7161ea2667c34eb0c1bf0bd10945e
> lens-examples/pom.xml a13ee33c7f95a684fd0d2236bc4f39278235ea9b
> lens-query-lib/pom.xml 29dbbe10f90689f3c1610619ced9ce41d10bdf8f
> lens-query-lib/src/main/java/org/apache/lens/lib/query/FilePersistentFormatter.java b3e640f76ce1007fe5370d461aa3f072fa7876bb
> lens-query-lib/src/test/java/org/apache/lens/lib/query/TestFileSerdeFormatter.java e40d0c2a09490719ff3bc4cd1abb01ebeb3a8742
> lens-server-api/pom.xml 8304279318c02dc2beb251ec6e87f3f061c86e2b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/ExplainQueryContext.java c7b9887d58c4372faff2419c219da09a16558b15
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PreparedQueryContext.java 0b08459c1107ce3fc79390d092b4a251a1937c75
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java 96846c1b89b47a971916ab230bb7f4691d52b524
> lens-server/pom.xml 75b8b321d1f6586d4f19abad81e9e86cfe9eb5ea
> lens-server/src/main/java/org/apache/lens/server/BaseLensService.java be31cd88d10dc61840c5a2390408ebed126340ac
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e002a75b8ea54c9b8672d50bfc8af25f7483
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java a1acd1a7aecb31d2a497811afeb58b900c5f73df
> lens-server/src/main/java/org/apache/lens/server/metastore/JAXBUtils.java 6fd19a006d6a69b396aad9733593c9a6d383fc24
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 1c0cd3520ed77f12141042f74d69abbf0fd99790
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java 72f5c534b02a1fc51ca1fdd98954b113564878e9
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java 2745b272de0d8ce1501c5e3dbde1cb41ec72e8ef
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 895a819a8469d56eac2e7932ea37a391ee956b17
> lens-server/src/main/java/org/apache/lens/server/stats/store/log/StatisticsLogPartitionHandler.java 6d92ab9c1f66d0088ffca58f6ae36e6547c37be6
> lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 0f55d9e49312a1668dbff3712032359bb92545fc
> lens-server/src/test/java/org/apache/lens/server/auth/FooBarAuthenticationProvider.java d4c3a838be8364689a0c69898b95387a5bb905a0
> lens-server/src/test/java/org/apache/lens/server/common/RestAPITestUtil.java 0e39b52ce86b9c036d48c8fd395b96fa86bbcd9e
> lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java 3b091f5ea576eb72ae473e060bbb5b3bef95f106
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 699fa686f5feb865b029b4024e049520a22840da
> lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java f66f89d933e0c380acd56af921e37da58f97c83d
> lens-server/src/test/java/org/apache/lens/server/rewrite/TestRewriting.java 0e640ec66ba45716f4819cfd3d944a6155305078
> lens-server/src/test/java/org/apache/lens/server/session/TestSessionClassLoaders.java 5feab715b9a283e0951570a407b464b3bd7007a6
> lens-server/src/test/java/org/apache/lens/server/session/TestSessionExpiry.java dbdaaecb58e2c8dcad7b8ce0157f9f74cbb38484
> lens-server/src/test/resources/hive-site.xml 441f2741c7c55bace15b6c723caaf8eb76724666
> lens-server/src/test/resources/lens-site.xml b5b3220be8eb9fc6c6ed488e274e198e91991f6a
> lens-storage-db/pom.xml ed583413d51258fea6c555c759905f82be77b7d9
> lens-storage-db/src/test/resources/hive-site.xml 2cb1d8d47fc7161ea2667c34eb0c1bf0bd10945e
> pom.xml 06cae12bbf64495bf1f17eca83fc7877b9441107
> tools/conf/server/lens-site.xml 0803da168c10f93ca915e47356151154148ae0c2
>
> Diff: https://reviews.apache.org/r/41794/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>