You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Amareshwari Sriramadasu <am...@apache.org> on 2015/08/03 13:48:45 UTC
Re: Review Request 36938: LENS-684: Assertion on phase 1 rewritten
query will fail intermittently
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36938/#review93891
-----------------------------------------------------------
lens-server/src/test/resources/lens-site.xml (line 125)
<https://reviews.apache.org/r/36938/#comment148311>
Are we sure doing this will not cause any intermittent test failures?
We should make this to be first class requirement for all code to handle all the cases.
What about query statistics, will that work with finished queries set to zero?
I would say instead of making tests change the value of finished queries to 1 for some tests, we should make all code to work fine even if finished queries is set to zero.
- Amareshwari Sriramadasu
On July 30, 2015, 1:57 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36938/
> -----------------------------------------------------------
>
> (Updated July 30, 2015, 1:57 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-684
> https://issues.apache.org/jira/browse/LENS-684
>
>
> Repository: lens
>
>
> Description
> -------
>
> TestQueryService#testExecuteAsync:
>
> Below asserts will fail, if the query context is removed by QueryPurger thread before the test running test case could retrieve it from allQueries collection using queryService. Provided lens.server.max.finished.queries=0 in lens-site.xml
>
> {code}
>
> QueryContext ctx = queryService.getQueryContext(lensQuery.getQueryHandle());
> assertNotNull(ctx.getPhase1RewrittenQuery());
> assertEquals(ctx.getPhase1RewrittenQuery(), ctx.getUserQuery());
>
> {code}
>
> These were added in LENS-24 Commit: https://github.com/apache/incubator-lens/commit/689c9a99b112b21e9ecba9a8461cf9672b88ba2f
>
>
> Diffs
> -----
>
> lens-cli/src/test/java/org/apache/lens/cli/TestLensQueryCommands.java 319077ac15eab95d85389ac3286a146b8cd59b42
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java ae6d6822f6b116f07dd9a204bdf5a7c8ee7671d5
> lens-server/src/test/java/org/apache/lens/server/LensAllApplicationJerseyTest.java a067924fc555160981b2d9ca32145ace449f5854
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 4b9962a9cda22002cdc4826ebce3db4c77216223
> lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java c17b28bd29112f9af054dcc478d069f71ba396b2
> lens-server/src/test/resources/lens-site.xml bd9382820d59ecdc075d1f9b7258e31f899ca6d5
>
> Diff: https://reviews.apache.org/r/36938/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.970s]
> [INFO] Lens .............................................. SUCCESS [2.769s]
> [INFO] Lens API .......................................... SUCCESS [21.835s]
> [INFO] Lens API for server and extensions ................ SUCCESS [22.005s]
> [INFO] Lens Cube ......................................... SUCCESS [5:59.384s]
> [INFO] Lens DB storage ................................... SUCCESS [20.477s]
> [INFO] Lens Query Library ................................ SUCCESS [13.952s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:46.394s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [35.238s]
> [INFO] Lens Server ....................................... SUCCESS [5:22.280s]
> [INFO] Lens client ....................................... SUCCESS [34.721s]
> [INFO] Lens CLI .......................................... SUCCESS [2:35.788s]
> [INFO] Lens Examples ..................................... SUCCESS [8.684s]
> [INFO] Lens Distribution ................................. SUCCESS [8.485s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:18.588s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.698s]
> [INFO] Lens Regression ................................... SUCCESS [11.397s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 21:06.507s
> [INFO] Finished at: Thu Jul 30 13:53:45 UTC 2015
> [INFO] Final Memory: 175M/1194M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 36938: LENS-684: Assertion on phase 1 rewritten
query will fail intermittently
Posted by Amareshwari Sriramadasu <am...@apache.org>.
> On Aug. 3, 2015, 11:48 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/test/resources/lens-site.xml, line 125
> > <https://reviews.apache.org/r/36938/diff/1/?file=1025088#file1025088line125>
> >
> > Are we sure doing this will not cause any intermittent test failures?
> >
> > We should make this to be first class requirement for all code to handle all the cases.
> >
> > What about query statistics, will that work with finished queries set to zero?
> >
> > I would say instead of making tests change the value of finished queries to 1 for some tests, we should make all code to work fine even if finished queries is set to zero.
>
> Rajat Khandelwal wrote:
> testing in memory results will require max finished queries > 0. And I think most of the special handling in tests is because of that.
Why do we have to make finished queries to zero, if that functionality itself is not supported; and special handling for some tests ?
- Amareshwari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36938/#review93891
-----------------------------------------------------------
On July 30, 2015, 1:57 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36938/
> -----------------------------------------------------------
>
> (Updated July 30, 2015, 1:57 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-684
> https://issues.apache.org/jira/browse/LENS-684
>
>
> Repository: lens
>
>
> Description
> -------
>
> TestQueryService#testExecuteAsync:
>
> Below asserts will fail, if the query context is removed by QueryPurger thread before the test running test case could retrieve it from allQueries collection using queryService. Provided lens.server.max.finished.queries=0 in lens-site.xml
>
> {code}
>
> QueryContext ctx = queryService.getQueryContext(lensQuery.getQueryHandle());
> assertNotNull(ctx.getPhase1RewrittenQuery());
> assertEquals(ctx.getPhase1RewrittenQuery(), ctx.getUserQuery());
>
> {code}
>
> These were added in LENS-24 Commit: https://github.com/apache/incubator-lens/commit/689c9a99b112b21e9ecba9a8461cf9672b88ba2f
>
>
> Diffs
> -----
>
> lens-cli/src/test/java/org/apache/lens/cli/TestLensQueryCommands.java 319077ac15eab95d85389ac3286a146b8cd59b42
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java ae6d6822f6b116f07dd9a204bdf5a7c8ee7671d5
> lens-server/src/test/java/org/apache/lens/server/LensAllApplicationJerseyTest.java a067924fc555160981b2d9ca32145ace449f5854
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 4b9962a9cda22002cdc4826ebce3db4c77216223
> lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java c17b28bd29112f9af054dcc478d069f71ba396b2
> lens-server/src/test/resources/lens-site.xml bd9382820d59ecdc075d1f9b7258e31f899ca6d5
>
> Diff: https://reviews.apache.org/r/36938/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.970s]
> [INFO] Lens .............................................. SUCCESS [2.769s]
> [INFO] Lens API .......................................... SUCCESS [21.835s]
> [INFO] Lens API for server and extensions ................ SUCCESS [22.005s]
> [INFO] Lens Cube ......................................... SUCCESS [5:59.384s]
> [INFO] Lens DB storage ................................... SUCCESS [20.477s]
> [INFO] Lens Query Library ................................ SUCCESS [13.952s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:46.394s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [35.238s]
> [INFO] Lens Server ....................................... SUCCESS [5:22.280s]
> [INFO] Lens client ....................................... SUCCESS [34.721s]
> [INFO] Lens CLI .......................................... SUCCESS [2:35.788s]
> [INFO] Lens Examples ..................................... SUCCESS [8.684s]
> [INFO] Lens Distribution ................................. SUCCESS [8.485s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:18.588s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.698s]
> [INFO] Lens Regression ................................... SUCCESS [11.397s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 21:06.507s
> [INFO] Finished at: Thu Jul 30 13:53:45 UTC 2015
> [INFO] Final Memory: 175M/1194M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 36938: LENS-684: Assertion on phase 1 rewritten
query will fail intermittently
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On Aug. 3, 2015, 5:18 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/test/resources/lens-site.xml, line 125
> > <https://reviews.apache.org/r/36938/diff/1/?file=1025088#file1025088line125>
> >
> > Are we sure doing this will not cause any intermittent test failures?
> >
> > We should make this to be first class requirement for all code to handle all the cases.
> >
> > What about query statistics, will that work with finished queries set to zero?
> >
> > I would say instead of making tests change the value of finished queries to 1 for some tests, we should make all code to work fine even if finished queries is set to zero.
testing in memory results will require max finished queries > 0. And I think most of the special handling in tests is because of that.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36938/#review93891
-----------------------------------------------------------
On July 30, 2015, 7:27 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36938/
> -----------------------------------------------------------
>
> (Updated July 30, 2015, 7:27 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-684
> https://issues.apache.org/jira/browse/LENS-684
>
>
> Repository: lens
>
>
> Description
> -------
>
> TestQueryService#testExecuteAsync:
>
> Below asserts will fail, if the query context is removed by QueryPurger thread before the test running test case could retrieve it from allQueries collection using queryService. Provided lens.server.max.finished.queries=0 in lens-site.xml
>
> {code}
>
> QueryContext ctx = queryService.getQueryContext(lensQuery.getQueryHandle());
> assertNotNull(ctx.getPhase1RewrittenQuery());
> assertEquals(ctx.getPhase1RewrittenQuery(), ctx.getUserQuery());
>
> {code}
>
> These were added in LENS-24 Commit: https://github.com/apache/incubator-lens/commit/689c9a99b112b21e9ecba9a8461cf9672b88ba2f
>
>
> Diffs
> -----
>
> lens-cli/src/test/java/org/apache/lens/cli/TestLensQueryCommands.java 319077ac15eab95d85389ac3286a146b8cd59b42
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java ae6d6822f6b116f07dd9a204bdf5a7c8ee7671d5
> lens-server/src/test/java/org/apache/lens/server/LensAllApplicationJerseyTest.java a067924fc555160981b2d9ca32145ace449f5854
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 4b9962a9cda22002cdc4826ebce3db4c77216223
> lens-server/src/test/java/org/apache/lens/server/query/TestResultFormatting.java c17b28bd29112f9af054dcc478d069f71ba396b2
> lens-server/src/test/resources/lens-site.xml bd9382820d59ecdc075d1f9b7258e31f899ca6d5
>
> Diff: https://reviews.apache.org/r/36938/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [1.970s]
> [INFO] Lens .............................................. SUCCESS [2.769s]
> [INFO] Lens API .......................................... SUCCESS [21.835s]
> [INFO] Lens API for server and extensions ................ SUCCESS [22.005s]
> [INFO] Lens Cube ......................................... SUCCESS [5:59.384s]
> [INFO] Lens DB storage ................................... SUCCESS [20.477s]
> [INFO] Lens Query Library ................................ SUCCESS [13.952s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:46.394s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [35.238s]
> [INFO] Lens Server ....................................... SUCCESS [5:22.280s]
> [INFO] Lens client ....................................... SUCCESS [34.721s]
> [INFO] Lens CLI .......................................... SUCCESS [2:35.788s]
> [INFO] Lens Examples ..................................... SUCCESS [8.684s]
> [INFO] Lens Distribution ................................. SUCCESS [8.485s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:18.588s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.698s]
> [INFO] Lens Regression ................................... SUCCESS [11.397s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 21:06.507s
> [INFO] Finished at: Thu Jul 30 13:53:45 UTC 2015
> [INFO] Final Memory: 175M/1194M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>