You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Rajat Khandelwal <ra...@gmail.com> on 2016/09/20 18:26:12 UTC
Review Request 52088: LENS-743: Query failure retries for transient
errors
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs
-----
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8f1983ee10e9e4a429dc1c8ceb5d4a65bab975c1
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/DefaultQueryRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/QueryRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 997b5a80ab1f9516ad6aaeb431f08a61f386180a
lens-server/src/main/java/org/apache/lens/server/query/MoreRetriesFirstComparator.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dafe93dde2ec8a040b855261f00723be6cd7b26e
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On Sept. 22, 2016, 11:53 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java, line 22
> > <https://reviews.apache.org/r/52088/diff/1/?file=1505804#file1505804line22>
> >
> > Will SingleRetryHandler have delay for the next try?
>
> Rajat Khandelwal wrote:
> Will add more policies. This is one of the simplest of policies which has only one retry, without any delay.
Have generalized the policy to call it ImmediateRetryPolicy. The constructor takes an integer argument which is the number of retries.
> On Sept. 22, 2016, 11:53 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java, lines 210-216
> > <https://reviews.apache.org/r/52088/diff/1/?file=1505811#file1505811line210>
> >
> > All these should be defined as transient? Or there is recovery upon restarts?
>
> Rajat Khandelwal wrote:
> The failed attempts will have recovery upon restart
>
> For retry policies, they need not be stored, but it might result in a query being retried more than the normal amount, in case the query failed and is going through a retry and fails again on restart. Then the retry cycle will restart, causing the retries happened before restart to become redundant. Which should be fine assuming retries will not be that frequent.
Upon second thuoght, I've kept all these non-transient. These will be persisted and recovered upon restart, which will maintain consistencey of retries across restart. Have added a test case for the same.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review149955
-----------------------------------------------------------
On Sept. 29, 2016, 2:39 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Sept. 29, 2016, 2:39 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 3bf5e8f7dc9d032e46d06c3245048096acb4572e
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 168b3cc8a2e32e4fddb66f480af38e77adef62c2
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 0bb106ed77306094710c126fc652d5c56a908d85
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java f755508529f6745caadd8eee0f63e81e1f14ed75
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On Sept. 22, 2016, 11:53 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java, lines 210-216
> > <https://reviews.apache.org/r/52088/diff/1/?file=1505811#file1505811line210>
> >
> > All these should be defined as transient? Or there is recovery upon restarts?
The failed attempts will have recovery upon restart
For retry policies, they need not be stored, but it might result in a query being retried more than the normal amount, in case the query failed and is going through a retry and fails again on restart. Then the retry cycle will restart, causing the retries happened before restart to become redundant. Which should be fine assuming retries will not be that frequent.
> On Sept. 22, 2016, 11:53 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java, line 35
> > <https://reviews.apache.org/r/52088/diff/1/?file=1505809#file1505809line35>
> >
> > Should we have errorcode as well here ?
Where will the error code be extracted from? Right now, the failed attempt is extracted from driver status, which only has error message.
> On Sept. 22, 2016, 11:53 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java, line 22
> > <https://reviews.apache.org/r/52088/diff/1/?file=1505804#file1505804line22>
> >
> > Will SingleRetryHandler have delay for the next try?
Will add more policies. This is one of the simplest of policies which has only one retry, without any delay.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review149955
-----------------------------------------------------------
On Sept. 20, 2016, 11:56 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Sept. 20, 2016, 11:56 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8f1983ee10e9e4a429dc1c8ceb5d4a65bab975c1
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/NoRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/DefaultQueryRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/QueryRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 997b5a80ab1f9516ad6aaeb431f08a61f386180a
> lens-server/src/main/java/org/apache/lens/server/query/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dafe93dde2ec8a040b855261f00723be6cd7b26e
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review149955
-----------------------------------------------------------
lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java (line 22)
<https://reviews.apache.org/r/52088/#comment217744>
Will SingleRetryHandler have delay for the next try?
lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java (line 35)
<https://reviews.apache.org/r/52088/#comment217745>
Should we have errorcode as well here ?
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java (lines 210 - 216)
<https://reviews.apache.org/r/52088/#comment217746>
All these should be defined as transient? Or there is recovery upon restarts?
- Amareshwari Sriramadasu
On Sept. 20, 2016, 6:26 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Sept. 20, 2016, 6:26 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8f1983ee10e9e4a429dc1c8ceb5d4a65bab975c1
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/NoRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/DefaultQueryRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/QueryRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 997b5a80ab1f9516ad6aaeb431f08a61f386180a
> lens-server/src/main/java/org/apache/lens/server/query/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dafe93dde2ec8a040b855261f00723be6cd7b26e
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review151768
-----------------------------------------------------------
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java (line 80)
<https://reviews.apache.org/r/52088/#comment220270>
Is this required? If not, lets remove it.
Also, why does removal of a driver set selectedDriver to null?
- Amareshwari Sriramadasu
On Oct. 6, 2016, 12:29 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Oct. 6, 2016, 12:29 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryConstraints.java e649a753efde2b3c293394400016ea49e580a6f9
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
> src/site/resources/images/querystate.png 47062bfcbf7d01210c184281a5c4fe9dc8aa069c
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> File Attachments
> ----------------
>
> Screen Shot 2016-10-05 at 1.38.44 pm.png
> https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On Oct. 14, 2016, 3:31 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java, line 916
> > <https://reviews.apache.org/r/52088/diff/18/?file=1534062#file1534062line916>
> >
> > On what basis server retry policy is decided? Is it one at server level for all queries ?
`Server retry policy "decider"` is one at server level for all queries. Upon query failure and driver refusing to retry, this decider is consulted to give a `Server Retry Policy`. At that point, it's purely a function of the error message.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review152645
-----------------------------------------------------------
On Oct. 13, 2016, 11:57 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Oct. 13, 2016, 11:57 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e41077cffd1fed891e12dbefe7650abed8d73bb7
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2ad7f76c9c25469a2e39122ed96d1afcb837a6a0
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 365a619619dcde3ef847c2955b6e818515f870a2
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java fc24fc6364f8e9770ddfe4029ec6bede46df64f7
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d0662f4ccfb71a2f56d3cb841c3c399a6cdf713c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java a540c3cb5060a2fdcad7f114ad6595737c754b8e
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java cb5961f10b60ef3b17b5f2f803abde569b6496d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 440c30bd2a1bce597932ccebcca2bbd9416f0a09
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> File Attachments
> ----------------
>
> Screen Shot 2016-10-05 at 1.38.44 pm.png
> https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review152645
-----------------------------------------------------------
Fix it, then Ship it!
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java (lines 212 - 215)
<https://reviews.apache.org/r/52088/#comment221753>
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 911)
<https://reviews.apache.org/r/52088/#comment221754>
On what basis server retry policy is decided? Is it one at server level for all queries ?
- Amareshwari Sriramadasu
On Oct. 13, 2016, 6:27 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Oct. 13, 2016, 6:27 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e41077cffd1fed891e12dbefe7650abed8d73bb7
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2ad7f76c9c25469a2e39122ed96d1afcb837a6a0
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 365a619619dcde3ef847c2955b6e818515f870a2
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java fc24fc6364f8e9770ddfe4029ec6bede46df64f7
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d0662f4ccfb71a2f56d3cb841c3c399a6cdf713c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java a540c3cb5060a2fdcad7f114ad6595737c754b8e
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java cb5961f10b60ef3b17b5f2f803abde569b6496d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 440c30bd2a1bce597932ccebcca2bbd9416f0a09
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> File Attachments
> ----------------
>
> Screen Shot 2016-10-05 at 1.38.44 pm.png
> https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 13, 2016, 11:57 a.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e41077cffd1fed891e12dbefe7650abed8d73bb7
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2ad7f76c9c25469a2e39122ed96d1afcb837a6a0
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 365a619619dcde3ef847c2955b6e818515f870a2
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java fc24fc6364f8e9770ddfe4029ec6bede46df64f7
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d0662f4ccfb71a2f56d3cb841c3c399a6cdf713c
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java a540c3cb5060a2fdcad7f114ad6595737c754b8e
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java cb5961f10b60ef3b17b5f2f803abde569b6496d0
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 440c30bd2a1bce597932ccebcca2bbd9416f0a09
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 12, 2016, 6:31 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e41077cffd1fed891e12dbefe7650abed8d73bb7
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2ad7f76c9c25469a2e39122ed96d1afcb837a6a0
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 365a619619dcde3ef847c2955b6e818515f870a2
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java fc24fc6364f8e9770ddfe4029ec6bede46df64f7
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d0662f4ccfb71a2f56d3cb841c3c399a6cdf713c
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java a540c3cb5060a2fdcad7f114ad6595737c754b8e
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java cb5961f10b60ef3b17b5f2f803abde569b6496d0
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 440c30bd2a1bce597932ccebcca2bbd9416f0a09
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 12, 2016, 5:57 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e41077cffd1fed891e12dbefe7650abed8d73bb7
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2ad7f76c9c25469a2e39122ed96d1afcb837a6a0
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 365a619619dcde3ef847c2955b6e818515f870a2
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java fc24fc6364f8e9770ddfe4029ec6bede46df64f7
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d0662f4ccfb71a2f56d3cb841c3c399a6cdf713c
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java cb5961f10b60ef3b17b5f2f803abde569b6496d0
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 440c30bd2a1bce597932ccebcca2bbd9416f0a09
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On Oct. 12, 2016, 4:51 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java, line 55
> > <https://reviews.apache.org/r/52088/diff/6-15/?file=1514896#file1514896line55>
> >
> > test disabled?
Commented to run the test in isolation. Forgot to uncomment. Will do.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review152294
-----------------------------------------------------------
On Oct. 10, 2016, 12:14 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Oct. 10, 2016, 12:14 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> File Attachments
> ----------------
>
> Screen Shot 2016-10-05 at 1.38.44 pm.png
> https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review152294
-----------------------------------------------------------
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java (line 55)
<https://reviews.apache.org/r/52088/#comment221269>
test disabled?
- Amareshwari Sriramadasu
On Oct. 10, 2016, 6:44 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Oct. 10, 2016, 6:44 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> File Attachments
> ----------------
>
> Screen Shot 2016-10-05 at 1.38.44 pm.png
> https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 10, 2016, 12:14 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 10, 2016, 11:14 a.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryConstraints.java e649a753efde2b3c293394400016ea49e580a6f9
lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 9, 2016, 12:10 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryConstraints.java e649a753efde2b3c293394400016ea49e580a6f9
lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 9, 2016, 11:42 a.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryConstraints.java e649a753efde2b3c293394400016ea49e580a6f9
lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 7, 2016, 11:41 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryConstraints.java e649a753efde2b3c293394400016ea49e580a6f9
lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
src/site/resources/images/querystate.png 47062bfcbf7d01210c184281a5c4fe9dc8aa069c
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 7, 2016, 4:51 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryConstraints.java e649a753efde2b3c293394400016ea49e580a6f9
lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
src/site/resources/images/querystate.png 47062bfcbf7d01210c184281a5c4fe9dc8aa069c
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 6, 2016, 5:59 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryConstraints.java e649a753efde2b3c293394400016ea49e580a6f9
lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
src/site/resources/images/querystate.png 47062bfcbf7d01210c184281a5c4fe9dc8aa069c
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 5, 2016, 1:40 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
src/site/resources/images/querystate.png 47062bfcbf7d01210c184281a5c4fe9dc8aa069c
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 5, 2016, 1:39 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
File Attachments (updated)
----------------
Screen Shot 2016-10-05 at 1.38.44 pm.png
https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Oct. 4, 2016, 1:51 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On Sept. 30, 2016, 12:34 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java, line 41
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514828#file1514828line41>
> >
> > why do we need clear() functionality ?
The file FailureContext is renamed to `StatusUpdateFailureContext`. It already had `clear` method. The method isn't propagated to the interface, but is retained in the implementation. The other implementation of the interface doesn't have a `clear` method.
And to answer your question, it's used in `QueryContext#updateDriverStatus(BackOffRetryHandler)`.
> On Sept. 30, 2016, 12:34 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java, line 27
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514837#file1514837line27>
> >
> > when can priority of the query can change? and why?
Again, pre-existing code, only moved to another package. AFAIK, it's not being used anywhere.
> On Sept. 30, 2016, 12:34 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java, line 385
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514876#file1514876line385>
> >
> > loadDrivers() called separately?
Yes. The method `loadDriversAndSelectors` was basically a bunch of code and a call to `loadSelectors`. So extracted that bunch of code into a method `loadDrivers`. Both are called sequencially in `init`
> On Sept. 30, 2016, 12:34 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java, line 905
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514876#file1514876line905>
> >
> > Would need re-run for jdbc queries, as connection would have closed.
I don't think any additional code is required for that. query status will be updated to FAILING, and then retry policy will be consulted. The retry policy should be taking care of forcing a retry in case a failure is due to connection close.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150993
-----------------------------------------------------------
On Sept. 29, 2016, 2:39 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Sept. 29, 2016, 2:39 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 3bf5e8f7dc9d032e46d06c3245048096acb4572e
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 168b3cc8a2e32e4fddb66f480af38e77adef62c2
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 0bb106ed77306094710c126fc652d5c56a908d85
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java f755508529f6745caadd8eee0f63e81e1f14ed75
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150993
-----------------------------------------------------------
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java (line 41)
<https://reviews.apache.org/r/52088/#comment219092>
why do we need clear() functionality ?
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java (line 27)
<https://reviews.apache.org/r/52088/#comment219093>
when can priority of the query can change? and why?
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java (line 25)
<https://reviews.apache.org/r/52088/#comment219094>
Add javadoc for all the methods.
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
<https://reviews.apache.org/r/52088/#comment219095>
loadDrivers() called separately?
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java (line 900)
<https://reviews.apache.org/r/52088/#comment219096>
Would need re-run for jdbc queries, as connection would have closed.
- Amareshwari Sriramadasu
On Sept. 29, 2016, 9:09 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Sept. 29, 2016, 9:09 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 3bf5e8f7dc9d032e46d06c3245048096acb4572e
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 168b3cc8a2e32e4fddb66f480af38e77adef62c2
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 0bb106ed77306094710c126fc652d5c56a908d85
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java f755508529f6745caadd8eee0f63e81e1f14ed75
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > checkstyle/src/main/resources/checkstyle.xml, line 65
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514787#file1514787line65>
> >
> > hmm, what file is extending this? Need to fix by doing some refactoring if required.
Query execution service impl is now 3558 lines. Will see if any refactoring can be done.
> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java, lines 303-304
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514794#file1514794line303>
> >
> > Adding resources specific to driver is removed?
This is being done in super class's method.
> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, line 365
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514796#file1514796line365>
> >
> > hiveConf would be required here.
Adding overlay in the next line. `super.configure` is supposed to add driver specific configurations and this can be asked by `getConf()`
> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, line 368
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514796#file1514796line368>
> >
> > hivedriver-site.xml is not added in old code.
Refer to the above comment, super class making sure `getConf()` returns driver specific configuration. And this line is to make sure the overlay takes effect.
> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java, lines 76-77
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514806#file1514806line76>
> >
> > This is enforcing drivertype in the file names. This will be a problem with -default.xml files, if people create driver type to be different than whats expected to be.
> >
> > Instead you can try using driver.getClass().getName().toLowercase() ?
I'm fine with either, but unable to see the merit in using one above the other.
> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java, line 82
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514810#file1514810line82>
> >
> > removing from context map? We might lose some details like plan and etc. May be should be moved another data structure
Those things will anyway get lost if the query succeeds on another driver. If there are no other drivers, this is restored.
> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java, line 113
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514790#file1514790line113>
> >
> > can you update state machine on user doc with the change?
Would need to generate the png file again.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150991
-----------------------------------------------------------
On Sept. 29, 2016, 2:39 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Sept. 29, 2016, 2:39 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 3bf5e8f7dc9d032e46d06c3245048096acb4572e
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 168b3cc8a2e32e4fddb66f480af38e77adef62c2
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 0bb106ed77306094710c126fc652d5c56a908d85
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java f755508529f6745caadd8eee0f63e81e1f14ed75
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java, lines 76-77
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514806#file1514806line76>
> >
> > This is enforcing drivertype in the file names. This will be a problem with -default.xml files, if people create driver type to be different than whats expected to be.
> >
> > Instead you can try using driver.getClass().getName().toLowercase() ?
>
> Rajat Khandelwal wrote:
> I'm fine with either, but unable to see the merit in using one above the other.
I changed it to using class name. But there is a slight inconsistency.
Now, if drivers are configured like "mockHive:org.apache.lens.driver.hive.HiveDriver", then the `configure` method looks for `hivedriver-site.xml` instead of `mockhivedriver-site.xml`, which is fine.
But the code that I added for unification of properties does the other thing. e.g. for `retry.policy`, first candidate is `lens.driver.<driverType>.retry.policy`, followed by `lens.driver.retry.policy`, followed by `retry.policy`. Now, driverType is `mockHive`, so basically it's looking for `lens.driver.mockHive.retry.policy` inside a file named `hivedriver-site.xml`. While I think that it should instead look for `lens.driver.hive.retry.policy`, but now, the only way to extract `hive` is through getting classname.
I've changed the logic to now check 4 candidates instead of 3 as mentioned in the above paragraph. The new candidate now be `lens.driver.<driverClassType>.retry.policy`, and it'll be checked after `lens.driver.<driverType>.retry.policy`.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150991
-----------------------------------------------------------
On Oct. 7, 2016, 4:51 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Oct. 7, 2016, 4:51 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java e4984798751d012f0a9d01980df833c206155546
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 87d7cb0d5036bd0de4361cc146404a7fee478bac
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryConstraints.java e649a753efde2b3c293394400016ea49e580a6f9
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 3f71aef9ca180d834538604925c71d1d87be8e26
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
> src/site/resources/images/querystate.png 47062bfcbf7d01210c184281a5c4fe9dc8aa069c
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> File Attachments
> ----------------
>
> Screen Shot 2016-10-05 at 1.38.44 pm.png
> https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java, line 113
> > <https://reviews.apache.org/r/52088/diff/6/?file=1514790#file1514790line113>
> >
> > can you update state machine on user doc with the change?
>
> Rajat Khandelwal wrote:
> Would need to generate the png file again.
Will update seperately. Adding here and submitting patch results in the patch not getting applied because some binary diff is present. Created LENS-1356 for the same.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150991
-----------------------------------------------------------
On Oct. 12, 2016, 5:57 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Oct. 12, 2016, 5:57 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 0d8810f6b7dfd9fb0173a4be6966f1a561107066
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java daf01ac64616da72ee55c00f71c2c0aa33651ee4
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java e41077cffd1fed891e12dbefe7650abed8d73bb7
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2ad7f76c9c25469a2e39122ed96d1afcb837a6a0
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java d80fba9aee91130b69a2a511d7f157c01a261ac4
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 365a619619dcde3ef847c2955b6e818515f870a2
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java fc24fc6364f8e9770ddfe4029ec6bede46df64f7
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java e160f58bfb45cffb05b9f6be19db3bc7605dde57
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java d0662f4ccfb71a2f56d3cb841c3c399a6cdf713c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ChainedRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/NoRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 3c18ac7767a6418a961aff560d62537f2eb66c96
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java cb5961f10b60ef3b17b5f2f803abde569b6496d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/rewrite/RewriteUtil.java 18c2f2c58196efe8faf3abc8bf7b25a1a96f86fc
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryIndependenceFromSessionClose.java bf4577c287d9e2d5da510ca75a06939599ba8621
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 440c30bd2a1bce597932ccebcca2bbd9416f0a09
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> File Attachments
> ----------------
>
> Screen Shot 2016-10-05 at 1.38.44 pm.png
> https://reviews.apache.org/media/uploaded/files/2016/10/05/b7b4e33f-7e57-46c2-aa55-9d4f1055b992__Screen_Shot_2016-10-05_at_1.38.44_pm.png
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150991
-----------------------------------------------------------
checkstyle/src/main/resources/checkstyle.xml (line 65)
<https://reviews.apache.org/r/52088/#comment219085>
hmm, what file is extending this? Need to fix by doing some refactoring if required.
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java (line 112)
<https://reviews.apache.org/r/52088/#comment219086>
can you update state machine on user doc with the change?
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java
<https://reviews.apache.org/r/52088/#comment219087>
Adding resources specific to driver is removed?
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java
<https://reviews.apache.org/r/52088/#comment219088>
hiveConf would be required here.
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java (line 336)
<https://reviews.apache.org/r/52088/#comment219089>
hivedriver-site.xml is not added in old code.
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java (lines 76 - 77)
<https://reviews.apache.org/r/52088/#comment219090>
This is enforcing drivertype in the file names. This will be a problem with -default.xml files, if people create driver type to be different than whats expected to be.
Instead you can try using driver.getClass().getName().toLowercase() ?
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java (line 82)
<https://reviews.apache.org/r/52088/#comment219091>
removing from context map? We might lose some details like plan and etc. May be should be moved another data structure
- Amareshwari Sriramadasu
On Sept. 29, 2016, 9:09 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> -----------------------------------------------------------
>
> (Updated Sept. 29, 2016, 9:09 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
>
>
> Repository: lens
>
>
> Description
> -------
>
> There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
>
> Right now, any such failure results in marking query as failed.
>
>
> Diffs
> -----
>
> checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
> lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
> lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
> lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
> lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
> lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 3bf5e8f7dc9d032e46d06c3245048096acb4572e
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
> lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
> lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
> lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 168b3cc8a2e32e4fddb66f480af38e77adef62c2
> lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
> lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
> lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
> lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
> lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
> lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
> lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
> lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
> lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
> lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 0bb106ed77306094710c126fc652d5c56a908d85
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
> lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
> lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
> lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
> lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
> lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java f755508529f6745caadd8eee0f63e81e1f14ed75
> lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
> lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
> lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
> lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
> lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
> lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/52088/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Sept. 29, 2016, 2:39 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
checkstyle/src/main/resources/checkstyle.xml 17e55ea1071c06c73e8bf387cd69a4444cca83bc
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 04d9ad6dee942c705b0cdf071f49f1305696da65
lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml a681bca7fd6e46fe992901e04e0666625a8a3106
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 70f9c932efc090095273566eaa52d380a2f23842
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-es/src/test/resources/hive-site.xml e49c876289d5352203d2523ddb9134d745c197e5
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 3bf5e8f7dc9d032e46d06c3245048096acb4572e
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 168b3cc8a2e32e4fddb66f480af38e77adef62c2
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server-api/src/test/java/org/apache/lens/server/api/retry/TestExponentialBackOffRetryHandler.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 0bb106ed77306094710c126fc652d5c56a908d85
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java f755508529f6745caadd8eee0f63e81e1f14ed75
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Sept. 28, 2016, 7:55 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 3bf5e8f7dc9d032e46d06c3245048096acb4572e
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java eaf3fee953e1efbcf151fc844a9441b74b84cc3e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java 8980b619c9feb86be326bf799fecc35c1579c763
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java a473a47606df84e2b5b204478ad63e63e69b0c23
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 3837087db6e51c1606573511bfc02b6108d1ef3d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java e80da6d2a417e1274fb186153ec01615c2da565b
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 81c53be905d015bd14c0a05de5c7a43f8a6c5d84
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java af8c8ee062c4c43ae987cea7316a36d84215d389
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java bdffbc2f37bae7a59add1ee3823de87887483e5e
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 5f347ad0e751d9cfc6b647d57d0acc25fd8dee01
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java c4e60e3d72108bb556f6cd9a5887230b39d67a02
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java 52aa50d60cd2c1c03a76351830de2942ee84687d
lens-server-api/src/main/java/org/apache/lens/server/api/query/QuerySuccess.java 298fdbb297baff7328a7ec4308fa21189afdddc5
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueuePositionChange.java 062e14e8e105ca29196324ff1a051874dd628d5c
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusChange.java 949ec20e4c89a39987306c1bda2863a3d1b390ae
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryAccepted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryCancelled.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryClosed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEnded.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryEvent.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryExecuted.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryFailed.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryLaunched.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueued.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRejected.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueryRunning.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QuerySuccess.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/QueuePositionChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/events/StatusChange.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 168b3cc8a2e32e4fddb66f480af38e77adef62c2
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java b88c717d49f3e3c866cb875be987c069d07948fe
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryEndHttpNotifier.java 5d2ddbe76c30269dc7e72b766e216a34bd21591e
lens-server/src/main/java/org/apache/lens/server/query/QueryEndNotifier.java 91fddc9e1738b41cb7589b256a375e20655c0d67
lens-server/src/main/java/org/apache/lens/server/query/QueryEventHttpNotifier.java f264603c03f3444d2d38e17be9f6b1fcd708d0bf
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 0bb106ed77306094710c126fc652d5c56a908d85
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionStatisticsGenerator.java 55cabe2e79d8e898a654c880e4f35621e144886f
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/ResultFormatter.java 41cf33bc311d3375b3eaab26d9b85bcc112a4b42
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerQueryEventListener.java 4192134af99258bbfa79f687b092aee48482e7f5
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java c683a2cfa8fb3c02ad65d38cdc21c42b1143b64f
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java 526acccd6176f3369eb48da11f3b9c157528422d
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java f755508529f6745caadd8eee0f63e81e1f14ed75
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/scheduler/util/SchedulerTestUtils.java d50474a2efac68661882e87e01f0c0f80c2b6ab8
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Sept. 28, 2016, 7:47 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java PRE-CREATION
lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 7d079a9ed99e3d4940d5f26370547683a6bba465
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java fceabeae9217676774a6978212cbc9ee9ca120f7
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 3bf5e8f7dc9d032e46d06c3245048096acb4572e
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java 3c34eb191966dff3ac3032eb79282336b5c6789b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueuedForRetry.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/OperationRetryHandlerFactory.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 168b3cc8a2e32e4fddb66f480af38e77adef62c2
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 0bb106ed77306094710c126fc652d5c56a908d85
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java f755508529f6745caadd8eee0f63e81e1f14ed75
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/MockRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Sept. 27, 2016, 5:29 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
contrib/clients/python/pom.xml a849038ba4c6b52fc898e3a7ae41c604e9452de1
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java 82b56470e2de09f7c41d1226ea59cffdd8d26a26
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 6e9086f9e8dc302ad2171d7f5636a24cb79dcb50
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java 70a34b0c0517ed3b7af46120906fb77ae57870ac
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java 88fbe43318315394873cabb9f5dd39f09f6c47f4
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java e0f1376d48457d53c5c9bafa673304df4b54cc50
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java 12d9562566e71477db3cf00e1e0ed84d43eeb6cc
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java 5f407af815c0e43c51d185a30cb0b639e7696af7
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 168b3cc8a2e32e4fddb66f480af38e77adef62c2
lens-server-api/src/test/java/org/apache/lens/server/api/query/TestQueryContext.java a530e9d65e2c84e674b9260aaffbb68a0cbf2f94
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java 122409b72cc789d11d39b6bc7c908f0d08a6e66a
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 0bb106ed77306094710c126fc652d5c56a908d85
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java 0a8d4c357b61b42d4da7057df13e78e82669ad71
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java f755508529f6745caadd8eee0f63e81e1f14ed75
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java ab030cc83e459cc46344b5722b39d8023ce5a3c4
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java d9727800c4397c7e00b396fead589965503ac52e
lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java 460190a7ef80d8ea591206d7793fe084bf799712
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/triple_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
Thanks,
Rajat Khandelwal
Re: Review Request 52088: LENS-743: Query failure retries for
transient errors
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
-----------------------------------------------------------
(Updated Sept. 26, 2016, 8:29 p.m.)
Review request for lens.
Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743
Repository: lens
Description
-------
There have to be retries for query failures for transient errors like network errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). Retries should be available for each phase - submission, execution, updating status, fetching results and formatting.
Right now, any such failure results in marking query as failed.
Diffs (updated)
-----
contrib/clients/python/pom.xml a849038ba4c6b52fc898e3a7ae41c604e9452de1
lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java daedbf9ea387cbde3ef39363c2beef3f96725223
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 8cf617b7181b9a305ea2c68f68c610f823ee75d7
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java 17bfba02244bca7141268ed12e4a41400e2bec82
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c
lens-server-api/src/main/java/org/apache/lens/server/api/common/NoRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java 03079e2dc353fa028f3c5d5a03a8ae2dbf728004
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java 033f6777b4be7a1b6d6cdc350e42a94ade1acbca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java e472de0a452ffbbcb217503c260aa26fe7f6ed99
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java 5ff59bd0da713651a7cc5261dbd18cd88e91fc76
lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java b58fcf964cedb8dc302099ae9c806b06acafa2d3
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java b584c6a6d414a6f0dac509e67c8944dbd2373325
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/DefaultQueryRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/QueryRetryPolicyDecider.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 8261d8aa19eb26e0336846d32c5bcd2162ddcb14
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 168b3cc8a2e32e4fddb66f480af38e77adef62c2
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java PRE-CREATION
lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 0aada406963f7b421e8b9726a72abcd85c63ddd4
lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 2702581fc88a560dbb420ec3a7de1cf442542bd7
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 0bb106ed77306094710c126fc652d5c56a908d85
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java 2c6d904a5fd35d39b5339275013386ae7852bb73
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java 48291b94d35dfcf3dde6b3667b2aae28d81f9366
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/QueryContextComparatorTest.java 46adb7b9b4e53c6f8182d13c33ebf76647c37a27
lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java e620dc561774b396b6a498fb5b2e0f4001aaa46f
lens-server/src/test/java/org/apache/lens/server/query/TestQueryNotifictaionResource.java 3a3f2f150505e43a9f8c3f9f0ec9d7b431f1c0d0
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java f755508529f6745caadd8eee0f63e81e1f14ed75
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java 8b72c81554f06cf2115364fa66062572cd5aa957
lens-server/src/test/java/org/apache/lens/server/query/retry/MockDriverForRetries.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java PRE-CREATION
lens-server/src/test/java/org/apache/lens/server/query/retry/TestServerRetryPolicyDecider.java PRE-CREATION
lens-server/src/test/resources/drivers/mock/single_failure/failing-query-driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/double_failure/driver-site.xml PRE-CREATION
lens-server/src/test/resources/drivers/retry/single_failure/driver-site.xml PRE-CREATION
Diff: https://reviews.apache.org/r/52088/diff/
Testing
-------
Thanks,
Rajat Khandelwal