You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Rajat Khandelwal <ra...@gmail.com> on 2015/06/01 08:12:10 UTC
Review Request 34876: LENS-555: Queue names for Hive driver need to be
dynamic
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/
-----------------------------------------------------------
Review request for lens.
Bugs: LENS-555
https://issues.apache.org/jira/browse/LENS-555
Repository: lens
Description
-------
Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
Diffs
-----
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dbc52e9fcda9bb897264c1e357a679454ef1983f
lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
Diff: https://reviews.apache.org/r/34876/diff/
Testing
-------
Thanks,
Rajat Khandelwal
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Amareshwari Sriramadasu <am...@apache.org>.
> On June 10, 2015, 9:01 a.m., Amareshwari Sriramadasu wrote:
> > Changes look fine.Can you add unit tests where drivers are calling preSubmit?
>
> Rajat Khandelwal wrote:
> How do I add test cases? changes done in presubmit are gone once the query finishes and is purged. The assert calls needs to happen after driver launch and before purge. How do I synchronize that?
We can add a test loader which sets a test flag on preSubmit and register it for the driver. And assert for the flag after execute call.
- Amareshwari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review87381
-----------------------------------------------------------
On June 10, 2015, 11:48 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 10, 2015, 11:48 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
> lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
> lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
> lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
> [INFO] Lens .............................................. SUCCESS [2.921s]
> [INFO] Lens API .......................................... SUCCESS [18.691s]
> [INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
> [INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
> [INFO] Lens DB storage ................................... SUCCESS [21.515s]
> [INFO] Lens Query Library ................................ SUCCESS [17.675s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
> [INFO] Lens Server ....................................... SUCCESS [5:25.651s]
> [INFO] Lens client ....................................... SUCCESS [36.611s]
> [INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
> [INFO] Lens Examples ..................................... SUCCESS [9.916s]
> [INFO] Lens Distribution ................................. SUCCESS [7.971s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
> [INFO] Lens Regression ................................... SUCCESS [8.378s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 19:44.006s
> [INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
> [INFO] Final Memory: 162M/1110M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On June 10, 2015, 2:31 p.m., Amareshwari Sriramadasu wrote:
> > Changes look fine.Can you add unit tests where drivers are calling preSubmit?
How do I add test cases? changes done in presubmit are gone once the query finishes and is purged. The assert calls needs to happen after driver launch and before purge. How do I synchronize that?
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review87381
-----------------------------------------------------------
On June 10, 2015, 5:18 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 10, 2015, 5:18 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
> lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
> lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
> lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
> [INFO] Lens .............................................. SUCCESS [2.921s]
> [INFO] Lens API .......................................... SUCCESS [18.691s]
> [INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
> [INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
> [INFO] Lens DB storage ................................... SUCCESS [21.515s]
> [INFO] Lens Query Library ................................ SUCCESS [17.675s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
> [INFO] Lens Server ....................................... SUCCESS [5:25.651s]
> [INFO] Lens client ....................................... SUCCESS [36.611s]
> [INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
> [INFO] Lens Examples ..................................... SUCCESS [9.916s]
> [INFO] Lens Distribution ................................. SUCCESS [7.971s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
> [INFO] Lens Regression ................................... SUCCESS [8.378s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 19:44.006s
> [INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
> [INFO] Final Memory: 162M/1110M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review87381
-----------------------------------------------------------
Changes look fine.Can you add unit tests where drivers are calling preSubmit?
- Amareshwari Sriramadasu
On June 10, 2015, 6:24 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 10, 2015, 6:24 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
> lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
> lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
> lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
> [INFO] Lens .............................................. SUCCESS [2.921s]
> [INFO] Lens API .......................................... SUCCESS [18.691s]
> [INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
> [INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
> [INFO] Lens DB storage ................................... SUCCESS [21.515s]
> [INFO] Lens Query Library ................................ SUCCESS [17.675s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
> [INFO] Lens Server ....................................... SUCCESS [5:25.651s]
> [INFO] Lens client ....................................... SUCCESS [36.611s]
> [INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
> [INFO] Lens Examples ..................................... SUCCESS [9.916s]
> [INFO] Lens Distribution ................................. SUCCESS [7.971s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
> [INFO] Lens Regression ................................... SUCCESS [8.378s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 19:44.006s
> [INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
> [INFO] Final Memory: 162M/1110M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review87677
-----------------------------------------------------------
Ship it!
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java
<https://reviews.apache.org/r/34876/#comment140096>
Should we change the log to debug ?
- Amareshwari Sriramadasu
On June 11, 2015, 12:18 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 11, 2015, 12:18 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java b21258a1386327e51c6a34e6e7abb69c94b9c965
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java b4086b858f2df8bfb6ea60ca216da8bb12a9cf5f
> lens-driver-jdbc/pom.xml a7994c5c6444ec6aa4b92b953784b16c2f8a3489
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2e3dcc7c5805733dd3784619c10e17e4690a4dca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
> lens-server-api/src/test/java/org/apache/lens/server/api/user/MockUserConfigLoader.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
> lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
> lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.279s]
> [INFO] Lens .............................................. SUCCESS [3.888s]
> [INFO] Lens API .......................................... SUCCESS [21.529s]
> [INFO] Lens API for server and extensions ................ SUCCESS [23.911s]
> [INFO] Lens Cube ......................................... SUCCESS [3:39.754s]
> [INFO] Lens DB storage ................................... SUCCESS [21.301s]
> [INFO] Lens Query Library ................................ SUCCESS [16.096s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:57.508s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [34.564s]
> [INFO] Lens Server ....................................... SUCCESS [5:21.248s]
> [INFO] Lens client ....................................... SUCCESS [35.348s]
> [INFO] Lens CLI .......................................... SUCCESS [4:51.950s]
> [INFO] Lens Examples ..................................... SUCCESS [13.121s]
> [INFO] Lens Distribution ................................. SUCCESS [10.219s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:33.169s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.729s]
> [INFO] Lens Regression ................................... SUCCESS [11.151s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 21:40.865s
> [INFO] Finished at: Thu Jun 11 12:12:59 UTC 2015
> [INFO] Final Memory: 168M/1187M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review87694
-----------------------------------------------------------
Ship it!
Ship It!
- Amareshwari Sriramadasu
On June 12, 2015, 7:17 a.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 12, 2015, 7:17 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java b21258a1386327e51c6a34e6e7abb69c94b9c965
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java b4086b858f2df8bfb6ea60ca216da8bb12a9cf5f
> lens-driver-jdbc/pom.xml a7994c5c6444ec6aa4b92b953784b16c2f8a3489
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2e3dcc7c5805733dd3784619c10e17e4690a4dca
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
> lens-server-api/src/test/java/org/apache/lens/server/api/user/MockUserConfigLoader.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
> lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
> lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.279s]
> [INFO] Lens .............................................. SUCCESS [3.888s]
> [INFO] Lens API .......................................... SUCCESS [21.529s]
> [INFO] Lens API for server and extensions ................ SUCCESS [23.911s]
> [INFO] Lens Cube ......................................... SUCCESS [3:39.754s]
> [INFO] Lens DB storage ................................... SUCCESS [21.301s]
> [INFO] Lens Query Library ................................ SUCCESS [16.096s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:57.508s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [34.564s]
> [INFO] Lens Server ....................................... SUCCESS [5:21.248s]
> [INFO] Lens client ....................................... SUCCESS [35.348s]
> [INFO] Lens CLI .......................................... SUCCESS [4:51.950s]
> [INFO] Lens Examples ..................................... SUCCESS [13.121s]
> [INFO] Lens Distribution ................................. SUCCESS [10.219s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:33.169s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [1.729s]
> [INFO] Lens Regression ................................... SUCCESS [11.151s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 21:40.865s
> [INFO] Finished at: Thu Jun 11 12:12:59 UTC 2015
> [INFO] Final Memory: 168M/1187M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/
-----------------------------------------------------------
(Updated June 12, 2015, 12:47 p.m.)
Review request for lens.
Bugs: LENS-555
https://issues.apache.org/jira/browse/LENS-555
Repository: lens
Description
-------
Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
Diffs (updated)
-----
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java b21258a1386327e51c6a34e6e7abb69c94b9c965
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java b4086b858f2df8bfb6ea60ca216da8bb12a9cf5f
lens-driver-jdbc/pom.xml a7994c5c6444ec6aa4b92b953784b16c2f8a3489
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2e3dcc7c5805733dd3784619c10e17e4690a4dca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
lens-server-api/src/test/java/org/apache/lens/server/api/user/MockUserConfigLoader.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
Diff: https://reviews.apache.org/r/34876/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.279s]
[INFO] Lens .............................................. SUCCESS [3.888s]
[INFO] Lens API .......................................... SUCCESS [21.529s]
[INFO] Lens API for server and extensions ................ SUCCESS [23.911s]
[INFO] Lens Cube ......................................... SUCCESS [3:39.754s]
[INFO] Lens DB storage ................................... SUCCESS [21.301s]
[INFO] Lens Query Library ................................ SUCCESS [16.096s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:57.508s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [34.564s]
[INFO] Lens Server ....................................... SUCCESS [5:21.248s]
[INFO] Lens client ....................................... SUCCESS [35.348s]
[INFO] Lens CLI .......................................... SUCCESS [4:51.950s]
[INFO] Lens Examples ..................................... SUCCESS [13.121s]
[INFO] Lens Distribution ................................. SUCCESS [10.219s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:33.169s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.729s]
[INFO] Lens Regression ................................... SUCCESS [11.151s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21:40.865s
[INFO] Finished at: Thu Jun 11 12:12:59 UTC 2015
[INFO] Final Memory: 168M/1187M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/
-----------------------------------------------------------
(Updated June 12, 2015, 12:24 p.m.)
Review request for lens.
Bugs: LENS-555
https://issues.apache.org/jira/browse/LENS-555
Repository: lens
Description
-------
Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
Diffs (updated)
-----
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java b21258a1386327e51c6a34e6e7abb69c94b9c965
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java b4086b858f2df8bfb6ea60ca216da8bb12a9cf5f
lens-driver-jdbc/pom.xml a7994c5c6444ec6aa4b92b953784b16c2f8a3489
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2e3dcc7c5805733dd3784619c10e17e4690a4dca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
lens-server-api/src/test/java/org/apache/lens/server/api/user/MockUserConfigLoader.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
Diff: https://reviews.apache.org/r/34876/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.279s]
[INFO] Lens .............................................. SUCCESS [3.888s]
[INFO] Lens API .......................................... SUCCESS [21.529s]
[INFO] Lens API for server and extensions ................ SUCCESS [23.911s]
[INFO] Lens Cube ......................................... SUCCESS [3:39.754s]
[INFO] Lens DB storage ................................... SUCCESS [21.301s]
[INFO] Lens Query Library ................................ SUCCESS [16.096s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:57.508s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [34.564s]
[INFO] Lens Server ....................................... SUCCESS [5:21.248s]
[INFO] Lens client ....................................... SUCCESS [35.348s]
[INFO] Lens CLI .......................................... SUCCESS [4:51.950s]
[INFO] Lens Examples ..................................... SUCCESS [13.121s]
[INFO] Lens Distribution ................................. SUCCESS [10.219s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:33.169s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.729s]
[INFO] Lens Regression ................................... SUCCESS [11.151s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21:40.865s
[INFO] Finished at: Thu Jun 11 12:12:59 UTC 2015
[INFO] Final Memory: 168M/1187M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/
-----------------------------------------------------------
(Updated June 11, 2015, 5:48 p.m.)
Review request for lens.
Bugs: LENS-555
https://issues.apache.org/jira/browse/LENS-555
Repository: lens
Description
-------
Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
Diffs (updated)
-----
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java b21258a1386327e51c6a34e6e7abb69c94b9c965
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java b4086b858f2df8bfb6ea60ca216da8bb12a9cf5f
lens-driver-jdbc/pom.xml a7994c5c6444ec6aa4b92b953784b16c2f8a3489
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2e3dcc7c5805733dd3784619c10e17e4690a4dca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
lens-server-api/src/test/java/org/apache/lens/server/api/user/MockUserConfigLoader.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
Diff: https://reviews.apache.org/r/34876/diff/
Testing (updated)
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [3.279s]
[INFO] Lens .............................................. SUCCESS [3.888s]
[INFO] Lens API .......................................... SUCCESS [21.529s]
[INFO] Lens API for server and extensions ................ SUCCESS [23.911s]
[INFO] Lens Cube ......................................... SUCCESS [3:39.754s]
[INFO] Lens DB storage ................................... SUCCESS [21.301s]
[INFO] Lens Query Library ................................ SUCCESS [16.096s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:57.508s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [34.564s]
[INFO] Lens Server ....................................... SUCCESS [5:21.248s]
[INFO] Lens client ....................................... SUCCESS [35.348s]
[INFO] Lens CLI .......................................... SUCCESS [4:51.950s]
[INFO] Lens Examples ..................................... SUCCESS [13.121s]
[INFO] Lens Distribution ................................. SUCCESS [10.219s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:33.169s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.729s]
[INFO] Lens Regression ................................... SUCCESS [11.151s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21:40.865s
[INFO] Finished at: Thu Jun 11 12:12:59 UTC 2015
[INFO] Final Memory: 168M/1187M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/
-----------------------------------------------------------
(Updated June 11, 2015, 4:31 p.m.)
Review request for lens.
Bugs: LENS-555
https://issues.apache.org/jira/browse/LENS-555
Repository: lens
Description
-------
Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
Diffs (updated)
-----
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java b21258a1386327e51c6a34e6e7abb69c94b9c965
lens-driver-jdbc/pom.xml a7994c5c6444ec6aa4b92b953784b16c2f8a3489
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 2e3dcc7c5805733dd3784619c10e17e4690a4dca
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
lens-server-api/src/test/java/org/apache/lens/server/api/user/MockUserConfigLoader.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
Diff: https://reviews.apache.org/r/34876/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
[INFO] Lens .............................................. SUCCESS [2.921s]
[INFO] Lens API .......................................... SUCCESS [18.691s]
[INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
[INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
[INFO] Lens DB storage ................................... SUCCESS [21.515s]
[INFO] Lens Query Library ................................ SUCCESS [17.675s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
[INFO] Lens Server ....................................... SUCCESS [5:25.651s]
[INFO] Lens client ....................................... SUCCESS [36.611s]
[INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
[INFO] Lens Examples ..................................... SUCCESS [9.916s]
[INFO] Lens Distribution ................................. SUCCESS [7.971s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
[INFO] Lens Regression ................................... SUCCESS [8.378s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 19:44.006s
[INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
[INFO] Final Memory: 162M/1110M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/
-----------------------------------------------------------
(Updated June 10, 2015, 5:18 p.m.)
Review request for lens.
Bugs: LENS-555
https://issues.apache.org/jira/browse/LENS-555
Repository: lens
Description
-------
Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
Diffs (updated)
-----
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
Diff: https://reviews.apache.org/r/34876/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
[INFO] Lens .............................................. SUCCESS [2.921s]
[INFO] Lens API .......................................... SUCCESS [18.691s]
[INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
[INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
[INFO] Lens DB storage ................................... SUCCESS [21.515s]
[INFO] Lens Query Library ................................ SUCCESS [17.675s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
[INFO] Lens Server ....................................... SUCCESS [5:25.651s]
[INFO] Lens client ....................................... SUCCESS [36.611s]
[INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
[INFO] Lens Examples ..................................... SUCCESS [9.916s]
[INFO] Lens Distribution ................................. SUCCESS [7.971s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
[INFO] Lens Regression ................................... SUCCESS [8.378s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 19:44.006s
[INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
[INFO] Final Memory: 162M/1110M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/
-----------------------------------------------------------
(Updated June 10, 2015, 11:54 a.m.)
Review request for lens.
Bugs: LENS-555
https://issues.apache.org/jira/browse/LENS-555
Repository: lens
Description
-------
Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
Diffs (updated)
-----
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 30c1a71a46ecba88c05bc8ed5020c7f79aedf68d
lens-server/src/main/java/org/apache/lens/server/LensServices.java c27717fc2846a947af825c350c09bb2ac13e005d
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java 7623743a816aea6b9d5c95e69943fa2f95ca6c74
lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
lens-server/src/test/java/org/apache/lens/server/user/FooBarConfigLoader.java 5b3344003efa201e6f56357ba8813603185b7ec5
lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java 4f399c27836303b487aef278a3b04bcbc97c7ff9
Diff: https://reviews.apache.org/r/34876/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
[INFO] Lens .............................................. SUCCESS [2.921s]
[INFO] Lens API .......................................... SUCCESS [18.691s]
[INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
[INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
[INFO] Lens DB storage ................................... SUCCESS [21.515s]
[INFO] Lens Query Library ................................ SUCCESS [17.675s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
[INFO] Lens Server ....................................... SUCCESS [5:25.651s]
[INFO] Lens client ....................................... SUCCESS [36.611s]
[INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
[INFO] Lens Examples ..................................... SUCCESS [9.916s]
[INFO] Lens Distribution ................................. SUCCESS [7.971s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
[INFO] Lens Regression ................................... SUCCESS [8.378s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 19:44.006s
[INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
[INFO] Final Memory: 162M/1110M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Amareshwari Sriramadasu <am...@apache.org>.
> On June 2, 2015, 5:08 a.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, line 753
> > <https://reviews.apache.org/r/34876/diff/1/?file=975496#file975496line753>
> >
> > doesn't need to called from tests? I was thinking above preSubmit call would NPE, if not called.
>
> Rajat Khandelwal wrote:
> Right now, Drivers only call presubmit if config loader is provided. But your point is correct. Considering that config loader will always be set by execution service, drivers need not check for null.
>
> Rajat Khandelwal wrote:
> I think let's keep the check of `if config loader set in driver, only then call pre submit`. Driver shouldn't fail if the hook wasn't registered.
yep, fine.
- Amareshwari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review86160
-----------------------------------------------------------
On June 1, 2015, 1:08 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 1, 2015, 1:08 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dbc52e9fcda9bb897264c1e357a679454ef1983f
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
> [INFO] Lens .............................................. SUCCESS [2.921s]
> [INFO] Lens API .......................................... SUCCESS [18.691s]
> [INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
> [INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
> [INFO] Lens DB storage ................................... SUCCESS [21.515s]
> [INFO] Lens Query Library ................................ SUCCESS [17.675s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
> [INFO] Lens Server ....................................... SUCCESS [5:25.651s]
> [INFO] Lens client ....................................... SUCCESS [36.611s]
> [INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
> [INFO] Lens Examples ..................................... SUCCESS [9.916s]
> [INFO] Lens Distribution ................................. SUCCESS [7.971s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
> [INFO] Lens Regression ................................... SUCCESS [8.378s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 19:44.006s
> [INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
> [INFO] Final Memory: 162M/1110M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Amareshwari Sriramadasu <am...@apache.org>.
> On June 2, 2015, 5:08 a.m., Amareshwari Sriramadasu wrote:
> > - Can you include unit tests?
> > - Did not see any changes wrt dynamic queue name in the loader? Are you planning include those changes in a follow up patch ?
>
> Rajat Khandelwal wrote:
> dynamic queue name is not generic enough to be included in lens. Lens as a platform should be satisfied with providing a hook for users to use. That hook can be used for any conf changes, including dynamic queues.
>
> Rajat Khandelwal wrote:
> I have two approaches in mind:
>
> 1. Don't change driver api, inside driver's executeAsync, driver should call preSubmit. Any new driver can choose not to call preSubmit in this case.
> 2. Change driver's api so that preSubmit is called from the execution service. This might break backward compatibility.
>
> I saw in the code that Approach 1 is taken for registering driver event handlers. That's why I too went ahead with approach 1. Would love other opinions.
Approach 1 looks fine.
- Amareshwari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review86160
-----------------------------------------------------------
On June 1, 2015, 1:08 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 1, 2015, 1:08 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dbc52e9fcda9bb897264c1e357a679454ef1983f
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
> [INFO] Lens .............................................. SUCCESS [2.921s]
> [INFO] Lens API .......................................... SUCCESS [18.691s]
> [INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
> [INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
> [INFO] Lens DB storage ................................... SUCCESS [21.515s]
> [INFO] Lens Query Library ................................ SUCCESS [17.675s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
> [INFO] Lens Server ....................................... SUCCESS [5:25.651s]
> [INFO] Lens client ....................................... SUCCESS [36.611s]
> [INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
> [INFO] Lens Examples ..................................... SUCCESS [9.916s]
> [INFO] Lens Distribution ................................. SUCCESS [7.971s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
> [INFO] Lens Regression ................................... SUCCESS [8.378s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 19:44.006s
> [INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
> [INFO] Final Memory: 162M/1110M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On June 2, 2015, 10:38 a.m., Amareshwari Sriramadasu wrote:
> > - Can you include unit tests?
> > - Did not see any changes wrt dynamic queue name in the loader? Are you planning include those changes in a follow up patch ?
>
> Rajat Khandelwal wrote:
> dynamic queue name is not generic enough to be included in lens. Lens as a platform should be satisfied with providing a hook for users to use. That hook can be used for any conf changes, including dynamic queues.
I have two approaches in mind:
1. Don't change driver api, inside driver's executeAsync, driver should call preSubmit. Any new driver can choose not to call preSubmit in this case.
2. Change driver's api so that preSubmit is called from the execution service. This might break backward compatibility.
I saw in the code that Approach 1 is taken for registering driver event handlers. That's why I too went ahead with approach 1. Would love other opinions.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review86160
-----------------------------------------------------------
On June 1, 2015, 6:38 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 1, 2015, 6:38 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dbc52e9fcda9bb897264c1e357a679454ef1983f
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
> [INFO] Lens .............................................. SUCCESS [2.921s]
> [INFO] Lens API .......................................... SUCCESS [18.691s]
> [INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
> [INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
> [INFO] Lens DB storage ................................... SUCCESS [21.515s]
> [INFO] Lens Query Library ................................ SUCCESS [17.675s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
> [INFO] Lens Server ....................................... SUCCESS [5:25.651s]
> [INFO] Lens client ....................................... SUCCESS [36.611s]
> [INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
> [INFO] Lens Examples ..................................... SUCCESS [9.916s]
> [INFO] Lens Distribution ................................. SUCCESS [7.971s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
> [INFO] Lens Regression ................................... SUCCESS [8.378s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 19:44.006s
> [INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
> [INFO] Final Memory: 162M/1110M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On June 2, 2015, 10:38 a.m., Amareshwari Sriramadasu wrote:
> > - Can you include unit tests?
> > - Did not see any changes wrt dynamic queue name in the loader? Are you planning include those changes in a follow up patch ?
dynamic queue name is not generic enough to be included in lens. Lens as a platform should be satisfied with providing a hook for users to use. That hook can be used for any conf changes, including dynamic queues.
> On June 2, 2015, 10:38 a.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, line 753
> > <https://reviews.apache.org/r/34876/diff/1/?file=975496#file975496line753>
> >
> > doesn't need to called from tests? I was thinking above preSubmit call would NPE, if not called.
Right now, Drivers only call presubmit if config loader is provided. But your point is correct. Considering that config loader will always be set by execution service, drivers need not check for null.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review86160
-----------------------------------------------------------
On June 1, 2015, 6:38 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 1, 2015, 6:38 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dbc52e9fcda9bb897264c1e357a679454ef1983f
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
> [INFO] Lens .............................................. SUCCESS [2.921s]
> [INFO] Lens API .......................................... SUCCESS [18.691s]
> [INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
> [INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
> [INFO] Lens DB storage ................................... SUCCESS [21.515s]
> [INFO] Lens Query Library ................................ SUCCESS [17.675s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
> [INFO] Lens Server ....................................... SUCCESS [5:25.651s]
> [INFO] Lens client ....................................... SUCCESS [36.611s]
> [INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
> [INFO] Lens Examples ..................................... SUCCESS [9.916s]
> [INFO] Lens Distribution ................................. SUCCESS [7.971s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
> [INFO] Lens Regression ................................... SUCCESS [8.378s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 19:44.006s
> [INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
> [INFO] Final Memory: 162M/1110M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
> On June 2, 2015, 10:38 a.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, line 753
> > <https://reviews.apache.org/r/34876/diff/1/?file=975496#file975496line753>
> >
> > doesn't need to called from tests? I was thinking above preSubmit call would NPE, if not called.
>
> Rajat Khandelwal wrote:
> Right now, Drivers only call presubmit if config loader is provided. But your point is correct. Considering that config loader will always be set by execution service, drivers need not check for null.
I think let's keep the check of `if config loader set in driver, only then call pre submit`. Driver shouldn't fail if the hook wasn't registered.
- Rajat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review86160
-----------------------------------------------------------
On June 1, 2015, 6:38 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 1, 2015, 6:38 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dbc52e9fcda9bb897264c1e357a679454ef1983f
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
> [INFO] Lens .............................................. SUCCESS [2.921s]
> [INFO] Lens API .......................................... SUCCESS [18.691s]
> [INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
> [INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
> [INFO] Lens DB storage ................................... SUCCESS [21.515s]
> [INFO] Lens Query Library ................................ SUCCESS [17.675s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
> [INFO] Lens Server ....................................... SUCCESS [5:25.651s]
> [INFO] Lens client ....................................... SUCCESS [36.611s]
> [INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
> [INFO] Lens Examples ..................................... SUCCESS [9.916s]
> [INFO] Lens Distribution ................................. SUCCESS [7.971s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
> [INFO] Lens Regression ................................... SUCCESS [8.378s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 19:44.006s
> [INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
> [INFO] Final Memory: 162M/1110M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/#review86160
-----------------------------------------------------------
- Can you include unit tests?
- Did not see any changes wrt dynamic queue name in the loader? Are you planning include those changes in a follow up patch ?
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java
<https://reviews.apache.org/r/34876/#comment138080>
doesn't need to called from tests? I was thinking above preSubmit call would NPE, if not called.
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java
<https://reviews.apache.org/r/34876/#comment138081>
Can you add more details about what this config and whats the loader here ?
We can also mention usecases.
- Amareshwari Sriramadasu
On June 1, 2015, 1:08 p.m., Rajat Khandelwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34876/
> -----------------------------------------------------------
>
> (Updated June 1, 2015, 1:08 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: LENS-555
> https://issues.apache.org/jira/browse/LENS-555
>
>
> Repository: lens
>
>
> Description
> -------
>
> Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
>
>
> Diffs
> -----
>
> lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
> lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dbc52e9fcda9bb897264c1e357a679454ef1983f
> lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
> lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
> lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
> lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
> lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
> lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
>
> Diff: https://reviews.apache.org/r/34876/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
> [INFO] Lens .............................................. SUCCESS [2.921s]
> [INFO] Lens API .......................................... SUCCESS [18.691s]
> [INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
> [INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
> [INFO] Lens DB storage ................................... SUCCESS [21.515s]
> [INFO] Lens Query Library ................................ SUCCESS [17.675s]
> [INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
> [INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
> [INFO] Lens Server ....................................... SUCCESS [5:25.651s]
> [INFO] Lens client ....................................... SUCCESS [36.611s]
> [INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
> [INFO] Lens Examples ..................................... SUCCESS [9.916s]
> [INFO] Lens Distribution ................................. SUCCESS [7.971s]
> [INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
> [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
> [INFO] Lens Regression ................................... SUCCESS [8.378s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 19:44.006s
> [INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
> [INFO] Final Memory: 162M/1110M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Rajat Khandelwal
>
>
Re: Review Request 34876: LENS-555: Queue names for Hive driver need
to be dynamic
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34876/
-----------------------------------------------------------
(Updated June 1, 2015, 6:38 p.m.)
Review request for lens.
Bugs: LENS-555
https://issues.apache.org/jira/browse/LENS-555
Repository: lens
Description
-------
Right now the hadoop queue where the hive job is triggered is fixed. This needs to be dynamic. We have one requirement where our yarn cluster has hierarchical queues based on job priority. We need capability to be able to choose the right queue based on priority of the job
Diffs
-----
lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 052d0e2bd0584e65dd06c0241dcf99dc1773045b
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 70deaa401fa5168ad7780804ef68cca8a2660eae
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 8511ce63b3845973d7532ffa15c7158e69726716
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoader.java PRE-CREATION
lens-server-api/src/main/java/org/apache/lens/server/api/user/UserConfigLoaderException.java PRE-CREATION
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java dbc52e9fcda9bb897264c1e357a679454ef1983f
lens-server/src/main/java/org/apache/lens/server/user/CustomUserConfigLoader.java 9a6a350a2808bb844ef870089fccf20a2136de72
lens-server/src/main/java/org/apache/lens/server/user/DatabaseUserConfigLoader.java d2679488aeb2b91105f087e34143bc848f1f3d52
lens-server/src/main/java/org/apache/lens/server/user/FixedUserConfigLoader.java b81b994ad00d09f5a0b9dd44cbdf731da3b2d0e6
lens-server/src/main/java/org/apache/lens/server/user/LDAPBackedDatabaseUserConfigLoader.java 108a0123a78736892a3adb8bb24ff401e5a81b4e
lens-server/src/main/java/org/apache/lens/server/user/PropertyBasedUserConfigLoader.java 16a9a709d097c8af8a84ea81282d0d43947fcd3d
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoader.java 21eea3bdbd5a84770e6144d1ed45806ad4dc6ee3
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderException.java 0899746f8350b4b84c1e03c3f262f553faef9817
lens-server/src/main/java/org/apache/lens/server/user/UserConfigLoaderFactory.java 383aaf5f0d5a3937b95b0077491d30678ffe46a3
Diff: https://reviews.apache.org/r/34876/diff/
Testing (updated)
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.690s]
[INFO] Lens .............................................. SUCCESS [2.921s]
[INFO] Lens API .......................................... SUCCESS [18.691s]
[INFO] Lens API for server and extensions ................ SUCCESS [21.498s]
[INFO] Lens Cube ......................................... SUCCESS [3:34.121s]
[INFO] Lens DB storage ................................... SUCCESS [21.515s]
[INFO] Lens Query Library ................................ SUCCESS [17.675s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:54.117s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [37.549s]
[INFO] Lens Server ....................................... SUCCESS [5:25.651s]
[INFO] Lens client ....................................... SUCCESS [36.611s]
[INFO] Lens CLI .......................................... SUCCESS [3:21.647s]
[INFO] Lens Examples ..................................... SUCCESS [9.916s]
[INFO] Lens Distribution ................................. SUCCESS [7.971s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:19.772s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [2.261s]
[INFO] Lens Regression ................................... SUCCESS [8.378s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 19:44.006s
[INFO] Finished at: Mon Jun 01 11:12:36 UTC 2015
[INFO] Final Memory: 162M/1110M
[INFO] ------------------------------------------------------------------------
Thanks,
Rajat Khandelwal