You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Rajat Khandelwal <ra...@gmail.com> on 2015/06/16 11:56:52 UTC
Re: Review Request 35499: LENS-569: Too many "set param key" on
session restoration upon restart
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35499/
-----------------------------------------------------------
(Updated June 16, 2015, 3:26 p.m.)
Review request for lens.
Bugs: LENS-569
https://issues.apache.org/jira/browse/LENS-569
Repository: lens
Description (updated)
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.696s]
[INFO] Lens .............................................. SUCCESS [2.915s]
[INFO] Lens API .......................................... SUCCESS [20.910s]
[INFO] Lens API for server and extensions ................ SUCCESS [20.542s]
[INFO] Lens Cube ......................................... SUCCESS [3:40.052s]
[INFO] Lens DB storage ................................... SUCCESS [20.514s]
[INFO] Lens Query Library ................................ SUCCESS [16.568s]
[INFO] Lens Hive Driver .................................. SUCCESS [3:00.986s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [35.559s]
[INFO] Lens Server ....................................... SUCCESS [5:36.976s]
[INFO] Lens client ....................................... SUCCESS [36.729s]
[INFO] Lens CLI .......................................... SUCCESS [3:34.542s]
[INFO] Lens Examples ..................................... SUCCESS [9.325s]
[INFO] Lens Distribution ................................. SUCCESS [8.691s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:22.352s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.774s]
[INFO] Lens Regression ................................... SUCCESS [11.609s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 20:23.719s
[INFO] Finished at: Tue Jun 16 09:35:13 UTC 2015
[INFO] Final Memory: 160M/1041M
[INFO] ------------------------------------------------------------------------
before the change:
```
21 May 2015 06:05:50,806 [main] INFO org.apache.lens.server.session.HiveSessionService - Set param key:lens.query.output.enable.compression value:true
21 May 2015 06:05:50,806 [main] INFO org.apache.lens.server.session.HiveSessionService - Request to Set param key:hive.metastore.client.socket.timeout value:600
21 May 2015 06:05:50,806 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /tmp/lens/26064577-7a38-4279-ad73-0daa4a32a7671862119368791234528.pipeout
21 May 2015 06:05:50,806 [main] INFO org.apache.lens.server.session.HiveSessionService - Set param key:hive.metastore.client.socket.timeout value:600
21 May 2015 06:05:50,806 [main] INFO org.apache.lens.server.session.HiveSessionService - Request to Set param key:hive.metastore.connect.retries value:5
21 May 2015 06:05:50,806 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /tmp/lens/26064577-7a38-4279-ad73-0daa4a32a7671862119368791234528.pipeout
21 May 2015 06:05:50,806 [main] INFO org.apache.lens.server.session.HiveSessionService - Set param key:hive.metastore.connect.retries value:5
21 May 2015 06:05:50,806 [main] INFO org.apache.lens.server.session.HiveSessionService - Request to Set param key:lens.query.enable.mail.notify value:true
21 May 2015 06:05:50,806 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /tmp/lens/26064577-7a38-4279-ad73-0daa4a32a7671862119368791234528.pipeout
21 May 2015 06:05:50,807 [main] INFO org.apache.lens.server.session.HiveSessionService - Set param key:lens.query.enable.mail.notify value:true
21 May 2015 06:05:50,807 [main] INFO org.apache.lens.server.session.HiveSessionService - Restored session 26064577-7a38-4279-ad73-0daa4a32a767
21 May 2015 06:05:50,807 [main] INFO org.apache.hadoop.hive.ql.session.SessionState - No Tez session required at this point. hive.execution.engine=mr.
21 May 2015 06:05:50,808 [main] INFO org.apache.hadoop.hive.ql.session.SessionState - No Tez session required at this point. hive.execution.engine=mr.
21 May 2015 06:05:50,808 [main] INFO org.apache.lens.server.session.HiveSessionService - Request to Set param key:hive.metastore.failure.retries value:3
21 May 2015 06:05:50,810 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /tmp/lens/4d63f0f8-699c-4a3e-ae2d-6eaf03a5a3437220243919237070139.pipeout
21 May 2015 06:05:50,810 [main] INFO org.apache.lens.server.session.HiveSessionService - Set param key:hive.metastore.failure.retries value:3
21 May 2015 06:05:50,810 [main] INFO org.apache.lens.server.session.HiveSessionService - Request to Set param key:hive.metastore.client.connect.retry.delay value:1
21 May 2015 06:05:50,811 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /tmp/lens/4d63f0f8-699c-4a3e-ae2d-6eaf03a5a3437220243919237070139.pipeout
```
After:
```
16 Jun 2015 14:36:31,607 [main] INFO org.apache.lens.server.session.HiveSessionService - Request to Set params:{hive.metastore.failure.retries=3, hive.metastore.client.connect.retry.delay=1, e=f, lens.server.base.url=http://0.0.0.0:9999/lensapi, hive.metastore.batch.retrieve.max=100, c=d, lens.session.cluster.user=rajat.khandelwal, a=b, hive.metastore.batch.retrieve.table.partition.max=500, lens.session.loggedin.user=anonymous, lens.client.user.name=anonymous, hive.metastore.client.socket.timeout=20, hive.metastore.connect.retries=5, x=y, lens.client.dbname=default}
16 Jun 2015 14:36:31,629 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,633 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,634 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,635 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,636 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,637 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,637 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,638 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,639 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,640 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,640 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,641 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,642 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,642 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,643 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /var/folders/m6/05_cdz4d6dn1jh51c_dyg46mhr0n9j/T/rajat.khandelwal/38be3e57-6980-4563-8204-476ee13fc14b1668070761390146027.pipeout
16 Jun 2015 14:36:31,643 [main] INFO org.apache.lens.server.session.HiveSessionService - Set params:{hive.metastore.failure.retries=3, hive.metastore.client.connect.retry.delay=1, e=f, lens.server.base.url=http://0.0.0.0:9999/lensapi, hive.metastore.batch.retrieve.max=100, c=d, lens.session.cluster.user=rajat.khandelwal, a=b, hive.metastore.batch.retrieve.table.partition.max=500, lens.session.loggedin.user=anonymous, lens.client.user.name=anonymous, hive.metastore.client.socket.timeout=20, hive.metastore.connect.retries=5, x=y, lens.client.dbname=default}
```
The actual params are different, but I'm just demonstrating here.
Diffs
-----
lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java fa061ec323b2917c185d8f77661e2cc39fc2ede0
lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 30016a2aee2845bb47ec6fb1a0ec812cfdf992ef
Diff: https://reviews.apache.org/r/35499/diff/
Testing
-------
Thanks,
Rajat Khandelwal