You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by "Jaideep Dhok (JIRA)" <ji...@apache.org> on 2015/03/18 11:40:38 UTC
[jira] [Commented] (LENS-413) Lens unable to restart when lot of
open session is persisted
[ https://issues.apache.org/jira/browse/LENS-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366949#comment-14366949 ]
Jaideep Dhok commented on LENS-413:
-----------------------------------
While restarting sessions, we set the parameters of restored sessions on underlying CLI service. This is done using an execute statement operation which is creating the file. We are not closing the execute statement operation. This might be the root cause.
> Lens unable to restart when lot of open session is persisted
> ------------------------------------------------------------
>
> Key: LENS-413
> URL: https://issues.apache.org/jira/browse/LENS-413
> Project: Apache Lens
> Issue Type: Bug
> Reporter: Arshad Matin
> Assignee: Jaideep Dhok
>
> Scenario :
> Started lens server
> Opened around 100 session
> stopped it
> Restarted it and restart failed
> Logs
> {noformat}
> 12 Mar 2015 09:51:48,356 [main] INFO org.apache.lens.server.session.HiveSessionService - Request to Set param key:hive.server2.authentication.ldap.security.protocol value:ssl
> 12 Mar 2015 09:51:48,356 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /tmp/dataqa/aba21db9-2ae9-47dc-aa60-358940a1ff2a87568105576796952.pipeout
> 12 Mar 2015 09:51:48,356 [main] ERROR org.apache.hive.service.cli.operation.Operation - Error in creating temp output file
> java.io.FileNotFoundException: /tmp/dataqa/aba21db9-2ae9-47dc-aa60-358940a1ff2a87568105576796952.pipeout (Too many open files)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
> at org.apache.hive.service.cli.operation.HiveCommandOperation.setupSessionIO(HiveCommandOperation.java:73)
> at org.apache.hive.service.cli.operation.HiveCommandOperation.<init>(HiveCommandOperation.java:65)
> at org.apache.hive.service.cli.operation.ExecuteStatementOperation.newExecuteStatementOperation(ExecuteStatementOperation.java:58)
> at org.apache.hive.service.cli.operation.OperationManager.newExecuteStatementOperation(OperationManager.java:76)
> at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:292)
> at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:277)
> at org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:238)
> at org.apache.lens.server.session.HiveSessionService.setSessionParameter(HiveSessionService.java:304)
> at org.apache.lens.server.session.HiveSessionService.start(HiveSessionService.java:369)
> at org.apache.hive.service.CompositeService.start(CompositeService.java:70)
> at org.apache.lens.server.LensServices.start(LensServices.java:235)
> at org.apache.lens.server.LensServer.startServices(LensServer.java:138)
> at org.apache.lens.server.LensServer.<init>(LensServer.java:91)
> at org.apache.lens.server.LensServer.createLensServer(LensServer.java:79)
> at org.apache.lens.server.LensServer.main(LensServer.java:190)
> 12 Mar 2015 09:51:48,357 [main] INFO org.apache.lens.server.session.HiveSessionService - Set param key:hive.server2.authentication.ldap.security.protocol value:ssl
> 12 Mar 2015 09:51:48,357 [main] INFO org.apache.lens.server.session.HiveSessionService - Request to Set param key:lens.session.loggedin.user value:test
> 12 Mar 2015 09:51:48,357 [main] INFO org.apache.hive.service.cli.operation.Operation - Putting temp output to file /tmp/dataqa/aba21db9-2ae9-47dc-aa60-358940a1ff2a87568105576796952.pipeout
> 12 Mar 2015 09:51:48,357 [main] ERROR org.apache.hive.service.cli.operation.Operation - Error in creating temp output file
> java.io.FileNotFoundException: /tmp/dataqa/aba21db9-2ae9-47dc-aa60-358940a1ff2a87568105576796952.pipeout (Too many open files)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
> at org.apache.hive.service.cli.operation.HiveCommandOperation.setupSessionIO(HiveCommandOperation.java:73)
> at org.apache.hive.service.cli.operation.HiveCommandOperation.<init>(HiveCommandOperation.java:65)
> at org.apache.hive.service.cli.operation.ExecuteStatementOperation.newExecuteStatementOperation(ExecuteStatementOperation.java:58)
> at org.apache.hive.service.cli.operation.OperationManager.newExecuteStatementOperation(OperationManager.java:76)
> at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:292)
> at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:277)
> at org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:238)
> at org.apache.lens.server.session.HiveSessionService.setSessionParameter(HiveSessionService.java:304)
> at org.apache.lens.server.session.HiveSessionService.start(HiveSessionService.java:369)
> at org.apache.hive.service.CompositeService.start(CompositeService.java:70)
> at org.apache.lens.server.LensServices.start(LensServices.java:235)
> at org.apache.lens.server.LensServer.startServices(LensServer.java:138)
> at org.apache.lens.server.LensServer.<init>(LensServer.java:91)
> at org.apache.lens.server.LensServer.createLensServer(LensServer.java:79)
> at org.apache.lens.server.LensServer.main(LensServer.java:190)
> 12 Mar 2015 09:51:48,357 [main] INFO org.apache.lens.server.session.HiveSessionService - Set param key:lens.session.loggedin.user value:test
> 12 Mar 2015 09:51:48,358 [main] INFO org.apache.lens.server.session.HiveSessionService - Restored session aba21db9-2ae9-47dc-aa60-358940a1ff2a
> 12 Mar 2015 09:51:48,366 [main] ERROR org.apache.hive.service.CompositeService - Error starting services lens_services
> java.lang.RuntimeException: org.apache.lens.api.LensException: Error restoring session <?xml version="1.0" encoding="UTF-8" standalone="yes"?><lensSessionHandle><publicId>3f6c815b-d7c5-48d4-9671-c78fe04a05ed</publicId><secretId>c413685e-10f2-43ee-b920-6cc26801680c</secretId></lensSessionHandle>
> at org.apache.lens.server.session.HiveSessionService.start(HiveSessionService.java:376)
> at org.apache.hive.service.CompositeService.start(CompositeService.java:70)
> at org.apache.lens.server.LensServices.start(LensServices.java:235)
> at org.apache.lens.server.LensServer.startServices(LensServer.java:138)
> at org.apache.lens.server.LensServer.<init>(LensServer.java:91)
> at org.apache.lens.server.LensServer.createLensServer(LensServer.java:79)
> at org.apache.lens.server.LensServer.main(LensServer.java:190)
> Caused by: org.apache.lens.api.LensException: Error restoring session <?xml version="1.0" encoding="UTF-8" standalone="yes"?><lensSessionHandle><publicId>3f6c815b-d7c5-48d4-9671-c78fe04a05ed</publicId><secretId>c413685e-10f2-43ee-b920-6cc26801680c</secretId></lensSessionHandle>
> at org.apache.lens.server.LensService.restoreSession(LensService.java:173)
> at org.apache.lens.server.session.HiveSessionService.start(HiveSessionService.java:350)
> ... 6 more
> Caused by: org.apache.hive.service.cli.HiveSQLException: Cannot initilize session class:org.apache.lens.server.session.LensSessionImpl
> at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:217)
> at org.apache.hive.service.cli.session.SessionManager.restoreSession(SessionManager.java:244)
> at org.apache.hive.service.cli.CLIService.restoreSession(CLIService.java:190)
> at org.apache.lens.server.LensService.restoreSession(LensService.java:167)
> ... 7 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.GeneratedConstructorAccessor8.newInstance(Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:534)
> at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:207)
> ... 10 more
> Caused by: java.lang.RuntimeException: java.io.IOException: Too many open files
> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:333)
> at org.apache.hive.service.cli.session.HiveSessionImpl.<init>(HiveSessionImpl.java:113)
> at org.apache.lens.server.session.LensSessionImpl.<init>(LensSessionImpl.java:161)
> ... 14 more
> Caused by: java.io.IOException: Too many open files
> at java.io.UnixFileSystem.createFileExclusively(Native Method)
> at java.io.File.checkAndCreate(File.java:1833)
> at java.io.File.createTempFile0(File.java:1857)
> at java.io.File.createTempFile(File.java:1934)
> at org.apache.hadoop.hive.ql.session.SessionState.createTempFile(SessionState.java:435)
> at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:331)
> ... 16 more
> 12 Mar 2015 09:51:48,367 [main] INFO org.apache.hive.service.AbstractService - Service:database-resources is stopped.
> 12 Mar 2015 09:51:48,367 [main] INFO org.apache.hive.service.AbstractService - Service:session is stopped.
> 12 Mar 2015 09:51:48,368 [main] INFO org.apache.lens.server.EventServiceImpl - Removed listener org.apache.lens.server.stats.store.log.LogStatisticsStore@2d9d4f83
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)