You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@apex.apache.org by Ananth Gundabattula <ag...@gmail.com> on 2016/05/02 13:27:55 UTC
WebService errors in DT 3.3/3.2
Hello All,
I am trying to launch my app on Apex/DT 3.3 version running on top of CDH
5.7. I tried on DT/Apex 3.2 as well running on CDH 5.7. Both these
deployments show the following behavior. (We have a requirement to run Apex
on CDH 5.7 )
I was wondering if you could advise on any workarounds for the following
issue:
I am not able to see any metrics in the web console when I open the
"Monitor" section. The application code seems to be working fine as I see
data flow happening fine as expected. However I see no metrics ( even the
top level container metrics) .
One more observed issue is that when I click on the container ID from the
monitor section, the UI crashes. ( In the sense the UI simply does not show
anything ). I am however able to get some preliminary UI ( barring the
metrics ) if I click on the container link from the monitor section if the
application is in the "ACCEPTED" state. The container link does not work
the moment the app moves from ACCEPTED to Running State.
The DtGateway shows the following logs :
2016-05-02 11:08:48,964 WARN com.datatorrent.stram.client.StramAgent:
> Caught exception when retrieving web service info for app
> application_1462007878015_0037
> org.codehaus.jettison.json.JSONException: A JSONObject text must begin
> with '{' at character 1 of <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
> <title>Error 500
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
> </head>
> <body><h2>HTTP ERROR 500</h2>
> <p>Problem accessing /ws. Reason:
> <pre>
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
> by:</h3><pre>java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
> at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
> at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> </pre>
> <hr /><i><small>Powered by Jetty://</small></i><br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
>
> </body>
> </html>
>
> at
> org.codehaus.jettison.json.JSONTokener.syntaxError(JSONTokener.java:439)
> at org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:169)
> at org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:266)
> at
> com.datatorrent.stram.client.StramAgent.retrieveWebServicesInfo(StramAgent.java:402)
> at
> com.datatorrent.stram.client.StramAgent.getWebServicesInfo(StramAgent.java:181)
> at
> com.datatorrent.stram.client.StramAgent.getStramWebURIBuilder(StramAgent.java:205)
> at
> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:275)
> at
> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:305)
> at
> com.datatorrent.stram.client.StramAgent.issueStramWebGetRequest(StramAgent.java:311)
> at
> com.datatorrent.gateway.resources.ws.v2.ApplicationsResource.getApplication(sc:378)
> at com.datatorrent.gateway.R.F(ud:255)
> at com.datatorrent.gateway.R.call(ud:157)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-05-02 11:08:48,965 WARN com.datatorrent.gateway.u: Cannot get
> application info for app application_1462007878015_0037. HTTP returns 404
>
Regards,
Ananth
Re: WebService errors in DT 3.3/3.2
Posted by David Yan <da...@datatorrent.com>.
Hi Ananth,
Is it possible for you to isolate the problem by removing any operator/code
that doesn't matter and any sensitive code, and send me the stripped down
project so that I can troubleshoot on my end?
David
On Mon, May 2, 2016 at 11:27 PM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
> Hello David,
>
> I just googled a bit and realized maybe this issue is arising because I
> added a jar in the lib folder of Apex server. I had to add
> javaee-api-7.0.jar in the lib folder of Apex engine because I am getting an
> error of ClassNotFoundException for a class javax.persistence.Persistence.
>
> More details in the problem I raised with Siyuan in the mailing list ( It
> is still active and not yet closed ). It has been responded to today
> itself.
>
>
> So the problem is this :
>
>
> Whenever I try to launch my application, the STRAM client validates my app
> and in the process is throwing a ClassNotFoundException for
> javax.persistence.Persistence.
>
> I am still stumped as to why the STRAM client is looking for this class (
> The App which I am trying to launch has Spring, JPA 2.1 , Hibernate
> dependencies ) . Since I could not solve the problem of class not found
> Exception inspite of adding it to my app pom dependencies, I copied the
> javaee-api-7.0 jar into the lib folder of Apex engine which solved the
> problem and helped me to atleast launch the app. It may be noted that the
> Apex engine cannot be restarted when this jar is in the lib folder. So
> every time I restart the Apex server, I have to move this jar aside ,
> restart Apex etc.
>
>
> I know the above is a completely stupid hack but have been left with no
> choice until I solve the classNotFoundException for an App having Spring +
> Jpa + Hibernate in the dependencies. I am sure it is not a packaging issue
> with my app because once my app launches, rest of the jars are picked up at
> runtime properly. Somehow there seems to be an issue explicitly in the
> validation phase of the Stram Client when it tries to launch.
>
>
> Please let me know if you think that the additional jar in the lib folder
> is causing the above issue.
>
> If that is the case, then I am struck as I will not be able to launch my
> app at all in the first place.
>
> Please advise.
>
> Regards,
> Ananth
>
> On Tue, May 3, 2016 at 4:14 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Hello David,
>>
>> Here is the response by querying the URL :
>>
>>
>> curl -D - http://dwh102.qa1.sac.int.threatmetrix.com:40279/ws/v2/stram
>>
>>
>> HTTP/1.1 500
>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>> Cache-Control: must-revalidate,no-cache,no-store
>> Date: Tue, 03 May 2016 06:13:01 GMT
>> Pragma: no-cache
>> Date: Tue, 03 May 2016 06:13:01 GMT
>> Pragma: no-cache
>> Content-Type: text/html; charset=iso-8859-1
>> Content-Length: 4293
>> Server: Jetty(6.1.26.cloudera.4)
>>
>> <html>
>> <head>
>> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
>> <title>Error 500
>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>> </head>
>> <body><h2>HTTP ERROR 500</h2>
>> <p>Problem accessing /ws/v2/stram. Reason:
>> <pre>
>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>> by:</h3><pre>java.lang.AbstractMethodError:
>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>> at
>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>> at
>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>> at
>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>> at
>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>> at
>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>> at
>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>> at
>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>> at
>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>> at
>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>> at
>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>> at
>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>> at
>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>> at
>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>> at org.mortbay.jetty.Server.handle(Server.java:326)
>> at
>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>> at
>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>> at
>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>> at
>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>> </pre>
>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>>
>> </body>
>> </html>
>>
>> Regards,
>> Ananth
>>
>> On Tue, May 3, 2016 at 3:38 PM, David Yan <da...@datatorrent.com> wrote:
>>
>>> Hi Ananth,
>>>
>>> In the app master log, look for a line that says something like:
>>>
>>> 10:52:21.182 [main] INFO c.d.stram.StreamingAppMasterService - Setting
>>> tracking URL to: dwh102.qa1.sac.int.threatmetrix.com:35066
>>>
>>> The above is taken from the app master log you sent earlier. Note the
>>> URL.
>>> Then append /ws/v2/stram to it and try to query the URL using curl. For
>>> example:
>>>
>>> curl -D - http://dwh102.qa1.sac.int.threatmetrix.com:35066/ws/v2/stram
>>>
>>> Let me know what it returns.
>>>
>>> Thanks,
>>>
>>> David
>>>
>>>
>>> On Mon, May 2, 2016 at 6:36 PM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Hello David,
>>>>
>>>> Here is the output
>>>>
>>>> curl -D -
>>>> http://RM:8088/proxy/application_1462007878015_0038/ws/v2/stram ( Ran
>>>> it for another app as the original app was killed and a new app was
>>>> launched which also shows the exact same behavior)
>>>>
>>>> HTTP/1.1 500 Internal Server Error
>>>>> Cache-Control: must-revalidate,no-cache,no-store
>>>>> Expires: Tue, 03 May 2016 01:33:27 GMT
>>>>> Date: Tue, 03 May 2016 01:33:27 GMT
>>>>> Pragma: no-cache
>>>>> Expires: Tue, 03 May 2016 01:33:27 GMT
>>>>> Content-Type: text/html; charset=iso-8859-1
>>>>> Content-Length: 4293
>>>>> Server: Jetty(6.1.26.cloudera.4)
>>>>> Connection: close
>>>>>
>>>>> <html>
>>>>> <head>
>>>>> <meta http-equiv="Content-Type" content="text/html;
>>>>> charset=ISO-8859-1"/>
>>>>> <title>Error 500
>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>>>>> </head>
>>>>> <body><h2>HTTP ERROR 500</h2>
>>>>> <p>Problem accessing /ws/v2/stram. Reason:
>>>>>
>>>>> <pre>
>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>>>>> by:</h3><pre>java.lang.AbstractMethodError:
>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>>>>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>>>>> at
>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>>>>> at
>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>>>>> at
>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>>>>> at
>>>>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>>>>> at
>>>>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>>>>> at
>>>>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>>>>> at
>>>>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>> at
>>>>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>> at
>>>>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>> at
>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>> at
>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>>>> at
>>>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>>> at
>>>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>>> at
>>>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>>>>> at
>>>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>>> at
>>>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>>> at
>>>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>>> at org.mortbay.jetty.Server.handle(Server.java:326)
>>>>> at
>>>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>>> at
>>>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>>> at
>>>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>>>>> at
>>>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>>> </pre>
>>>>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>>
>>>>> </body>
>>>>> </html>
>>>>>
>>>>
>>>> On Tue, May 3, 2016 at 11:20 AM, David Yan <da...@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Hi Ananth,
>>>>>
>>>>> I don't see anything that stands out in the application master log.
>>>>> Can you send me the output of this command?
>>>>>
>>>>> curl -D - http://<rm_ip>:8088/proxy/<your_app_id>/ws/v2/stram
>>>>>
>>>>> rm_ip is the IP address of the resource manager in your hadoop cluster.
>>>>>
>>>>> David
>>>>>
>>>>> On Mon, May 2, 2016 at 6:07 PM, Ananth Gundabattula <
>>>>> agundabattula@gmail.com> wrote:
>>>>>
>>>>>> Hello David,
>>>>>>
>>>>>> Thanks for taking this thread up.
>>>>>>
>>>>>> Here is the log of the application master.
>>>>>>
>>>>>> Please note that I could not upload the entire log as the log size
>>>>>> was too much and the attached log contains the debug version of the logs
>>>>>> for the first few hours. I believe the error started occuring in the first
>>>>>> few minutes of launching the app itself as described in the mail above.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Ananth
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, May 3, 2016 at 1:53 AM, David Yan <da...@datatorrent.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ananth,
>>>>>>>
>>>>>>> This error indicates that dtGateway has trouble accessing the app
>>>>>>> master.
>>>>>>> Can you check the app master log for
>>>>>>> application_1462007878015_0037?
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> On Mon, May 2, 2016 at 4:27 AM, Ananth Gundabattula <
>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello All,
>>>>>>>>
>>>>>>>> I am trying to launch my app on Apex/DT 3.3 version running on top
>>>>>>>> of CDH 5.7. I tried on DT/Apex 3.2 as well running on CDH 5.7. Both these
>>>>>>>> deployments show the following behavior. (We have a requirement to run Apex
>>>>>>>> on CDH 5.7 )
>>>>>>>>
>>>>>>>> I was wondering if you could advise on any workarounds for the
>>>>>>>> following issue:
>>>>>>>>
>>>>>>>> I am not able to see any metrics in the web console when I open the
>>>>>>>> "Monitor" section. The application code seems to be working fine as I see
>>>>>>>> data flow happening fine as expected. However I see no metrics ( even the
>>>>>>>> top level container metrics) .
>>>>>>>>
>>>>>>>> One more observed issue is that when I click on the container ID
>>>>>>>> from the monitor section, the UI crashes. ( In the sense the UI simply does
>>>>>>>> not show anything ). I am however able to get some preliminary UI ( barring
>>>>>>>> the metrics ) if I click on the container link from the monitor section if
>>>>>>>> the application is in the "ACCEPTED" state. The container link does not
>>>>>>>> work the moment the app moves from ACCEPTED to Running State.
>>>>>>>>
>>>>>>>>
>>>>>>>> The DtGateway shows the following logs :
>>>>>>>>
>>>>>>>> 2016-05-02 11:08:48,964 WARN
>>>>>>>>> com.datatorrent.stram.client.StramAgent: Caught exception when retrieving
>>>>>>>>> web service info for app application_1462007878015_0037
>>>>>>>>> org.codehaus.jettison.json.JSONException: A JSONObject text must
>>>>>>>>> begin with '{' at character 1 of <html>
>>>>>>>>> <head>
>>>>>>>>> <meta http-equiv="Content-Type" content="text/html;
>>>>>>>>> charset=ISO-8859-1"/>
>>>>>>>>> <title>Error 500
>>>>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>>>>>>>>> </head>
>>>>>>>>> <body><h2>HTTP ERROR 500</h2>
>>>>>>>>> <p>Problem accessing /ws. Reason:
>>>>>>>>> <pre>
>>>>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>>>>>>>>> by:</h3><pre>java.lang.AbstractMethodError:
>>>>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>>>>>>>>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>>>>>>>>> at
>>>>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>>>>>>>>> at
>>>>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>>>>>>>>> at
>>>>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>>>>>>>>> at
>>>>>>>>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>>>>>>>>> at
>>>>>>>>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>>>>>>>>> at
>>>>>>>>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>>>>>>>>> at
>>>>>>>>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>>>> at
>>>>>>>>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>>>> at
>>>>>>>>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>>>> at
>>>>>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>>>> at
>>>>>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>>>>>>> at org.mortbay.jetty.Server.handle(Server.java:326)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>>>>>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>>>>>>>> at
>>>>>>>>> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>>>>>>> at
>>>>>>>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>>>>>>>>> at
>>>>>>>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>>>>>>> </pre>
>>>>>>>>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>> <br/>
>>>>>>>>>
>>>>>>>>> </body>
>>>>>>>>> </html>
>>>>>>>>>
>>>>>>>>> at
>>>>>>>>> org.codehaus.jettison.json.JSONTokener.syntaxError(JSONTokener.java:439)
>>>>>>>>> at
>>>>>>>>> org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:169)
>>>>>>>>> at
>>>>>>>>> org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:266)
>>>>>>>>> at
>>>>>>>>> com.datatorrent.stram.client.StramAgent.retrieveWebServicesInfo(StramAgent.java:402)
>>>>>>>>> at
>>>>>>>>> com.datatorrent.stram.client.StramAgent.getWebServicesInfo(StramAgent.java:181)
>>>>>>>>> at
>>>>>>>>> com.datatorrent.stram.client.StramAgent.getStramWebURIBuilder(StramAgent.java:205)
>>>>>>>>> at
>>>>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:275)
>>>>>>>>> at
>>>>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:305)
>>>>>>>>> at
>>>>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebGetRequest(StramAgent.java:311)
>>>>>>>>> at
>>>>>>>>> com.datatorrent.gateway.resources.ws.v2.ApplicationsResource.getApplication(sc:378)
>>>>>>>>> at com.datatorrent.gateway.R.F(ud:255)
>>>>>>>>> at com.datatorrent.gateway.R.call(ud:157)
>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>>>> 2016-05-02 11:08:48,965 WARN com.datatorrent.gateway.u: Cannot get
>>>>>>>>> application info for app application_1462007878015_0037. HTTP returns 404
>>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Ananth
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: WebService errors in DT 3.3/3.2
Posted by Ananth Gundabattula <ag...@gmail.com>.
Hello David,
I just googled a bit and realized maybe this issue is arising because I
added a jar in the lib folder of Apex server. I had to add
javaee-api-7.0.jar in the lib folder of Apex engine because I am getting an
error of ClassNotFoundException for a class javax.persistence.Persistence.
More details in the problem I raised with Siyuan in the mailing list ( It
is still active and not yet closed ). It has been responded to today
itself.
So the problem is this :
Whenever I try to launch my application, the STRAM client validates my app
and in the process is throwing a ClassNotFoundException for
javax.persistence.Persistence.
I am still stumped as to why the STRAM client is looking for this class (
The App which I am trying to launch has Spring, JPA 2.1 , Hibernate
dependencies ) . Since I could not solve the problem of class not found
Exception inspite of adding it to my app pom dependencies, I copied the
javaee-api-7.0 jar into the lib folder of Apex engine which solved the
problem and helped me to atleast launch the app. It may be noted that the
Apex engine cannot be restarted when this jar is in the lib folder. So
every time I restart the Apex server, I have to move this jar aside ,
restart Apex etc.
I know the above is a completely stupid hack but have been left with no
choice until I solve the classNotFoundException for an App having Spring +
Jpa + Hibernate in the dependencies. I am sure it is not a packaging issue
with my app because once my app launches, rest of the jars are picked up at
runtime properly. Somehow there seems to be an issue explicitly in the
validation phase of the Stram Client when it tries to launch.
Please let me know if you think that the additional jar in the lib folder
is causing the above issue.
If that is the case, then I am struck as I will not be able to launch my
app at all in the first place.
Please advise.
Regards,
Ananth
On Tue, May 3, 2016 at 4:14 PM, Ananth Gundabattula <agundabattula@gmail.com
> wrote:
> Hello David,
>
> Here is the response by querying the URL :
>
>
> curl -D - http://dwh102.qa1.sac.int.threatmetrix.com:40279/ws/v2/stram
>
>
> HTTP/1.1 500
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> Cache-Control: must-revalidate,no-cache,no-store
> Date: Tue, 03 May 2016 06:13:01 GMT
> Pragma: no-cache
> Date: Tue, 03 May 2016 06:13:01 GMT
> Pragma: no-cache
> Content-Type: text/html; charset=iso-8859-1
> Content-Length: 4293
> Server: Jetty(6.1.26.cloudera.4)
>
> <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
> <title>Error 500
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
> </head>
> <body><h2>HTTP ERROR 500</h2>
> <p>Problem accessing /ws/v2/stram. Reason:
> <pre>
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
> by:</h3><pre>java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
> at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
> at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> </pre>
> <hr /><i><small>Powered by Jetty://</small></i><br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
>
> </body>
> </html>
>
> Regards,
> Ananth
>
> On Tue, May 3, 2016 at 3:38 PM, David Yan <da...@datatorrent.com> wrote:
>
>> Hi Ananth,
>>
>> In the app master log, look for a line that says something like:
>>
>> 10:52:21.182 [main] INFO c.d.stram.StreamingAppMasterService - Setting
>> tracking URL to: dwh102.qa1.sac.int.threatmetrix.com:35066
>>
>> The above is taken from the app master log you sent earlier. Note the
>> URL.
>> Then append /ws/v2/stram to it and try to query the URL using curl. For
>> example:
>>
>> curl -D - http://dwh102.qa1.sac.int.threatmetrix.com:35066/ws/v2/stram
>>
>> Let me know what it returns.
>>
>> Thanks,
>>
>> David
>>
>>
>> On Mon, May 2, 2016 at 6:36 PM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> Hello David,
>>>
>>> Here is the output
>>>
>>> curl -D -
>>> http://RM:8088/proxy/application_1462007878015_0038/ws/v2/stram ( Ran
>>> it for another app as the original app was killed and a new app was
>>> launched which also shows the exact same behavior)
>>>
>>> HTTP/1.1 500 Internal Server Error
>>>> Cache-Control: must-revalidate,no-cache,no-store
>>>> Expires: Tue, 03 May 2016 01:33:27 GMT
>>>> Date: Tue, 03 May 2016 01:33:27 GMT
>>>> Pragma: no-cache
>>>> Expires: Tue, 03 May 2016 01:33:27 GMT
>>>> Content-Type: text/html; charset=iso-8859-1
>>>> Content-Length: 4293
>>>> Server: Jetty(6.1.26.cloudera.4)
>>>> Connection: close
>>>>
>>>> <html>
>>>> <head>
>>>> <meta http-equiv="Content-Type" content="text/html;
>>>> charset=ISO-8859-1"/>
>>>> <title>Error 500
>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>>>> </head>
>>>> <body><h2>HTTP ERROR 500</h2>
>>>> <p>Problem accessing /ws/v2/stram. Reason:
>>>>
>>>> <pre>
>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>>>> by:</h3><pre>java.lang.AbstractMethodError:
>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>>>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>>>> at
>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>>>> at
>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>>>> at
>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>>>> at
>>>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>>>> at
>>>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>>>> at
>>>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>>>> at
>>>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> at
>>>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> at
>>>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> at
>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> at
>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>>> at
>>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>> at
>>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>> at
>>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>>>> at
>>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>> at
>>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>> at
>>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>> at org.mortbay.jetty.Server.handle(Server.java:326)
>>>> at
>>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>> at
>>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>> at
>>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>>>> at
>>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>> </pre>
>>>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>>
>>>> </body>
>>>> </html>
>>>>
>>>
>>> On Tue, May 3, 2016 at 11:20 AM, David Yan <da...@datatorrent.com>
>>> wrote:
>>>
>>>> Hi Ananth,
>>>>
>>>> I don't see anything that stands out in the application master log.
>>>> Can you send me the output of this command?
>>>>
>>>> curl -D - http://<rm_ip>:8088/proxy/<your_app_id>/ws/v2/stram
>>>>
>>>> rm_ip is the IP address of the resource manager in your hadoop cluster.
>>>>
>>>> David
>>>>
>>>> On Mon, May 2, 2016 at 6:07 PM, Ananth Gundabattula <
>>>> agundabattula@gmail.com> wrote:
>>>>
>>>>> Hello David,
>>>>>
>>>>> Thanks for taking this thread up.
>>>>>
>>>>> Here is the log of the application master.
>>>>>
>>>>> Please note that I could not upload the entire log as the log size was
>>>>> too much and the attached log contains the debug version of the logs for
>>>>> the first few hours. I believe the error started occuring in the first few
>>>>> minutes of launching the app itself as described in the mail above.
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>> Ananth
>>>>>
>>>>>
>>>>>
>>>>> On Tue, May 3, 2016 at 1:53 AM, David Yan <da...@datatorrent.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ananth,
>>>>>>
>>>>>> This error indicates that dtGateway has trouble accessing the app
>>>>>> master.
>>>>>> Can you check the app master log for application_1462007878015_0037?
>>>>>>
>>>>>> David
>>>>>>
>>>>>> On Mon, May 2, 2016 at 4:27 AM, Ananth Gundabattula <
>>>>>> agundabattula@gmail.com> wrote:
>>>>>>
>>>>>>> Hello All,
>>>>>>>
>>>>>>> I am trying to launch my app on Apex/DT 3.3 version running on top
>>>>>>> of CDH 5.7. I tried on DT/Apex 3.2 as well running on CDH 5.7. Both these
>>>>>>> deployments show the following behavior. (We have a requirement to run Apex
>>>>>>> on CDH 5.7 )
>>>>>>>
>>>>>>> I was wondering if you could advise on any workarounds for the
>>>>>>> following issue:
>>>>>>>
>>>>>>> I am not able to see any metrics in the web console when I open the
>>>>>>> "Monitor" section. The application code seems to be working fine as I see
>>>>>>> data flow happening fine as expected. However I see no metrics ( even the
>>>>>>> top level container metrics) .
>>>>>>>
>>>>>>> One more observed issue is that when I click on the container ID
>>>>>>> from the monitor section, the UI crashes. ( In the sense the UI simply does
>>>>>>> not show anything ). I am however able to get some preliminary UI ( barring
>>>>>>> the metrics ) if I click on the container link from the monitor section if
>>>>>>> the application is in the "ACCEPTED" state. The container link does not
>>>>>>> work the moment the app moves from ACCEPTED to Running State.
>>>>>>>
>>>>>>>
>>>>>>> The DtGateway shows the following logs :
>>>>>>>
>>>>>>> 2016-05-02 11:08:48,964 WARN
>>>>>>>> com.datatorrent.stram.client.StramAgent: Caught exception when retrieving
>>>>>>>> web service info for app application_1462007878015_0037
>>>>>>>> org.codehaus.jettison.json.JSONException: A JSONObject text must
>>>>>>>> begin with '{' at character 1 of <html>
>>>>>>>> <head>
>>>>>>>> <meta http-equiv="Content-Type" content="text/html;
>>>>>>>> charset=ISO-8859-1"/>
>>>>>>>> <title>Error 500
>>>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>>>>>>>> </head>
>>>>>>>> <body><h2>HTTP ERROR 500</h2>
>>>>>>>> <p>Problem accessing /ws. Reason:
>>>>>>>> <pre>
>>>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>>>>>>>> by:</h3><pre>java.lang.AbstractMethodError:
>>>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>>>>>>>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>>>>>>>> at
>>>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>>>>>>>> at
>>>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>>>>>>>> at
>>>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>>>>>>>> at
>>>>>>>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>>>>>>>> at
>>>>>>>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>>>>>>>> at
>>>>>>>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>>>>>>>> at
>>>>>>>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>>>>>> at org.mortbay.jetty.Server.handle(Server.java:326)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>>>>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>>>>>>> at
>>>>>>>> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>>>>>> at
>>>>>>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>>>>>>>> at
>>>>>>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>>>>>> </pre>
>>>>>>>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>> <br/>
>>>>>>>>
>>>>>>>> </body>
>>>>>>>> </html>
>>>>>>>>
>>>>>>>> at
>>>>>>>> org.codehaus.jettison.json.JSONTokener.syntaxError(JSONTokener.java:439)
>>>>>>>> at
>>>>>>>> org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:169)
>>>>>>>> at
>>>>>>>> org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:266)
>>>>>>>> at
>>>>>>>> com.datatorrent.stram.client.StramAgent.retrieveWebServicesInfo(StramAgent.java:402)
>>>>>>>> at
>>>>>>>> com.datatorrent.stram.client.StramAgent.getWebServicesInfo(StramAgent.java:181)
>>>>>>>> at
>>>>>>>> com.datatorrent.stram.client.StramAgent.getStramWebURIBuilder(StramAgent.java:205)
>>>>>>>> at
>>>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:275)
>>>>>>>> at
>>>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:305)
>>>>>>>> at
>>>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebGetRequest(StramAgent.java:311)
>>>>>>>> at
>>>>>>>> com.datatorrent.gateway.resources.ws.v2.ApplicationsResource.getApplication(sc:378)
>>>>>>>> at com.datatorrent.gateway.R.F(ud:255)
>>>>>>>> at com.datatorrent.gateway.R.call(ud:157)
>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>>> 2016-05-02 11:08:48,965 WARN com.datatorrent.gateway.u: Cannot get
>>>>>>>> application info for app application_1462007878015_0037. HTTP returns 404
>>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ananth
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: WebService errors in DT 3.3/3.2
Posted by Ananth Gundabattula <ag...@gmail.com>.
Hello David,
Here is the response by querying the URL :
curl -D - http://dwh102.qa1.sac.int.threatmetrix.com:40279/ws/v2/stram
HTTP/1.1 500
javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
Cache-Control: must-revalidate,no-cache,no-store
Date: Tue, 03 May 2016 06:13:01 GMT
Pragma: no-cache
Date: Tue, 03 May 2016 06:13:01 GMT
Pragma: no-cache
Content-Type: text/html; charset=iso-8859-1
Content-Length: 4293
Server: Jetty(6.1.26.cloudera.4)
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 500
javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /ws/v2/stram. Reason:
<pre>
javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
by:</h3><pre>java.lang.AbstractMethodError:
javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
</pre>
<hr /><i><small>Powered by Jetty://</small></i><br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
</body>
</html>
Regards,
Ananth
On Tue, May 3, 2016 at 3:38 PM, David Yan <da...@datatorrent.com> wrote:
> Hi Ananth,
>
> In the app master log, look for a line that says something like:
>
> 10:52:21.182 [main] INFO c.d.stram.StreamingAppMasterService - Setting
> tracking URL to: dwh102.qa1.sac.int.threatmetrix.com:35066
>
> The above is taken from the app master log you sent earlier. Note the URL.
> Then append /ws/v2/stram to it and try to query the URL using curl. For
> example:
>
> curl -D - http://dwh102.qa1.sac.int.threatmetrix.com:35066/ws/v2/stram
>
> Let me know what it returns.
>
> Thanks,
>
> David
>
>
> On Mon, May 2, 2016 at 6:36 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Hello David,
>>
>> Here is the output
>>
>> curl -D - http://RM:8088/proxy/application_1462007878015_0038/ws/v2/stram
>> ( Ran it for another app as the original app was killed and a new app was
>> launched which also shows the exact same behavior)
>>
>> HTTP/1.1 500 Internal Server Error
>>> Cache-Control: must-revalidate,no-cache,no-store
>>> Expires: Tue, 03 May 2016 01:33:27 GMT
>>> Date: Tue, 03 May 2016 01:33:27 GMT
>>> Pragma: no-cache
>>> Expires: Tue, 03 May 2016 01:33:27 GMT
>>> Content-Type: text/html; charset=iso-8859-1
>>> Content-Length: 4293
>>> Server: Jetty(6.1.26.cloudera.4)
>>> Connection: close
>>>
>>> <html>
>>> <head>
>>> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
>>> <title>Error 500
>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>>> </head>
>>> <body><h2>HTTP ERROR 500</h2>
>>> <p>Problem accessing /ws/v2/stram. Reason:
>>>
>>> <pre>
>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>>> by:</h3><pre>java.lang.AbstractMethodError:
>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>>> at
>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>>> at
>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>>> at
>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>>> at
>>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>>> at
>>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>>> at
>>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>>> at
>>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>>> at
>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>> at
>>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>> at
>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>> at
>>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>>> at
>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>> at
>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>> at
>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>> at
>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>> at
>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>> at
>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>> at
>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>> at
>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>> at
>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>>> at
>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>> at
>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>> at
>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>> at org.mortbay.jetty.Server.handle(Server.java:326)
>>> at
>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>> at
>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>> at
>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>>> at
>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>> </pre>
>>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>> <br/>
>>>
>>> </body>
>>> </html>
>>>
>>
>> On Tue, May 3, 2016 at 11:20 AM, David Yan <da...@datatorrent.com> wrote:
>>
>>> Hi Ananth,
>>>
>>> I don't see anything that stands out in the application master log.
>>> Can you send me the output of this command?
>>>
>>> curl -D - http://<rm_ip>:8088/proxy/<your_app_id>/ws/v2/stram
>>>
>>> rm_ip is the IP address of the resource manager in your hadoop cluster.
>>>
>>> David
>>>
>>> On Mon, May 2, 2016 at 6:07 PM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Hello David,
>>>>
>>>> Thanks for taking this thread up.
>>>>
>>>> Here is the log of the application master.
>>>>
>>>> Please note that I could not upload the entire log as the log size was
>>>> too much and the attached log contains the debug version of the logs for
>>>> the first few hours. I believe the error started occuring in the first few
>>>> minutes of launching the app itself as described in the mail above.
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>>
>>>>
>>>> On Tue, May 3, 2016 at 1:53 AM, David Yan <da...@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Hi Ananth,
>>>>>
>>>>> This error indicates that dtGateway has trouble accessing the app
>>>>> master.
>>>>> Can you check the app master log for application_1462007878015_0037?
>>>>>
>>>>> David
>>>>>
>>>>> On Mon, May 2, 2016 at 4:27 AM, Ananth Gundabattula <
>>>>> agundabattula@gmail.com> wrote:
>>>>>
>>>>>> Hello All,
>>>>>>
>>>>>> I am trying to launch my app on Apex/DT 3.3 version running on top of
>>>>>> CDH 5.7. I tried on DT/Apex 3.2 as well running on CDH 5.7. Both these
>>>>>> deployments show the following behavior. (We have a requirement to run Apex
>>>>>> on CDH 5.7 )
>>>>>>
>>>>>> I was wondering if you could advise on any workarounds for the
>>>>>> following issue:
>>>>>>
>>>>>> I am not able to see any metrics in the web console when I open the
>>>>>> "Monitor" section. The application code seems to be working fine as I see
>>>>>> data flow happening fine as expected. However I see no metrics ( even the
>>>>>> top level container metrics) .
>>>>>>
>>>>>> One more observed issue is that when I click on the container ID from
>>>>>> the monitor section, the UI crashes. ( In the sense the UI simply does not
>>>>>> show anything ). I am however able to get some preliminary UI ( barring the
>>>>>> metrics ) if I click on the container link from the monitor section if the
>>>>>> application is in the "ACCEPTED" state. The container link does not work
>>>>>> the moment the app moves from ACCEPTED to Running State.
>>>>>>
>>>>>>
>>>>>> The DtGateway shows the following logs :
>>>>>>
>>>>>> 2016-05-02 11:08:48,964 WARN com.datatorrent.stram.client.StramAgent:
>>>>>>> Caught exception when retrieving web service info for app
>>>>>>> application_1462007878015_0037
>>>>>>> org.codehaus.jettison.json.JSONException: A JSONObject text must
>>>>>>> begin with '{' at character 1 of <html>
>>>>>>> <head>
>>>>>>> <meta http-equiv="Content-Type" content="text/html;
>>>>>>> charset=ISO-8859-1"/>
>>>>>>> <title>Error 500
>>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>>>>>>> </head>
>>>>>>> <body><h2>HTTP ERROR 500</h2>
>>>>>>> <p>Problem accessing /ws. Reason:
>>>>>>> <pre>
>>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>>>>>>> by:</h3><pre>java.lang.AbstractMethodError:
>>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>>>>>>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>>>>>>> at
>>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>>>>>>> at
>>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>>>>>>> at
>>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>>>>>>> at
>>>>>>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>>>>>>> at
>>>>>>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>>>>>>> at
>>>>>>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>>>>>>> at
>>>>>>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>>>>>>> at
>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>> at
>>>>>>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>>>>>> at
>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>> at
>>>>>>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>>>>>>> at
>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>> at
>>>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>>>> at
>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>> at
>>>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>>>> at
>>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>>> at
>>>>>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>>>>>> at
>>>>>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>>>>> at
>>>>>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>>>>> at
>>>>>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>>>>>>> at
>>>>>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>>>>> at
>>>>>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>>>>> at
>>>>>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>>>>> at org.mortbay.jetty.Server.handle(Server.java:326)
>>>>>>> at
>>>>>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>>>>> at
>>>>>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>>>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>>>>>> at
>>>>>>> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>>>>>> at
>>>>>>> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>>>>> at
>>>>>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>>>>>>> at
>>>>>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>>>>> </pre>
>>>>>>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>> <br/>
>>>>>>>
>>>>>>> </body>
>>>>>>> </html>
>>>>>>>
>>>>>>> at
>>>>>>> org.codehaus.jettison.json.JSONTokener.syntaxError(JSONTokener.java:439)
>>>>>>> at
>>>>>>> org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:169)
>>>>>>> at
>>>>>>> org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:266)
>>>>>>> at
>>>>>>> com.datatorrent.stram.client.StramAgent.retrieveWebServicesInfo(StramAgent.java:402)
>>>>>>> at
>>>>>>> com.datatorrent.stram.client.StramAgent.getWebServicesInfo(StramAgent.java:181)
>>>>>>> at
>>>>>>> com.datatorrent.stram.client.StramAgent.getStramWebURIBuilder(StramAgent.java:205)
>>>>>>> at
>>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:275)
>>>>>>> at
>>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:305)
>>>>>>> at
>>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebGetRequest(StramAgent.java:311)
>>>>>>> at
>>>>>>> com.datatorrent.gateway.resources.ws.v2.ApplicationsResource.getApplication(sc:378)
>>>>>>> at com.datatorrent.gateway.R.F(ud:255)
>>>>>>> at com.datatorrent.gateway.R.call(ud:157)
>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>>> at
>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>>>> at
>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>> 2016-05-02 11:08:48,965 WARN com.datatorrent.gateway.u: Cannot get
>>>>>>> application info for app application_1462007878015_0037. HTTP returns 404
>>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Ananth
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: WebService errors in DT 3.3/3.2
Posted by David Yan <da...@datatorrent.com>.
Hi Ananth,
In the app master log, look for a line that says something like:
10:52:21.182 [main] INFO c.d.stram.StreamingAppMasterService - Setting
tracking URL to: dwh102.qa1.sac.int.threatmetrix.com:35066
The above is taken from the app master log you sent earlier. Note the URL.
Then append /ws/v2/stram to it and try to query the URL using curl. For
example:
curl -D - http://dwh102.qa1.sac.int.threatmetrix.com:35066/ws/v2/stram
Let me know what it returns.
Thanks,
David
On Mon, May 2, 2016 at 6:36 PM, Ananth Gundabattula <agundabattula@gmail.com
> wrote:
> Hello David,
>
> Here is the output
>
> curl -D - http://RM:8088/proxy/application_1462007878015_0038/ws/v2/stram
> ( Ran it for another app as the original app was killed and a new app was
> launched which also shows the exact same behavior)
>
> HTTP/1.1 500 Internal Server Error
>> Cache-Control: must-revalidate,no-cache,no-store
>> Expires: Tue, 03 May 2016 01:33:27 GMT
>> Date: Tue, 03 May 2016 01:33:27 GMT
>> Pragma: no-cache
>> Expires: Tue, 03 May 2016 01:33:27 GMT
>> Content-Type: text/html; charset=iso-8859-1
>> Content-Length: 4293
>> Server: Jetty(6.1.26.cloudera.4)
>> Connection: close
>>
>> <html>
>> <head>
>> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
>> <title>Error 500
>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>> </head>
>> <body><h2>HTTP ERROR 500</h2>
>> <p>Problem accessing /ws/v2/stram. Reason:
>>
>> <pre>
>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>> by:</h3><pre>java.lang.AbstractMethodError:
>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>> at
>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>> at
>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>> at
>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>> at
>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>> at
>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>> at
>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>> at
>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>> at
>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>> at
>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>> at
>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>> at
>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>> at
>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>> at
>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>> at org.mortbay.jetty.Server.handle(Server.java:326)
>> at
>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>> at
>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>> at
>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>> at
>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>> </pre>
>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>>
>> </body>
>> </html>
>>
>
> On Tue, May 3, 2016 at 11:20 AM, David Yan <da...@datatorrent.com> wrote:
>
>> Hi Ananth,
>>
>> I don't see anything that stands out in the application master log.
>> Can you send me the output of this command?
>>
>> curl -D - http://<rm_ip>:8088/proxy/<your_app_id>/ws/v2/stram
>>
>> rm_ip is the IP address of the resource manager in your hadoop cluster.
>>
>> David
>>
>> On Mon, May 2, 2016 at 6:07 PM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> Hello David,
>>>
>>> Thanks for taking this thread up.
>>>
>>> Here is the log of the application master.
>>>
>>> Please note that I could not upload the entire log as the log size was
>>> too much and the attached log contains the debug version of the logs for
>>> the first few hours. I believe the error started occuring in the first few
>>> minutes of launching the app itself as described in the mail above.
>>>
>>>
>>>
>>> Regards,
>>> Ananth
>>>
>>>
>>>
>>> On Tue, May 3, 2016 at 1:53 AM, David Yan <da...@datatorrent.com> wrote:
>>>
>>>> Hi Ananth,
>>>>
>>>> This error indicates that dtGateway has trouble accessing the app
>>>> master.
>>>> Can you check the app master log for application_1462007878015_0037?
>>>>
>>>> David
>>>>
>>>> On Mon, May 2, 2016 at 4:27 AM, Ananth Gundabattula <
>>>> agundabattula@gmail.com> wrote:
>>>>
>>>>> Hello All,
>>>>>
>>>>> I am trying to launch my app on Apex/DT 3.3 version running on top of
>>>>> CDH 5.7. I tried on DT/Apex 3.2 as well running on CDH 5.7. Both these
>>>>> deployments show the following behavior. (We have a requirement to run Apex
>>>>> on CDH 5.7 )
>>>>>
>>>>> I was wondering if you could advise on any workarounds for the
>>>>> following issue:
>>>>>
>>>>> I am not able to see any metrics in the web console when I open the
>>>>> "Monitor" section. The application code seems to be working fine as I see
>>>>> data flow happening fine as expected. However I see no metrics ( even the
>>>>> top level container metrics) .
>>>>>
>>>>> One more observed issue is that when I click on the container ID from
>>>>> the monitor section, the UI crashes. ( In the sense the UI simply does not
>>>>> show anything ). I am however able to get some preliminary UI ( barring the
>>>>> metrics ) if I click on the container link from the monitor section if the
>>>>> application is in the "ACCEPTED" state. The container link does not work
>>>>> the moment the app moves from ACCEPTED to Running State.
>>>>>
>>>>>
>>>>> The DtGateway shows the following logs :
>>>>>
>>>>> 2016-05-02 11:08:48,964 WARN com.datatorrent.stram.client.StramAgent:
>>>>>> Caught exception when retrieving web service info for app
>>>>>> application_1462007878015_0037
>>>>>> org.codehaus.jettison.json.JSONException: A JSONObject text must
>>>>>> begin with '{' at character 1 of <html>
>>>>>> <head>
>>>>>> <meta http-equiv="Content-Type" content="text/html;
>>>>>> charset=ISO-8859-1"/>
>>>>>> <title>Error 500
>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>>>>>> </head>
>>>>>> <body><h2>HTTP ERROR 500</h2>
>>>>>> <p>Problem accessing /ws. Reason:
>>>>>> <pre>
>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>>>>>> by:</h3><pre>java.lang.AbstractMethodError:
>>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>>>>>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>>>>>> at
>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>>>>>> at
>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>>>>>> at
>>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>>>>>> at
>>>>>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>>>>>> at
>>>>>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>>>>>> at
>>>>>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>>>>>> at
>>>>>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>>>>>> at
>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>> at
>>>>>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>>>>> at
>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>> at
>>>>>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>>>>>> at
>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>> at
>>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>>> at
>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>> at
>>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>>> at
>>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>>> at
>>>>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>>>>> at
>>>>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>>>> at
>>>>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>>>> at
>>>>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>>>>>> at
>>>>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>>>> at
>>>>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>>>> at
>>>>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>>>> at org.mortbay.jetty.Server.handle(Server.java:326)
>>>>>> at
>>>>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>>>> at
>>>>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>>>>> at
>>>>>> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>>>>> at
>>>>>> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>>>> at
>>>>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>>>>>> at
>>>>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>>>> </pre>
>>>>>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>> <br/>
>>>>>>
>>>>>> </body>
>>>>>> </html>
>>>>>>
>>>>>> at
>>>>>> org.codehaus.jettison.json.JSONTokener.syntaxError(JSONTokener.java:439)
>>>>>> at
>>>>>> org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:169)
>>>>>> at
>>>>>> org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:266)
>>>>>> at
>>>>>> com.datatorrent.stram.client.StramAgent.retrieveWebServicesInfo(StramAgent.java:402)
>>>>>> at
>>>>>> com.datatorrent.stram.client.StramAgent.getWebServicesInfo(StramAgent.java:181)
>>>>>> at
>>>>>> com.datatorrent.stram.client.StramAgent.getStramWebURIBuilder(StramAgent.java:205)
>>>>>> at
>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:275)
>>>>>> at
>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:305)
>>>>>> at
>>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebGetRequest(StramAgent.java:311)
>>>>>> at
>>>>>> com.datatorrent.gateway.resources.ws.v2.ApplicationsResource.getApplication(sc:378)
>>>>>> at com.datatorrent.gateway.R.F(ud:255)
>>>>>> at com.datatorrent.gateway.R.call(ud:157)
>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>> 2016-05-02 11:08:48,965 WARN com.datatorrent.gateway.u: Cannot get
>>>>>> application info for app application_1462007878015_0037. HTTP returns 404
>>>>>>
>>>>>
>>>>> Regards,
>>>>> Ananth
>>>>>
>>>>
>>>>
>>>
>>
>
Re: WebService errors in DT 3.3/3.2
Posted by Ananth Gundabattula <ag...@gmail.com>.
Hello David,
Here is the output
curl -D - http://RM:8088/proxy/application_1462007878015_0038/ws/v2/stram (
Ran it for another app as the original app was killed and a new app was
launched which also shows the exact same behavior)
HTTP/1.1 500 Internal Server Error
> Cache-Control: must-revalidate,no-cache,no-store
> Expires: Tue, 03 May 2016 01:33:27 GMT
> Date: Tue, 03 May 2016 01:33:27 GMT
> Pragma: no-cache
> Expires: Tue, 03 May 2016 01:33:27 GMT
> Content-Type: text/html; charset=iso-8859-1
> Content-Length: 4293
> Server: Jetty(6.1.26.cloudera.4)
> Connection: close
>
> <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
> <title>Error 500
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
> </head>
> <body><h2>HTTP ERROR 500</h2>
> <p>Problem accessing /ws/v2/stram. Reason:
> <pre>
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
> by:</h3><pre>java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
> at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
> at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> </pre>
> <hr /><i><small>Powered by Jetty://</small></i><br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
>
> </body>
> </html>
>
On Tue, May 3, 2016 at 11:20 AM, David Yan <da...@datatorrent.com> wrote:
> Hi Ananth,
>
> I don't see anything that stands out in the application master log.
> Can you send me the output of this command?
>
> curl -D - http://<rm_ip>:8088/proxy/<your_app_id>/ws/v2/stram
>
> rm_ip is the IP address of the resource manager in your hadoop cluster.
>
> David
>
> On Mon, May 2, 2016 at 6:07 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Hello David,
>>
>> Thanks for taking this thread up.
>>
>> Here is the log of the application master.
>>
>> Please note that I could not upload the entire log as the log size was
>> too much and the attached log contains the debug version of the logs for
>> the first few hours. I believe the error started occuring in the first few
>> minutes of launching the app itself as described in the mail above.
>>
>>
>>
>> Regards,
>> Ananth
>>
>>
>>
>> On Tue, May 3, 2016 at 1:53 AM, David Yan <da...@datatorrent.com> wrote:
>>
>>> Hi Ananth,
>>>
>>> This error indicates that dtGateway has trouble accessing the app master.
>>> Can you check the app master log for application_1462007878015_0037?
>>>
>>> David
>>>
>>> On Mon, May 2, 2016 at 4:27 AM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Hello All,
>>>>
>>>> I am trying to launch my app on Apex/DT 3.3 version running on top of
>>>> CDH 5.7. I tried on DT/Apex 3.2 as well running on CDH 5.7. Both these
>>>> deployments show the following behavior. (We have a requirement to run Apex
>>>> on CDH 5.7 )
>>>>
>>>> I was wondering if you could advise on any workarounds for the
>>>> following issue:
>>>>
>>>> I am not able to see any metrics in the web console when I open the
>>>> "Monitor" section. The application code seems to be working fine as I see
>>>> data flow happening fine as expected. However I see no metrics ( even the
>>>> top level container metrics) .
>>>>
>>>> One more observed issue is that when I click on the container ID from
>>>> the monitor section, the UI crashes. ( In the sense the UI simply does not
>>>> show anything ). I am however able to get some preliminary UI ( barring the
>>>> metrics ) if I click on the container link from the monitor section if the
>>>> application is in the "ACCEPTED" state. The container link does not work
>>>> the moment the app moves from ACCEPTED to Running State.
>>>>
>>>>
>>>> The DtGateway shows the following logs :
>>>>
>>>> 2016-05-02 11:08:48,964 WARN com.datatorrent.stram.client.StramAgent:
>>>>> Caught exception when retrieving web service info for app
>>>>> application_1462007878015_0037
>>>>> org.codehaus.jettison.json.JSONException: A JSONObject text must begin
>>>>> with '{' at character 1 of <html>
>>>>> <head>
>>>>> <meta http-equiv="Content-Type" content="text/html;
>>>>> charset=ISO-8859-1"/>
>>>>> <title>Error 500
>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>>>>> </head>
>>>>> <body><h2>HTTP ERROR 500</h2>
>>>>> <p>Problem accessing /ws. Reason:
>>>>> <pre>
>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>>>>> by:</h3><pre>java.lang.AbstractMethodError:
>>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>>>>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>>>>> at
>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>>>>> at
>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>>>>> at
>>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>>>>> at
>>>>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>>>>> at
>>>>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>>>>> at
>>>>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>>>>> at
>>>>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>> at
>>>>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>> at
>>>>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>> at
>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>> at
>>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>>> at
>>>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>>>> at
>>>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>>> at
>>>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>>> at
>>>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>>>>> at
>>>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>>> at
>>>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>>> at
>>>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>>> at org.mortbay.jetty.Server.handle(Server.java:326)
>>>>> at
>>>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>>> at
>>>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>>> at
>>>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>>>>> at
>>>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>>> </pre>
>>>>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>> <br/>
>>>>>
>>>>> </body>
>>>>> </html>
>>>>>
>>>>> at
>>>>> org.codehaus.jettison.json.JSONTokener.syntaxError(JSONTokener.java:439)
>>>>> at
>>>>> org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:169)
>>>>> at
>>>>> org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:266)
>>>>> at
>>>>> com.datatorrent.stram.client.StramAgent.retrieveWebServicesInfo(StramAgent.java:402)
>>>>> at
>>>>> com.datatorrent.stram.client.StramAgent.getWebServicesInfo(StramAgent.java:181)
>>>>> at
>>>>> com.datatorrent.stram.client.StramAgent.getStramWebURIBuilder(StramAgent.java:205)
>>>>> at
>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:275)
>>>>> at
>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:305)
>>>>> at
>>>>> com.datatorrent.stram.client.StramAgent.issueStramWebGetRequest(StramAgent.java:311)
>>>>> at
>>>>> com.datatorrent.gateway.resources.ws.v2.ApplicationsResource.getApplication(sc:378)
>>>>> at com.datatorrent.gateway.R.F(ud:255)
>>>>> at com.datatorrent.gateway.R.call(ud:157)
>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>> 2016-05-02 11:08:48,965 WARN com.datatorrent.gateway.u: Cannot get
>>>>> application info for app application_1462007878015_0037. HTTP returns 404
>>>>>
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>
>>>
>>
>
Re: WebService errors in DT 3.3/3.2
Posted by David Yan <da...@datatorrent.com>.
Hi Ananth,
I don't see anything that stands out in the application master log.
Can you send me the output of this command?
curl -D - http://<rm_ip>:8088/proxy/<your_app_id>/ws/v2/stram
rm_ip is the IP address of the resource manager in your hadoop cluster.
David
On Mon, May 2, 2016 at 6:07 PM, Ananth Gundabattula <agundabattula@gmail.com
> wrote:
> Hello David,
>
> Thanks for taking this thread up.
>
> Here is the log of the application master.
>
> Please note that I could not upload the entire log as the log size was too
> much and the attached log contains the debug version of the logs for the
> first few hours. I believe the error started occuring in the first few
> minutes of launching the app itself as described in the mail above.
>
>
>
> Regards,
> Ananth
>
>
>
> On Tue, May 3, 2016 at 1:53 AM, David Yan <da...@datatorrent.com> wrote:
>
>> Hi Ananth,
>>
>> This error indicates that dtGateway has trouble accessing the app master.
>> Can you check the app master log for application_1462007878015_0037?
>>
>> David
>>
>> On Mon, May 2, 2016 at 4:27 AM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> Hello All,
>>>
>>> I am trying to launch my app on Apex/DT 3.3 version running on top of
>>> CDH 5.7. I tried on DT/Apex 3.2 as well running on CDH 5.7. Both these
>>> deployments show the following behavior. (We have a requirement to run Apex
>>> on CDH 5.7 )
>>>
>>> I was wondering if you could advise on any workarounds for the following
>>> issue:
>>>
>>> I am not able to see any metrics in the web console when I open the
>>> "Monitor" section. The application code seems to be working fine as I see
>>> data flow happening fine as expected. However I see no metrics ( even the
>>> top level container metrics) .
>>>
>>> One more observed issue is that when I click on the container ID from
>>> the monitor section, the UI crashes. ( In the sense the UI simply does not
>>> show anything ). I am however able to get some preliminary UI ( barring the
>>> metrics ) if I click on the container link from the monitor section if the
>>> application is in the "ACCEPTED" state. The container link does not work
>>> the moment the app moves from ACCEPTED to Running State.
>>>
>>>
>>> The DtGateway shows the following logs :
>>>
>>> 2016-05-02 11:08:48,964 WARN com.datatorrent.stram.client.StramAgent:
>>>> Caught exception when retrieving web service info for app
>>>> application_1462007878015_0037
>>>> org.codehaus.jettison.json.JSONException: A JSONObject text must begin
>>>> with '{' at character 1 of <html>
>>>> <head>
>>>> <meta http-equiv="Content-Type" content="text/html;
>>>> charset=ISO-8859-1"/>
>>>> <title>Error 500
>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>>>> </head>
>>>> <body><h2>HTTP ERROR 500</h2>
>>>> <p>Problem accessing /ws. Reason:
>>>> <pre>
>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>>>> by:</h3><pre>java.lang.AbstractMethodError:
>>>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>>>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>>>> at
>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>>>> at
>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>>>> at
>>>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>>>> at
>>>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>>>> at
>>>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>>>> at
>>>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>>>> at
>>>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> at
>>>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> at
>>>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> at
>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> at
>>>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>> at
>>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>>> at
>>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>> at
>>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>> at
>>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>>>> at
>>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>> at
>>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>> at
>>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>> at org.mortbay.jetty.Server.handle(Server.java:326)
>>>> at
>>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>> at
>>>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>> at
>>>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>>>> at
>>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>>> </pre>
>>>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>> <br/>
>>>>
>>>> </body>
>>>> </html>
>>>>
>>>> at
>>>> org.codehaus.jettison.json.JSONTokener.syntaxError(JSONTokener.java:439)
>>>> at org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:169)
>>>> at org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:266)
>>>> at
>>>> com.datatorrent.stram.client.StramAgent.retrieveWebServicesInfo(StramAgent.java:402)
>>>> at
>>>> com.datatorrent.stram.client.StramAgent.getWebServicesInfo(StramAgent.java:181)
>>>> at
>>>> com.datatorrent.stram.client.StramAgent.getStramWebURIBuilder(StramAgent.java:205)
>>>> at
>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:275)
>>>> at
>>>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:305)
>>>> at
>>>> com.datatorrent.stram.client.StramAgent.issueStramWebGetRequest(StramAgent.java:311)
>>>> at
>>>> com.datatorrent.gateway.resources.ws.v2.ApplicationsResource.getApplication(sc:378)
>>>> at com.datatorrent.gateway.R.F(ud:255)
>>>> at com.datatorrent.gateway.R.call(ud:157)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> 2016-05-02 11:08:48,965 WARN com.datatorrent.gateway.u: Cannot get
>>>> application info for app application_1462007878015_0037. HTTP returns 404
>>>>
>>>
>>> Regards,
>>> Ananth
>>>
>>
>>
>
Re: WebService errors in DT 3.3/3.2
Posted by David Yan <da...@datatorrent.com>.
Hi Ananth,
This error indicates that dtGateway has trouble accessing the app master.
Can you check the app master log for application_1462007878015_0037?
David
On Mon, May 2, 2016 at 4:27 AM, Ananth Gundabattula <agundabattula@gmail.com
> wrote:
> Hello All,
>
> I am trying to launch my app on Apex/DT 3.3 version running on top of CDH
> 5.7. I tried on DT/Apex 3.2 as well running on CDH 5.7. Both these
> deployments show the following behavior. (We have a requirement to run Apex
> on CDH 5.7 )
>
> I was wondering if you could advise on any workarounds for the following
> issue:
>
> I am not able to see any metrics in the web console when I open the
> "Monitor" section. The application code seems to be working fine as I see
> data flow happening fine as expected. However I see no metrics ( even the
> top level container metrics) .
>
> One more observed issue is that when I click on the container ID from the
> monitor section, the UI crashes. ( In the sense the UI simply does not show
> anything ). I am however able to get some preliminary UI ( barring the
> metrics ) if I click on the container link from the monitor section if the
> application is in the "ACCEPTED" state. The container link does not work
> the moment the app moves from ACCEPTED to Running State.
>
>
> The DtGateway shows the following logs :
>
> 2016-05-02 11:08:48,964 WARN com.datatorrent.stram.client.StramAgent:
>> Caught exception when retrieving web service info for app
>> application_1462007878015_0037
>> org.codehaus.jettison.json.JSONException: A JSONObject text must begin
>> with '{' at character 1 of <html>
>> <head>
>> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
>> <title>Error 500
>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</title>
>> </head>
>> <body><h2>HTTP ERROR 500</h2>
>> <p>Problem accessing /ws. Reason:
>> <pre>
>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;</pre></p><h3>Caused
>> by:</h3><pre>java.lang.AbstractMethodError:
>> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
>> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)
>> at
>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:870)
>> at
>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>> at
>> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>> at
>> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>> at
>> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>> at
>> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>> at
>> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1286)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>> at
>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at
>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>> at
>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>> at
>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>> at
>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>> at
>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>> at
>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>> at
>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>> at org.mortbay.jetty.Server.handle(Server.java:326)
>> at
>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>> at
>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>> at
>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>> at
>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>> </pre>
>> <hr /><i><small>Powered by Jetty://</small></i><br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>> <br/>
>>
>> </body>
>> </html>
>>
>> at
>> org.codehaus.jettison.json.JSONTokener.syntaxError(JSONTokener.java:439)
>> at org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:169)
>> at org.codehaus.jettison.json.JSONObject.<init>(JSONObject.java:266)
>> at
>> com.datatorrent.stram.client.StramAgent.retrieveWebServicesInfo(StramAgent.java:402)
>> at
>> com.datatorrent.stram.client.StramAgent.getWebServicesInfo(StramAgent.java:181)
>> at
>> com.datatorrent.stram.client.StramAgent.getStramWebURIBuilder(StramAgent.java:205)
>> at
>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:275)
>> at
>> com.datatorrent.stram.client.StramAgent.issueStramWebRequest(StramAgent.java:305)
>> at
>> com.datatorrent.stram.client.StramAgent.issueStramWebGetRequest(StramAgent.java:311)
>> at
>> com.datatorrent.gateway.resources.ws.v2.ApplicationsResource.getApplication(sc:378)
>> at com.datatorrent.gateway.R.F(ud:255)
>> at com.datatorrent.gateway.R.call(ud:157)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> 2016-05-02 11:08:48,965 WARN com.datatorrent.gateway.u: Cannot get
>> application info for app application_1462007878015_0037. HTTP returns 404
>>
>
> Regards,
> Ananth
>