You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@apex.apache.org by Sandeep Deshmukh <sa...@datatorrent.com> on 2016/06/01 04:09:15 UTC
Re: Question about running yahoofinance example in apex-malhar
Hi Junguk,
Could you please give us some details:
1. Hadoop cluster configuration: no of machines and their configuration
(memory in particular)
2. Version of Hadoop and Apex
Regards,
Sandeep
On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <jm...@gmail.com> wrote:
> Hi,
>
> I am new to apex.
> I tried to run demo examples in incubator-apex-malhar.
>
> When I ran yahoofinance example, I think it was stuck.
> Log file said
> 2016-05-31 20:34:00,628 INFO
> com.datatorrent.bufferserver.server.Server: Server started listening
> at /0:0:0:0:0:0:0:0:45546
> 2016-05-31 20:34:01,668 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
>
> I think binding IP address has a problem.
>
> Does anyone help me?
>
>
> Thanks,
> Junguk
>
Re: Question about running yahoofinance example in apex-malhar
Posted by dbs <db...@softiron.co.uk>.
I'm having the same problem with the yahoofinance demo.
It never gets past "waiting for pending...".
Also, I see repeated msgs in dt.log on all the containers saying
removing container --> removed container --> requested container
every 30 seconds. (see below)
Aside from dt.log, there's no other output. All the container stdout/stderr
files are empty.
Are these normal for this demo?
Is it possible there's a configuration missing?
Here's an example of the repeated message in the app master dt.log msgs:
2016-09-15 16:07:39,394 INFO com.datatorrent.stram.ResourceRequestHandler:
Strict anti-affinity = [] for container with operators
PTOperator[id=10,name=priceSMAConsole]
2016-09-15 16:07:39,394 INFO com.datatorrent.stram.ResourceRequestHandler:
Found host null
2016-09-15 16:07:39,394 INFO com.datatorrent.stram.ResourceRequestHandler:
Strict anti-affinity = [] for container with operators
PTOperator[id=6,name=DailyVolume]
2016-09-15 16:07:39,394 INFO com.datatorrent.stram.ResourceRequestHandler:
Found host null
2016-09-15 16:07:39,394 INFO
com.datatorrent.stram.StreamingAppMasterService: Removing container
request: [Capability[<memory:4096, vCores:1>]Priority[9],
Capability[<memory:4096, vCores:1>]Priority[8]]
2016-09-15 16:07:39,394 INFO
com.datatorrent.stram.StreamingAppMasterService: Removed container:
Capability[<memory:4096, vCores:1>]Priority[9]
2016-09-15 16:07:39,394 INFO
com.datatorrent.stram.StreamingAppMasterService: Removed container:
Capability[<memory:4096, vCores:1>]Priority[8]
2016-09-15 16:07:39,394 INFO
com.datatorrent.stram.StreamingAppMasterService: Asking RM for containers:
[Capability[<memory:4096, vCores:1>]Priority[9], Capability[<memory:4096,
vCores:1>]Priority[8]]
2016-09-15 16:07:39,394 INFO
com.datatorrent.stram.StreamingAppMasterService: Requested container:
Capability[<memory:4096, vCores:1>]Priority[9] on host: [null]
2016-09-15 16:07:39,394 INFO
com.datatorrent.stram.StreamingAppMasterService: Requested container:
Capability[<memory:4096, vCores:1>]Priority[8] on host: [null]
...repeats...
Junguk Cho wrote
> Hi, Sandeep.
>
> Thank you for reply.
> No, I only got
>
> 2016-05-31 20:34:00,628 INFO
> com.datatorrent.bufferserver.server.Server: Server started listening
> at /0:0:0:0:0:0:0:0:45546
> 2016-05-31 20:34:01,668 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-31 20:34:02,172 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-31 20:34:02,676 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-31 20:34:03,179 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
>
> I can continuously see only this message.
> Do you have any guest why it did not show "Deploy request"?
>
> When I ran wordcount example, I can see "Deploy request".
> I attached screenshot of web UI.
>
> Thanks,
> Junguk
>
> ....
--
View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/Question-about-running-yahoofinance-example-in-apex-malhar-tp269p926.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.
Re: Question about running yahoofinance example in apex-malhar
Posted by Junguk Cho <jm...@gmail.com>.
Hi, Sandeep.
Thank you for reply.
No, I only got
2016-05-31 20:34:00,628 INFO
com.datatorrent.bufferserver.server.Server: Server started listening
at /0:0:0:0:0:0:0:0:45546
2016-05-31 20:34:01,668 INFO
com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
request.
2016-05-31 20:34:02,172 INFO
com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
request.
2016-05-31 20:34:02,676 INFO
com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
request.
2016-05-31 20:34:03,179 INFO
com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
request.
I can continuously see only this message.
Do you have any guest why it did not show "Deploy request"?
When I ran wordcount example, I can see "Deploy request".
I attached screenshot of web UI.
Thanks,
Junguk
2016-06-01 1:10 GMT-06:00 Sandeep Deshmukh <sa...@datatorrent.com>:
> Thanks for details.
>
> Typically the "Waiting for pending status" is followed by "Deploy request"
> as highlighted below. Are you getting the highlighted line in your log? If
> not, the operators is not getting deployed correctly. Please let us know.
>
>
> 2016-05-23 14:46:51,649 DEBUG
> com.datatorrent.stram.engine.StreamingContainer: buffer server memory 512
> 2016-05-23 14:46:51,655 INFO
> com.datatorrent.bufferserver.storage.DiskStorage: using
> /data/hadoop-2.7.2/tmp/nm-local-dir/usercache/sandeep/appcache/application_1463980780220_0003/container_1463980780220_0003_01_000002/tmp
> as the basepath for spooling.
> 2016-05-23 14:46:51,663 INFO com.datatorrent.bufferserver.server.Server:
> Server started listening at /0:0:0:0:0:0:0:0:42358
> 2016-05-23 14:46:51,665 DEBUG
> com.datatorrent.stram.engine.StreamingContainer: Buffer server started:
> /0:0:0:0:0:0:0:0:42358
> 2016-05-23 14:46:51,700 DEBUG
> com.datatorrent.stram.engine.StreamingContainer: Entering heartbeat loop
> (interval is 1000 ms)
> 2016-05-23 14:46:52,750 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-23 14:46:53,255 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-23 14:46:56,287 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-23 14:46:56,791 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-23 14:46:57,295 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-23 14:46:57,809 INFO
> com.datatorrent.stram.engine.StreamingContainer: Deploy request:
> [OperatorDeployInfo[id=1,name=Input,type=INPUT,checkpoint={ffffffffffffffff,
> 0,
> 0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=enrich,bufferServer=goofy]]]]
> 2016-05-23 14:46:57,811 DEBUG
> com.datatorrent.stram.engine.StreamingContainer: Restoring operator 1 to
> checkpoint ffffffffffffffff stateless=false.
> 2016-05-23 14:46:57,876 DEBUG com.datatorrent.stram.engine.Node: type=INPUT,
> operator class=class com.datatorrent.InputFileReader
> 2016-05-23 14:46:57,889 DEBUG
> com.datatorrent.stram.engine.StreamingContainer: Marking operator 1 as
> deployed.
>
>
> Regards,
> Sandeep
>
> Regards,
> Sandeep
>
> On Wed, Jun 1, 2016 at 12:29 PM, Junguk Cho <jm...@gmail.com> wrote:
>>
>> Hi,
>>
>> 1. I used only one machine and use docker image to run it.
>> When I access to ip:8088, it shows I use 8 GB memory.
>> (Is there a configuration file I can check?)
>>
>> 2. Hadoop version : 2.7.1
>> I tested both Apex 3.2.2 and Apex 3.3.
>>
>> Thanks,
>> Junguk
>>
>>
>> 2016-05-31 22:09 GMT-06:00 Sandeep Deshmukh <sa...@datatorrent.com>:
>> > Hi Junguk,
>> >
>> > Could you please give us some details:
>> >
>> > Hadoop cluster configuration: no of machines and their configuration
>> > (memory
>> > in particular)
>> > Version of Hadoop and Apex
>> >
>> > Regards,
>> > Sandeep
>> >
>> > On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <jm...@gmail.com> wrote:
>> >>
>> >> Hi,
>> >>
>> >> I am new to apex.
>> >> I tried to run demo examples in incubator-apex-malhar.
>> >>
>> >> When I ran yahoofinance example, I think it was stuck.
>> >> Log file said
>> >> 2016-05-31 20:34:00,628 INFO
>> >> com.datatorrent.bufferserver.server.Server: Server started listening
>> >> at /0:0:0:0:0:0:0:0:45546
>> >> 2016-05-31 20:34:01,668 INFO
>> >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>> >> request.
>> >>
>> >> I think binding IP address has a problem.
>> >>
>> >> Does anyone help me?
>> >>
>> >>
>> >> Thanks,
>> >> Junguk
>> >
>> >
>
>
Re: Question about running yahoofinance example in apex-malhar
Posted by Tushar Gosavi <tu...@datatorrent.com>.
Do you see any exception logged in files inside failed/killed
container. Most probably you will find log in logs/userlogs directory
inside hadoop installation directory, or in $YARN_LOG_DIR.
Do check master container log for any out of memory errors, the
master container id will be container_1464766676472_0003_01_*001
Regards,
- Tushar.
On Wed, Jun 1, 2016 at 1:38 PM, Junguk Cho <jm...@gmail.com> wrote:
> Hi.
>
> Sorry for many emails.
> However, I think something is wrong.
> It continuously created log files
> container_1464766676472_0003_01_xxxx, which meant java process keep
> dead.
>
> Is it right?
> Was it supposed to do?
>
> Thanks in advance.
> Junguk
>
> 2016-06-01 1:57 GMT-06:00 Junguk Cho <jm...@gmail.com>:
>> Hi, Tushar.
>>
>> Thanks.
>> Yes. It worked.
>>
>> I have one more question.
>> When I access to hadoop web UI and there is menu "ApplicationMaster"
>> in Tracking UI menu.
>> When I click this, it gave me error.
>>
>> Sorry, got error 404
>>
>> Please consult RFC 2616 for meanings of the error code.
>>
>> Error Details
>>
>> org.apache.hadoop.yarn.webapp.WebAppException: /: controller for
>> default not found
>> at org.apache.hadoop.yarn.webapp.Router.resolveDefault(Router.java:232)
>> at org.apache.hadoop.yarn.webapp.Router.resolve(Router.java:140)
>> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:134)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>> at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
>> at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
>> at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
>> at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
>> at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
>> at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>> at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>> at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>> at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>> at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1243)
>> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>> at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>> at org.mortbay.jetty.Server.handle(Server.java:326)
>> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>> at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>
>> The URL is "http://<IP address>:8088/proxy/application_1464766676472_0002/.
>>
>> With this menu, what we can see?
>>
>>
>> Thanks,
>> Junguk
>>
>> 2016-06-01 1:24 GMT-06:00 Tushar Gosavi <tu...@datatorrent.com>:
>>> Hi Junguk,
>>>
>>> Please check your yarn configuration settings. I normally set
>>> minimum-allocation-mb to 256MB for single node install, you can add
>>> following lines to your yarn-site.xml and restart the yarn.
>>>
>>> <property>
>>> <name>yarn.scheduler.minimum-allocation-mb</name>
>>> <value>256</value>
>>> </property>
>>>
>>> I am able to run YahooFinanceDemo app with above setting on my laptop
>>> with 5GB memory usage.
>>>
>>> Regards,
>>> -Tushar.
>>>
>>> On Wed, Jun 1, 2016 at 12:40 PM, Sandeep Deshmukh
>>> <sa...@datatorrent.com> wrote:
>>>> Thanks for details.
>>>>
>>>> Typically the "Waiting for pending status" is followed by "Deploy request"
>>>> as highlighted below. Are you getting the highlighted line in your log? If
>>>> not, the operators is not getting deployed correctly. Please let us know.
>>>>
>>>>
>>>> 2016-05-23 14:46:51,649 DEBUG
>>>> com.datatorrent.stram.engine.StreamingContainer: buffer server memory 512
>>>> 2016-05-23 14:46:51,655 INFO
>>>> com.datatorrent.bufferserver.storage.DiskStorage: using
>>>> /data/hadoop-2.7.2/tmp/nm-local-dir/usercache/sandeep/appcache/application_1463980780220_0003/container_1463980780220_0003_01_000002/tmp
>>>> as the basepath for spooling.
>>>> 2016-05-23 14:46:51,663 INFO com.datatorrent.bufferserver.server.Server:
>>>> Server started listening at /0:0:0:0:0:0:0:0:42358
>>>> 2016-05-23 14:46:51,665 DEBUG
>>>> com.datatorrent.stram.engine.StreamingContainer: Buffer server started:
>>>> /0:0:0:0:0:0:0:0:42358
>>>> 2016-05-23 14:46:51,700 DEBUG
>>>> com.datatorrent.stram.engine.StreamingContainer: Entering heartbeat loop
>>>> (interval is 1000 ms)
>>>> 2016-05-23 14:46:52,750 INFO
>>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>>> request.
>>>> 2016-05-23 14:46:53,255 INFO
>>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>>> request.
>>>> 2016-05-23 14:46:56,287 INFO
>>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>>> request.
>>>> 2016-05-23 14:46:56,791 INFO
>>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>>> request.
>>>> 2016-05-23 14:46:57,295 INFO
>>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>>> request.
>>>> 2016-05-23 14:46:57,809 INFO
>>>> com.datatorrent.stram.engine.StreamingContainer: Deploy request:
>>>> [OperatorDeployInfo[id=1,name=Input,type=INPUT,checkpoint={ffffffffffffffff,
>>>> 0,
>>>> 0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=enrich,bufferServer=goofy]]]]
>>>> 2016-05-23 14:46:57,811 DEBUG
>>>> com.datatorrent.stram.engine.StreamingContainer: Restoring operator 1 to
>>>> checkpoint ffffffffffffffff stateless=false.
>>>> 2016-05-23 14:46:57,876 DEBUG com.datatorrent.stram.engine.Node: type=INPUT,
>>>> operator class=class com.datatorrent.InputFileReader
>>>> 2016-05-23 14:46:57,889 DEBUG
>>>> com.datatorrent.stram.engine.StreamingContainer: Marking operator 1 as
>>>> deployed.
>>>>
>>>>
>>>> Regards,
>>>> Sandeep
>>>>
>>>> Regards,
>>>> Sandeep
>>>>
>>>> On Wed, Jun 1, 2016 at 12:29 PM, Junguk Cho <jm...@gmail.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> 1. I used only one machine and use docker image to run it.
>>>>> When I access to ip:8088, it shows I use 8 GB memory.
>>>>> (Is there a configuration file I can check?)
>>>>>
>>>>> 2. Hadoop version : 2.7.1
>>>>> I tested both Apex 3.2.2 and Apex 3.3.
>>>>>
>>>>> Thanks,
>>>>> Junguk
>>>>>
>>>>>
>>>>> 2016-05-31 22:09 GMT-06:00 Sandeep Deshmukh <sa...@datatorrent.com>:
>>>>> > Hi Junguk,
>>>>> >
>>>>> > Could you please give us some details:
>>>>> >
>>>>> > Hadoop cluster configuration: no of machines and their configuration
>>>>> > (memory
>>>>> > in particular)
>>>>> > Version of Hadoop and Apex
>>>>> >
>>>>> > Regards,
>>>>> > Sandeep
>>>>> >
>>>>> > On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <jm...@gmail.com> wrote:
>>>>> >>
>>>>> >> Hi,
>>>>> >>
>>>>> >> I am new to apex.
>>>>> >> I tried to run demo examples in incubator-apex-malhar.
>>>>> >>
>>>>> >> When I ran yahoofinance example, I think it was stuck.
>>>>> >> Log file said
>>>>> >> 2016-05-31 20:34:00,628 INFO
>>>>> >> com.datatorrent.bufferserver.server.Server: Server started listening
>>>>> >> at /0:0:0:0:0:0:0:0:45546
>>>>> >> 2016-05-31 20:34:01,668 INFO
>>>>> >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>>>> >> request.
>>>>> >>
>>>>> >> I think binding IP address has a problem.
>>>>> >>
>>>>> >> Does anyone help me?
>>>>> >>
>>>>> >>
>>>>> >> Thanks,
>>>>> >> Junguk
>>>>> >
>>>>> >
>>>>
>>>>
Re: Question about running yahoofinance example in apex-malhar
Posted by Junguk Cho <jm...@gmail.com>.
Hi.
Sorry for many emails.
However, I think something is wrong.
It continuously created log files
container_1464766676472_0003_01_xxxx, which meant java process keep
dead.
Is it right?
Was it supposed to do?
Thanks in advance.
Junguk
2016-06-01 1:57 GMT-06:00 Junguk Cho <jm...@gmail.com>:
> Hi, Tushar.
>
> Thanks.
> Yes. It worked.
>
> I have one more question.
> When I access to hadoop web UI and there is menu "ApplicationMaster"
> in Tracking UI menu.
> When I click this, it gave me error.
>
> Sorry, got error 404
>
> Please consult RFC 2616 for meanings of the error code.
>
> Error Details
>
> org.apache.hadoop.yarn.webapp.WebAppException: /: controller for
> default not found
> at org.apache.hadoop.yarn.webapp.Router.resolveDefault(Router.java:232)
> at org.apache.hadoop.yarn.webapp.Router.resolve(Router.java:140)
> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:134)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
> at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
> at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
> at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
> at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
> at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
> at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
> at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
> at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
> at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1243)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>
> The URL is "http://<IP address>:8088/proxy/application_1464766676472_0002/.
>
> With this menu, what we can see?
>
>
> Thanks,
> Junguk
>
> 2016-06-01 1:24 GMT-06:00 Tushar Gosavi <tu...@datatorrent.com>:
>> Hi Junguk,
>>
>> Please check your yarn configuration settings. I normally set
>> minimum-allocation-mb to 256MB for single node install, you can add
>> following lines to your yarn-site.xml and restart the yarn.
>>
>> <property>
>> <name>yarn.scheduler.minimum-allocation-mb</name>
>> <value>256</value>
>> </property>
>>
>> I am able to run YahooFinanceDemo app with above setting on my laptop
>> with 5GB memory usage.
>>
>> Regards,
>> -Tushar.
>>
>> On Wed, Jun 1, 2016 at 12:40 PM, Sandeep Deshmukh
>> <sa...@datatorrent.com> wrote:
>>> Thanks for details.
>>>
>>> Typically the "Waiting for pending status" is followed by "Deploy request"
>>> as highlighted below. Are you getting the highlighted line in your log? If
>>> not, the operators is not getting deployed correctly. Please let us know.
>>>
>>>
>>> 2016-05-23 14:46:51,649 DEBUG
>>> com.datatorrent.stram.engine.StreamingContainer: buffer server memory 512
>>> 2016-05-23 14:46:51,655 INFO
>>> com.datatorrent.bufferserver.storage.DiskStorage: using
>>> /data/hadoop-2.7.2/tmp/nm-local-dir/usercache/sandeep/appcache/application_1463980780220_0003/container_1463980780220_0003_01_000002/tmp
>>> as the basepath for spooling.
>>> 2016-05-23 14:46:51,663 INFO com.datatorrent.bufferserver.server.Server:
>>> Server started listening at /0:0:0:0:0:0:0:0:42358
>>> 2016-05-23 14:46:51,665 DEBUG
>>> com.datatorrent.stram.engine.StreamingContainer: Buffer server started:
>>> /0:0:0:0:0:0:0:0:42358
>>> 2016-05-23 14:46:51,700 DEBUG
>>> com.datatorrent.stram.engine.StreamingContainer: Entering heartbeat loop
>>> (interval is 1000 ms)
>>> 2016-05-23 14:46:52,750 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> request.
>>> 2016-05-23 14:46:53,255 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> request.
>>> 2016-05-23 14:46:56,287 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> request.
>>> 2016-05-23 14:46:56,791 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> request.
>>> 2016-05-23 14:46:57,295 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> request.
>>> 2016-05-23 14:46:57,809 INFO
>>> com.datatorrent.stram.engine.StreamingContainer: Deploy request:
>>> [OperatorDeployInfo[id=1,name=Input,type=INPUT,checkpoint={ffffffffffffffff,
>>> 0,
>>> 0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=enrich,bufferServer=goofy]]]]
>>> 2016-05-23 14:46:57,811 DEBUG
>>> com.datatorrent.stram.engine.StreamingContainer: Restoring operator 1 to
>>> checkpoint ffffffffffffffff stateless=false.
>>> 2016-05-23 14:46:57,876 DEBUG com.datatorrent.stram.engine.Node: type=INPUT,
>>> operator class=class com.datatorrent.InputFileReader
>>> 2016-05-23 14:46:57,889 DEBUG
>>> com.datatorrent.stram.engine.StreamingContainer: Marking operator 1 as
>>> deployed.
>>>
>>>
>>> Regards,
>>> Sandeep
>>>
>>> Regards,
>>> Sandeep
>>>
>>> On Wed, Jun 1, 2016 at 12:29 PM, Junguk Cho <jm...@gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> 1. I used only one machine and use docker image to run it.
>>>> When I access to ip:8088, it shows I use 8 GB memory.
>>>> (Is there a configuration file I can check?)
>>>>
>>>> 2. Hadoop version : 2.7.1
>>>> I tested both Apex 3.2.2 and Apex 3.3.
>>>>
>>>> Thanks,
>>>> Junguk
>>>>
>>>>
>>>> 2016-05-31 22:09 GMT-06:00 Sandeep Deshmukh <sa...@datatorrent.com>:
>>>> > Hi Junguk,
>>>> >
>>>> > Could you please give us some details:
>>>> >
>>>> > Hadoop cluster configuration: no of machines and their configuration
>>>> > (memory
>>>> > in particular)
>>>> > Version of Hadoop and Apex
>>>> >
>>>> > Regards,
>>>> > Sandeep
>>>> >
>>>> > On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <jm...@gmail.com> wrote:
>>>> >>
>>>> >> Hi,
>>>> >>
>>>> >> I am new to apex.
>>>> >> I tried to run demo examples in incubator-apex-malhar.
>>>> >>
>>>> >> When I ran yahoofinance example, I think it was stuck.
>>>> >> Log file said
>>>> >> 2016-05-31 20:34:00,628 INFO
>>>> >> com.datatorrent.bufferserver.server.Server: Server started listening
>>>> >> at /0:0:0:0:0:0:0:0:45546
>>>> >> 2016-05-31 20:34:01,668 INFO
>>>> >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>>> >> request.
>>>> >>
>>>> >> I think binding IP address has a problem.
>>>> >>
>>>> >> Does anyone help me?
>>>> >>
>>>> >>
>>>> >> Thanks,
>>>> >> Junguk
>>>> >
>>>> >
>>>
>>>
Re: Question about running yahoofinance example in apex-malhar
Posted by Junguk Cho <jm...@gmail.com>.
Hi, Tushar.
Thanks.
Yes. It worked.
I have one more question.
When I access to hadoop web UI and there is menu "ApplicationMaster"
in Tracking UI menu.
When I click this, it gave me error.
Sorry, got error 404
Please consult RFC 2616 for meanings of the error code.
Error Details
org.apache.hadoop.yarn.webapp.WebAppException: /: controller for
default not found
at org.apache.hadoop.yarn.webapp.Router.resolveDefault(Router.java:232)
at org.apache.hadoop.yarn.webapp.Router.resolve(Router.java:140)
at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:134)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1243)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
The URL is "http://<IP address>:8088/proxy/application_1464766676472_0002/.
With this menu, what we can see?
Thanks,
Junguk
2016-06-01 1:24 GMT-06:00 Tushar Gosavi <tu...@datatorrent.com>:
> Hi Junguk,
>
> Please check your yarn configuration settings. I normally set
> minimum-allocation-mb to 256MB for single node install, you can add
> following lines to your yarn-site.xml and restart the yarn.
>
> <property>
> <name>yarn.scheduler.minimum-allocation-mb</name>
> <value>256</value>
> </property>
>
> I am able to run YahooFinanceDemo app with above setting on my laptop
> with 5GB memory usage.
>
> Regards,
> -Tushar.
>
> On Wed, Jun 1, 2016 at 12:40 PM, Sandeep Deshmukh
> <sa...@datatorrent.com> wrote:
>> Thanks for details.
>>
>> Typically the "Waiting for pending status" is followed by "Deploy request"
>> as highlighted below. Are you getting the highlighted line in your log? If
>> not, the operators is not getting deployed correctly. Please let us know.
>>
>>
>> 2016-05-23 14:46:51,649 DEBUG
>> com.datatorrent.stram.engine.StreamingContainer: buffer server memory 512
>> 2016-05-23 14:46:51,655 INFO
>> com.datatorrent.bufferserver.storage.DiskStorage: using
>> /data/hadoop-2.7.2/tmp/nm-local-dir/usercache/sandeep/appcache/application_1463980780220_0003/container_1463980780220_0003_01_000002/tmp
>> as the basepath for spooling.
>> 2016-05-23 14:46:51,663 INFO com.datatorrent.bufferserver.server.Server:
>> Server started listening at /0:0:0:0:0:0:0:0:42358
>> 2016-05-23 14:46:51,665 DEBUG
>> com.datatorrent.stram.engine.StreamingContainer: Buffer server started:
>> /0:0:0:0:0:0:0:0:42358
>> 2016-05-23 14:46:51,700 DEBUG
>> com.datatorrent.stram.engine.StreamingContainer: Entering heartbeat loop
>> (interval is 1000 ms)
>> 2016-05-23 14:46:52,750 INFO
>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>> request.
>> 2016-05-23 14:46:53,255 INFO
>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>> request.
>> 2016-05-23 14:46:56,287 INFO
>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>> request.
>> 2016-05-23 14:46:56,791 INFO
>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>> request.
>> 2016-05-23 14:46:57,295 INFO
>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>> request.
>> 2016-05-23 14:46:57,809 INFO
>> com.datatorrent.stram.engine.StreamingContainer: Deploy request:
>> [OperatorDeployInfo[id=1,name=Input,type=INPUT,checkpoint={ffffffffffffffff,
>> 0,
>> 0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=enrich,bufferServer=goofy]]]]
>> 2016-05-23 14:46:57,811 DEBUG
>> com.datatorrent.stram.engine.StreamingContainer: Restoring operator 1 to
>> checkpoint ffffffffffffffff stateless=false.
>> 2016-05-23 14:46:57,876 DEBUG com.datatorrent.stram.engine.Node: type=INPUT,
>> operator class=class com.datatorrent.InputFileReader
>> 2016-05-23 14:46:57,889 DEBUG
>> com.datatorrent.stram.engine.StreamingContainer: Marking operator 1 as
>> deployed.
>>
>>
>> Regards,
>> Sandeep
>>
>> Regards,
>> Sandeep
>>
>> On Wed, Jun 1, 2016 at 12:29 PM, Junguk Cho <jm...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> 1. I used only one machine and use docker image to run it.
>>> When I access to ip:8088, it shows I use 8 GB memory.
>>> (Is there a configuration file I can check?)
>>>
>>> 2. Hadoop version : 2.7.1
>>> I tested both Apex 3.2.2 and Apex 3.3.
>>>
>>> Thanks,
>>> Junguk
>>>
>>>
>>> 2016-05-31 22:09 GMT-06:00 Sandeep Deshmukh <sa...@datatorrent.com>:
>>> > Hi Junguk,
>>> >
>>> > Could you please give us some details:
>>> >
>>> > Hadoop cluster configuration: no of machines and their configuration
>>> > (memory
>>> > in particular)
>>> > Version of Hadoop and Apex
>>> >
>>> > Regards,
>>> > Sandeep
>>> >
>>> > On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <jm...@gmail.com> wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> I am new to apex.
>>> >> I tried to run demo examples in incubator-apex-malhar.
>>> >>
>>> >> When I ran yahoofinance example, I think it was stuck.
>>> >> Log file said
>>> >> 2016-05-31 20:34:00,628 INFO
>>> >> com.datatorrent.bufferserver.server.Server: Server started listening
>>> >> at /0:0:0:0:0:0:0:0:45546
>>> >> 2016-05-31 20:34:01,668 INFO
>>> >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>>> >> request.
>>> >>
>>> >> I think binding IP address has a problem.
>>> >>
>>> >> Does anyone help me?
>>> >>
>>> >>
>>> >> Thanks,
>>> >> Junguk
>>> >
>>> >
>>
>>
Re: Question about running yahoofinance example in apex-malhar
Posted by Tushar Gosavi <tu...@datatorrent.com>.
Hi Junguk,
Please check your yarn configuration settings. I normally set
minimum-allocation-mb to 256MB for single node install, you can add
following lines to your yarn-site.xml and restart the yarn.
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>256</value>
</property>
I am able to run YahooFinanceDemo app with above setting on my laptop
with 5GB memory usage.
Regards,
-Tushar.
On Wed, Jun 1, 2016 at 12:40 PM, Sandeep Deshmukh
<sa...@datatorrent.com> wrote:
> Thanks for details.
>
> Typically the "Waiting for pending status" is followed by "Deploy request"
> as highlighted below. Are you getting the highlighted line in your log? If
> not, the operators is not getting deployed correctly. Please let us know.
>
>
> 2016-05-23 14:46:51,649 DEBUG
> com.datatorrent.stram.engine.StreamingContainer: buffer server memory 512
> 2016-05-23 14:46:51,655 INFO
> com.datatorrent.bufferserver.storage.DiskStorage: using
> /data/hadoop-2.7.2/tmp/nm-local-dir/usercache/sandeep/appcache/application_1463980780220_0003/container_1463980780220_0003_01_000002/tmp
> as the basepath for spooling.
> 2016-05-23 14:46:51,663 INFO com.datatorrent.bufferserver.server.Server:
> Server started listening at /0:0:0:0:0:0:0:0:42358
> 2016-05-23 14:46:51,665 DEBUG
> com.datatorrent.stram.engine.StreamingContainer: Buffer server started:
> /0:0:0:0:0:0:0:0:42358
> 2016-05-23 14:46:51,700 DEBUG
> com.datatorrent.stram.engine.StreamingContainer: Entering heartbeat loop
> (interval is 1000 ms)
> 2016-05-23 14:46:52,750 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-23 14:46:53,255 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-23 14:46:56,287 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-23 14:46:56,791 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-23 14:46:57,295 INFO
> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> request.
> 2016-05-23 14:46:57,809 INFO
> com.datatorrent.stram.engine.StreamingContainer: Deploy request:
> [OperatorDeployInfo[id=1,name=Input,type=INPUT,checkpoint={ffffffffffffffff,
> 0,
> 0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=enrich,bufferServer=goofy]]]]
> 2016-05-23 14:46:57,811 DEBUG
> com.datatorrent.stram.engine.StreamingContainer: Restoring operator 1 to
> checkpoint ffffffffffffffff stateless=false.
> 2016-05-23 14:46:57,876 DEBUG com.datatorrent.stram.engine.Node: type=INPUT,
> operator class=class com.datatorrent.InputFileReader
> 2016-05-23 14:46:57,889 DEBUG
> com.datatorrent.stram.engine.StreamingContainer: Marking operator 1 as
> deployed.
>
>
> Regards,
> Sandeep
>
> Regards,
> Sandeep
>
> On Wed, Jun 1, 2016 at 12:29 PM, Junguk Cho <jm...@gmail.com> wrote:
>>
>> Hi,
>>
>> 1. I used only one machine and use docker image to run it.
>> When I access to ip:8088, it shows I use 8 GB memory.
>> (Is there a configuration file I can check?)
>>
>> 2. Hadoop version : 2.7.1
>> I tested both Apex 3.2.2 and Apex 3.3.
>>
>> Thanks,
>> Junguk
>>
>>
>> 2016-05-31 22:09 GMT-06:00 Sandeep Deshmukh <sa...@datatorrent.com>:
>> > Hi Junguk,
>> >
>> > Could you please give us some details:
>> >
>> > Hadoop cluster configuration: no of machines and their configuration
>> > (memory
>> > in particular)
>> > Version of Hadoop and Apex
>> >
>> > Regards,
>> > Sandeep
>> >
>> > On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <jm...@gmail.com> wrote:
>> >>
>> >> Hi,
>> >>
>> >> I am new to apex.
>> >> I tried to run demo examples in incubator-apex-malhar.
>> >>
>> >> When I ran yahoofinance example, I think it was stuck.
>> >> Log file said
>> >> 2016-05-31 20:34:00,628 INFO
>> >> com.datatorrent.bufferserver.server.Server: Server started listening
>> >> at /0:0:0:0:0:0:0:0:45546
>> >> 2016-05-31 20:34:01,668 INFO
>> >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>> >> request.
>> >>
>> >> I think binding IP address has a problem.
>> >>
>> >> Does anyone help me?
>> >>
>> >>
>> >> Thanks,
>> >> Junguk
>> >
>> >
>
>
Re: Question about running yahoofinance example in apex-malhar
Posted by Sandeep Deshmukh <sa...@datatorrent.com>.
Thanks for details.
Typically the "Waiting for pending status" is followed by "Deploy request"
as highlighted below. Are you getting the highlighted line in your log? If
not, the operators is not getting deployed correctly. Please let us know.
2016-05-23 14:46:51,649 DEBUG
com.datatorrent.stram.engine.StreamingContainer: buffer server memory 512
2016-05-23 14:46:51,655 INFO
com.datatorrent.bufferserver.storage.DiskStorage: using
/data/hadoop-2.7.2/tmp/nm-local-dir/usercache/sandeep/appcache/application_1463980780220_0003/container_1463980780220_0003_01_000002/tmp
as the basepath for spooling.
2016-05-23 14:46:51,663 INFO com.datatorrent.bufferserver.server.Server:
Server started listening at /0:0:0:0:0:0:0:0:42358
2016-05-23 14:46:51,665 DEBUG
com.datatorrent.stram.engine.StreamingContainer: Buffer server started:
/0:0:0:0:0:0:0:0:42358
2016-05-23 14:46:51,700 DEBUG
com.datatorrent.stram.engine.StreamingContainer: Entering heartbeat loop
(interval is 1000 ms)
2016-05-23 14:46:52,750 INFO
com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
request.
2016-05-23 14:46:53,255 INFO
com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
request.
2016-05-23 14:46:56,287 INFO
com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
request.
2016-05-23 14:46:56,791 INFO
com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
request.
2016-05-23 14:46:57,295 INFO
com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
request.
*2016-05-23 14:46:57,809 INFO
com.datatorrent.stram.engine.StreamingContainer: Deploy request:
*[OperatorDeployInfo[id=1,name=Input,type=INPUT,checkpoint={ffffffffffffffff,
0,
0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=enrich,bufferServer=goofy]]]]
2016-05-23 14:46:57,811 DEBUG
com.datatorrent.stram.engine.StreamingContainer: Restoring operator 1 to
checkpoint ffffffffffffffff stateless=false.
2016-05-23 14:46:57,876 DEBUG com.datatorrent.stram.engine.Node:
type=INPUT, operator class=class com.datatorrent.InputFileReader
2016-05-23 14:46:57,889 DEBUG
com.datatorrent.stram.engine.StreamingContainer: Marking operator 1 as
deployed.
Regards,
Sandeep
Regards,
Sandeep
On Wed, Jun 1, 2016 at 12:29 PM, Junguk Cho <jm...@gmail.com> wrote:
> Hi,
>
> 1. I used only one machine and use docker image to run it.
> When I access to ip:8088, it shows I use 8 GB memory.
> (Is there a configuration file I can check?)
>
> 2. Hadoop version : 2.7.1
> I tested both Apex 3.2.2 and Apex 3.3.
>
> Thanks,
> Junguk
>
>
> 2016-05-31 22:09 GMT-06:00 Sandeep Deshmukh <sa...@datatorrent.com>:
> > Hi Junguk,
> >
> > Could you please give us some details:
> >
> > Hadoop cluster configuration: no of machines and their configuration
> (memory
> > in particular)
> > Version of Hadoop and Apex
> >
> > Regards,
> > Sandeep
> >
> > On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <jm...@gmail.com> wrote:
> >>
> >> Hi,
> >>
> >> I am new to apex.
> >> I tried to run demo examples in incubator-apex-malhar.
> >>
> >> When I ran yahoofinance example, I think it was stuck.
> >> Log file said
> >> 2016-05-31 20:34:00,628 INFO
> >> com.datatorrent.bufferserver.server.Server: Server started listening
> >> at /0:0:0:0:0:0:0:0:45546
> >> 2016-05-31 20:34:01,668 INFO
> >> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
> >> request.
> >>
> >> I think binding IP address has a problem.
> >>
> >> Does anyone help me?
> >>
> >>
> >> Thanks,
> >> Junguk
> >
> >
>
Re: Question about running yahoofinance example in apex-malhar
Posted by Junguk Cho <jm...@gmail.com>.
Hi,
1. I used only one machine and use docker image to run it.
When I access to ip:8088, it shows I use 8 GB memory.
(Is there a configuration file I can check?)
2. Hadoop version : 2.7.1
I tested both Apex 3.2.2 and Apex 3.3.
Thanks,
Junguk
2016-05-31 22:09 GMT-06:00 Sandeep Deshmukh <sa...@datatorrent.com>:
> Hi Junguk,
>
> Could you please give us some details:
>
> Hadoop cluster configuration: no of machines and their configuration (memory
> in particular)
> Version of Hadoop and Apex
>
> Regards,
> Sandeep
>
> On Wed, Jun 1, 2016 at 5:04 AM, Junguk Cho <jm...@gmail.com> wrote:
>>
>> Hi,
>>
>> I am new to apex.
>> I tried to run demo examples in incubator-apex-malhar.
>>
>> When I ran yahoofinance example, I think it was stuck.
>> Log file said
>> 2016-05-31 20:34:00,628 INFO
>> com.datatorrent.bufferserver.server.Server: Server started listening
>> at /0:0:0:0:0:0:0:0:45546
>> 2016-05-31 20:34:01,668 INFO
>> com.datatorrent.stram.engine.StreamingContainer: Waiting for pending
>> request.
>>
>> I think binding IP address has a problem.
>>
>> Does anyone help me?
>>
>>
>> Thanks,
>> Junguk
>
>