You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@apex.apache.org by db...@softiron.co.uk on 2016/09/20 19:36:18 UTC
malhar twitter demo problem
I'm trying to run the RollingTopWordsDemo option (#1) in the malhar
version 3.4 twitter demo,
and the TweetSampler operator gets the exception:
java.lang.RuntimeException: java.lang.InterruptedException: sleep
interrupted
at com.google.common.base.Throwables.propagate(Throwables.java:156)
at com.datatorrent.stram.engine.InputNode.run(InputNode.java:209)
at
com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1407)
Caused by: java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at com.datatorrent.stram.engine.InputNode.run(InputNode.java:101)
... 1 more
I'm new to Apex, so I'm not sure how to debug this.
Any suggestions or ideas?
Thanks in advance,
-david
Re: malhar twitter demo problem
Posted by db...@softiron.co.uk.
On 2016-09-26 13:42, dbs@softiron.co.uk wrote:
> On 2016-09-25 23:10, Chinmay Kolhatkar wrote:
>> Hi,
>>
>> I can see RollingTopWordsDemo running fine in my local setup.
>
> What variables did you set to configure your twitter credentials?
> What xml file did you write them to?
>
>> Can you please share complete exception?
2nd attempt to answer this question because I think the first answered a
different one.
Hopefully this one is better.
Here's what I see in the dt.log file from one of the containers that is
killed:
2016-09-26 14:23:44,229 DEBUG twitter4j.TwitterStreamImpl: 404:The URI
requested is invalid or the resource requested, such as a user, does not
exist.
Unknown URL. See Twitter Streaming API documentation at
http://dev.twitter.com/pages/streaming_api
2016-09-26 14:23:44,230 ERROR
com.datatorrent.contrib.twitter.TwitterSampleInput: Sampling Error
404:The URI requested is invalid or the resource requested, such as a
user, does not exist.
Unknown URL. See Twitter Streaming API documentation at
http://dev.twitter.com/pages/streaming_api
Relevant discussions can be found on the Internet at:
http://www.google.co.jp/search?q=ec814753 or
http://www.google.co.jp/search?q=0a74cca1
TwitterException{exceptionCode=[ec814753-0a74cca1], statusCode=404,
retryAfter=-1, rateLimitStatus=null,
featureSpecificRateLimitStatus=null, version=2.2.6}
at
twitter4j.internal.http.HttpClientImpl.request(HttpClientImpl.java:185)
at
twitter4j.internal.http.HttpClientWrapper.request(HttpClientWrapper.java:65)
at
twitter4j.internal.http.HttpClientWrapper.get(HttpClientWrapper.java:93)
at
twitter4j.TwitterStreamImpl.getSampleStream(TwitterStreamImpl.java:160)
at twitter4j.TwitterStreamImpl$4.getStream(TwitterStreamImpl.java:149)
at twitter4j.TwitterStreamImpl$4.getStream(TwitterStreamImpl.java:147)
at
twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java:426)
2016-09-26 14:23:44,236 DEBUG
com.datatorrent.contrib.twitter.TwitterSampleInput: reconnect: false
2016-09-26 14:23:44,236 DEBUG twitter4j.TwitterStreamImpl: Twitter
Stream consumer-1[Disposing thread]
2016-09-26 14:23:44,237 ERROR
com.datatorrent.stram.engine.StreamingContainer: Operator set
[OperatorDeployInfo[id=1,name=TweetSampler,type=INPUT,checkpoint={57e9911c0000012b,
0,
0},inputs=[],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=text,streamId=TweetStream,bufferServer=<null>]]]]
stopped running due to an exception.
java.lang.RuntimeException: java.lang.InterruptedException: sleep
interrupted
at com.google.common.base.Throwables.propagate(Throwables.java:156)
at com.datatorrent.stram.engine.InputNode.run(InputNode.java:209)
at
com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1407)
Caused by: java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at com.datatorrent.stram.engine.InputNode.run(InputNode.java:101)
... 1 more
>
>> -Chinmay.
>
> Thanks much,
> -david
>
>
>> On Wed, Sep 21, 2016 at 1:06 AM, <db...@softiron.co.uk> wrote:
>>
>>> I'm trying to run the RollingTopWordsDemo option (#1) in the malhar
>>> version 3.4 twitter demo,
>>> and the TweetSampler operator gets the exception:
>>>
>>> java.lang.RuntimeException: java.lang.InterruptedException: sleep
>>> interrupted
>>> at
>>> com.google.common.base.Throwables.propagate(Throwables.java:156)
>>> at
>>> com.datatorrent.stram.engine.InputNode.run(InputNode.java:209)
>>> at
>>>
>> com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1407)
>>> Caused by: java.lang.InterruptedException: sleep interrupted
>>> at java.lang.Thread.sleep(Native Method)
>>> at
>>> com.datatorrent.stram.engine.InputNode.run(InputNode.java:101)
>>> ... 1 more
>>>
>>> I'm new to Apex, so I'm not sure how to debug this.
>>> Any suggestions or ideas?
>>>
>>> Thanks in advance,
>>> -david
Re: malhar twitter demo problem
Posted by db...@softiron.co.uk.
On 2016-09-25 23:10, Chinmay Kolhatkar wrote:
> Hi,
>
> I can see RollingTopWordsDemo running fine in my local setup.
What variables did you set to configure your twitter credentials?
What xml file did you write them to?
> Can you please share complete exception?
All I see is:
2016-09-26 13:27:15,173 INFO
com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception in
processing web service: java.lang.IllegalArgumentException: Wrong FS:
hdfs://namenode:9000/user/dtadmin/datatorrent/appPackages, expected:
file:///
> Also, Do you see TwitterSampleInput class mentioned in exception
> anywhere?
I grep'd everything in /var/log/datatorrent and the only place it shows
up is in
dtgateway.access.log:
<IPADDR> - - [22/Sep/2016:22:12:37 +0000] "GET
/ws/v2/applications/application_1474394828255_0012/operatorClasses/com.datatorrent.contrib.twitter.TwitterSampleInput
HTTP/1.1" 200 1495 "http://namenode:9090/static/" "Mozilla/5.0
<CONFIG_INFO>
I do get the following exception in dtgateway.log when shutting down the
gateway.
I don't know if it's significant.
2016-09-26 13:40:44,366 INFO com.datatorrent.gateway.DTGateway: Shutting
down
2016-09-26 13:40:44,427 ERROR com.datatorrent.gateway.I: DFS Directory
cannot be written to with exception:
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:808)
at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:2041)
at
org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:707)
at
org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:703)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:714)
at com.datatorrent.stram.client.FSAgent.deleteFile(FSAgent.java:94)
at com.datatorrent.gateway.I.h(gc:30)
at com.datatorrent.gateway.l.K(pc:158)
at com.datatorrent.gateway.l.h(pc:65)
at com.datatorrent.gateway.K.run(yc:156)
at java.lang.Thread.run(Thread.java:745)
2016-09-26 13:40:44,444 INFO com.datatorrent.gateway.DTGateway: Shutdown
complete
>
> -Chinmay.
Thanks much,
-david
> On Wed, Sep 21, 2016 at 1:06 AM, <db...@softiron.co.uk> wrote:
>
>> I'm trying to run the RollingTopWordsDemo option (#1) in the malhar
>> version 3.4 twitter demo,
>> and the TweetSampler operator gets the exception:
>>
>> java.lang.RuntimeException: java.lang.InterruptedException: sleep
>> interrupted
>> at
>> com.google.common.base.Throwables.propagate(Throwables.java:156)
>> at
>> com.datatorrent.stram.engine.InputNode.run(InputNode.java:209)
>> at
>>
> com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1407)
>> Caused by: java.lang.InterruptedException: sleep interrupted
>> at java.lang.Thread.sleep(Native Method)
>> at
>> com.datatorrent.stram.engine.InputNode.run(InputNode.java:101)
>> ... 1 more
>>
>> I'm new to Apex, so I'm not sure how to debug this.
>> Any suggestions or ideas?
>>
>> Thanks in advance,
>> -david
Re: malhar twitter demo problem
Posted by Chinmay Kolhatkar <ch...@datatorrent.com>.
Hi,
I can see RollingTopWordsDemo running fine in my local setup.
Can you please share complete exception?
Also, Do you see TwitterSampleInput class mentioned in exception anywhere?
-Chinmay.
On Wed, Sep 21, 2016 at 1:06 AM, <db...@softiron.co.uk> wrote:
> I'm trying to run the RollingTopWordsDemo option (#1) in the malhar
> version 3.4 twitter demo,
> and the TweetSampler operator gets the exception:
>
> java.lang.RuntimeException: java.lang.InterruptedException: sleep
> interrupted
> at com.google.common.base.Throwables.propagate(Throwables.java:
> 156)
> at com.datatorrent.stram.engine.InputNode.run(InputNode.java:209)
> at com.datatorrent.stram.engine.StreamingContainer$2.run(Stream
> ingContainer.java:1407)
> Caused by: java.lang.InterruptedException: sleep interrupted
> at java.lang.Thread.sleep(Native Method)
> at com.datatorrent.stram.engine.InputNode.run(InputNode.java:101)
> ... 1 more
>
> I'm new to Apex, so I'm not sure how to debug this.
> Any suggestions or ideas?
>
> Thanks in advance,
> -david
>
>