You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by pjmccarthy <pm...@eatonvance.com> on 2015/07/23 21:23:41 UTC
Twitter4J streaming question
Hopefully this is an easy one. I am trying to filter a twitter dstream by
user ScreenName - my code is as follows
val stream = TwitterUtils.createStream(ssc, None)
.filter(_.getUser.getScreenName.contains("markets"))
however nothing gets returned and I can see that Bloomberg has tweeted. If
I remove the filter I get tweets
If I change the code to looke for engligh or french tweets that works
Is there a better way to do it ?
Can anyone assist ?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Twitter4J-streaming-question-tp23974.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Twitter4J streaming question
Posted by Patrick McCarthy <PM...@EatonVance.Com>.
Ahh
Makes sense - thanks for the help
Sent from my iPhone
On Jul 23, 2015, at 4:29 PM, Enno Shioji <es...@gmail.com>> wrote:
You need to pay a lot of money to get the full stream, so unless you are doing that, it's the sample stream!
On Thu, Jul 23, 2015 at 9:26 PM, Patrick McCarthy <PM...@eatonvance.com>> wrote:
How can I tell if it's the sample stream or full stream ?
Thanks
Sent from my iPhone
On Jul 23, 2015, at 4:17 PM, Enno Shioji <es...@gmail.com>> wrote:
You are probably listening to the sample stream, and THEN filtering. This means you listen to 1% of the twitter stream, and then looking for the tweet by Bloomberg, so there is a very good chance you don't see the particular tweet.
In order to get all Bloomberg related tweets, you must connect to twitter using the filter API and not the sample API: https://dev.twitter.com/streaming/reference/post/statuses/filter
On Thu, Jul 23, 2015 at 8:23 PM, pjmccarthy <pm...@eatonvance.com>> wrote:
Hopefully this is an easy one. I am trying to filter a twitter dstream by
user ScreenName - my code is as follows
val stream = TwitterUtils.createStream(ssc, None)
.filter(_.getUser.getScreenName.contains("markets"))
however nothing gets returned and I can see that Bloomberg has tweeted. If
I remove the filter I get tweets
If I change the code to looke for engligh or french tweets that works
Is there a better way to do it ?
Can anyone assist ?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Twitter4J-streaming-question-tp23974.html
Sent from the Apache Spark User List mailing list archive at Nabble.com<http://Nabble.com>.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org<ma...@spark.apache.org>
For additional commands, e-mail: user-help@spark.apache.org<ma...@spark.apache.org>
Re: Twitter4J streaming question
Posted by Jörn Franke <jo...@gmail.com>.
He should still see something. I think you need to subscribe to the
Screenname first and not filter it out only in the filter method. I do not
have the apis from mobile at hand, but there should be a method.
Le jeu. 23 juil. 2015 à 22:30, Enno Shioji <es...@gmail.com> a écrit :
> You need to pay a lot of money to get the full stream, so unless you are
> doing that, it's the sample stream!
>
> On Thu, Jul 23, 2015 at 9:26 PM, Patrick McCarthy <
> PMcCarthy@eatonvance.com> wrote:
>
>> How can I tell if it's the sample stream or full stream ?
>> Thanks
>>
>> Sent from my iPhone
>>
>> On Jul 23, 2015, at 4:17 PM, Enno Shioji <es...@gmail.com> wrote:
>>
>> You are probably listening to the sample stream, and THEN filtering.
>> This means you listen to 1% of the twitter stream, and then looking for the
>> tweet by Bloomberg, so there is a very good chance you don't see the
>> particular tweet.
>>
>> In order to get all Bloomberg related tweets, you must connect to
>> twitter using the filter API and not the sample API:
>> https://dev.twitter.com/streaming/reference/post/statuses/filter
>>
>> On Thu, Jul 23, 2015 at 8:23 PM, pjmccarthy <pm...@eatonvance.com>
>> wrote:
>>
>>> Hopefully this is an easy one. I am trying to filter a twitter dstream
>>> by
>>> user ScreenName - my code is as follows
>>> val stream = TwitterUtils.createStream(ssc, None)
>>> .filter(_.getUser.getScreenName.contains("markets"))
>>>
>>> however nothing gets returned and I can see that Bloomberg has tweeted.
>>> If
>>> I remove the filter I get tweets
>>> If I change the code to looke for engligh or french tweets that works
>>>
>>> Is there a better way to do it ?
>>>
>>> Can anyone assist ?
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-spark-user-list.1001560.n3.nabble.com/Twitter4J-streaming-question-tp23974.html
>>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: user-help@spark.apache.org
>>>
>>>
>>
>
Re: Twitter4J streaming question
Posted by Enno Shioji <es...@gmail.com>.
You need to pay a lot of money to get the full stream, so unless you are
doing that, it's the sample stream!
On Thu, Jul 23, 2015 at 9:26 PM, Patrick McCarthy <PM...@eatonvance.com>
wrote:
> How can I tell if it's the sample stream or full stream ?
> Thanks
>
> Sent from my iPhone
>
> On Jul 23, 2015, at 4:17 PM, Enno Shioji <es...@gmail.com> wrote:
>
> You are probably listening to the sample stream, and THEN filtering.
> This means you listen to 1% of the twitter stream, and then looking for the
> tweet by Bloomberg, so there is a very good chance you don't see the
> particular tweet.
>
> In order to get all Bloomberg related tweets, you must connect to
> twitter using the filter API and not the sample API:
> https://dev.twitter.com/streaming/reference/post/statuses/filter
>
> On Thu, Jul 23, 2015 at 8:23 PM, pjmccarthy <pm...@eatonvance.com>
> wrote:
>
>> Hopefully this is an easy one. I am trying to filter a twitter dstream by
>> user ScreenName - my code is as follows
>> val stream = TwitterUtils.createStream(ssc, None)
>> .filter(_.getUser.getScreenName.contains("markets"))
>>
>> however nothing gets returned and I can see that Bloomberg has tweeted.
>> If
>> I remove the filter I get tweets
>> If I change the code to looke for engligh or french tweets that works
>>
>> Is there a better way to do it ?
>>
>> Can anyone assist ?
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/Twitter4J-streaming-question-tp23974.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
Re: Twitter4J streaming question
Posted by Patrick McCarthy <PM...@EatonVance.Com>.
How can I tell if it's the sample stream or full stream ?
Thanks
Sent from my iPhone
On Jul 23, 2015, at 4:17 PM, Enno Shioji <es...@gmail.com>> wrote:
You are probably listening to the sample stream, and THEN filtering. This means you listen to 1% of the twitter stream, and then looking for the tweet by Bloomberg, so there is a very good chance you don't see the particular tweet.
In order to get all Bloomberg related tweets, you must connect to twitter using the filter API and not the sample API: https://dev.twitter.com/streaming/reference/post/statuses/filter
On Thu, Jul 23, 2015 at 8:23 PM, pjmccarthy <pm...@eatonvance.com>> wrote:
Hopefully this is an easy one. I am trying to filter a twitter dstream by
user ScreenName - my code is as follows
val stream = TwitterUtils.createStream(ssc, None)
.filter(_.getUser.getScreenName.contains("markets"))
however nothing gets returned and I can see that Bloomberg has tweeted. If
I remove the filter I get tweets
If I change the code to looke for engligh or french tweets that works
Is there a better way to do it ?
Can anyone assist ?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Twitter4J-streaming-question-tp23974.html
Sent from the Apache Spark User List mailing list archive at Nabble.com<http://Nabble.com>.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org<ma...@spark.apache.org>
For additional commands, e-mail: user-help@spark.apache.org<ma...@spark.apache.org>
Re: Twitter4J streaming question
Posted by Enno Shioji <es...@gmail.com>.
You are probably listening to the sample stream, and THEN filtering. This
means you listen to 1% of the twitter stream, and then looking for the
tweet by Bloomberg, so there is a very good chance you don't see the
particular tweet.
In order to get all Bloomberg related tweets, you must connect to twitter
using the filter API and not the sample API:
https://dev.twitter.com/streaming/reference/post/statuses/filter
On Thu, Jul 23, 2015 at 8:23 PM, pjmccarthy <pm...@eatonvance.com>
wrote:
> Hopefully this is an easy one. I am trying to filter a twitter dstream by
> user ScreenName - my code is as follows
> val stream = TwitterUtils.createStream(ssc, None)
> .filter(_.getUser.getScreenName.contains("markets"))
>
> however nothing gets returned and I can see that Bloomberg has tweeted. If
> I remove the filter I get tweets
> If I change the code to looke for engligh or french tweets that works
>
> Is there a better way to do it ?
>
> Can anyone assist ?
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Twitter4J-streaming-question-tp23974.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>