You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@bahir.apache.org by Nidhal Selmi <ns...@asu.edu> on 2016/10/13 21:40:57 UTC
Fwd: Twitter API request parameters
Greetings,
I am trying to get a twitter stream for a research project using :
TwitterUtils.createStream(ssc, None, filters)
But it seems that not all possible parameters are implemented :
https://dev.twitter.com/streaming/overview/request-parameters
I checked the repo but I don't have a grasp of what is going on :
https://github.com/apache/bahir/blob/master/streaming-
twitter/src/main/scala/org/apache/spark/streaming/
twitter/TwitterInputDStream.scala
How can I fix this so that I can have a stream that is bounded
geographically for example ?
Thank you!
Nidhal.
Re: Twitter API request parameters
Posted by Nidhal Selmi <ns...@asu.edu>.
I tried making a filter similar to this that has "en-US" but I didn't get
any tweets.
filters = ("en-US")
TwitterUtils.createStream(ssc, None, filters)
On Fri, Oct 14, 2016 at 2:28 AM, Nidhal Selmi <ns...@asu.edu> wrote:
> Now on the twitter4j repo we have https://github.com/yusuke/
> twitter4j/blob/master/twitter4j-stream/src/main/
> java/twitter4j/FilterQuery.java
>
> public FilterQuery(int count, long[] follow, String[] track,
> double[][] locations, String[] language) {
> this.count = count;
> this.follow = follow;
> this.track = track;
> this.locations = locations;
> this.language = language;
> }
>
> I am sorry if my question is stupid, I don't have a good grasp of how this
> works :(
>
> On Fri, Oct 14, 2016 at 2:24 AM, Nidhal Selmi <ns...@asu.edu> wrote:
>
>> I think it's a twitter4j issue although I am not sure.
>>
>> I am not getting any errors but the API allows for more types of filters,
>> while only the "track"one is implemented, which takes a list of strings as
>> input.
>> https://dev.twitter.com/streaming/overview/request-parameters
>>
>> notice this piece of code from here https://github.com/apache
>> /bahir/blob/master/streaming-twitter/src/main/scala/org/
>> apache/spark/streaming/twitter/TwitterInputDStream.scala
>>
>> val query = new FilterQuery
>> if (filters.size > 0) {
>> query.track(filters.mkString(","))
>> newTwitterStream.filter(query)
>> } else {
>> newTwitterStream.sample()
>> }
>>
>>
>>
>>
>>
>> On Thu, Oct 13, 2016 at 3:52 PM, Luciano Resende <lu...@gmail.com>
>> wrote:
>>
>>> The TwitterInputDStream supports using filters, which is then mapped to
>>> the query filters you linked below without much manipulation on the
>>> connector side.
>>>
>>> What filters are you using ? Are they just being ignored or are you
>>> getting specific errors ?
>>>
>>>
>>>
>>> On Thu, Oct 13, 2016 at 2:40 PM, Nidhal Selmi <ns...@asu.edu> wrote:
>>>
>>>> Greetings,
>>>>
>>>> I am trying to get a twitter stream for a research project using :
>>>> TwitterUtils.createStream(ssc, None, filters)
>>>>
>>>> But it seems that not all possible parameters are implemented :
>>>> https://dev.twitter.com/streaming/overview/request-parameters
>>>>
>>>> I checked the repo but I don't have a grasp of what is going on :
>>>> https://github.com/apache/bahir/blob/master/streaming-twitte
>>>> r/src/main/scala/org/apache/spark/streaming/twitter/TwitterI
>>>> nputDStream.scala
>>>>
>>>> How can I fix this so that I can have a stream that is bounded
>>>> geographically for example ?
>>>>
>>>> Thank you!
>>>> Nidhal.
>>>>
>>>>
>>>
>>>
>>> --
>>> Luciano Resende
>>> http://twitter.com/lresende1975
>>> http://lresende.blogspot.com/
>>>
>>
>>
>
Re: Twitter API request parameters
Posted by Nidhal Selmi <ns...@asu.edu>.
Now on the twitter4j repo we have
https://github.com/yusuke/twitter4j/blob/master/twitter4j-stream/src/main/java/twitter4j/FilterQuery.java
public FilterQuery(int count, long[] follow, String[] track, double[][]
locations, String[] language) {
this.count = count;
this.follow = follow;
this.track = track;
this.locations = locations;
this.language = language;
}
I am sorry if my question is stupid, I don't have a good grasp of how this
works :(
On Fri, Oct 14, 2016 at 2:24 AM, Nidhal Selmi <ns...@asu.edu> wrote:
> I think it's a twitter4j issue although I am not sure.
>
> I am not getting any errors but the API allows for more types of filters,
> while only the "track"one is implemented, which takes a list of strings as
> input.
> https://dev.twitter.com/streaming/overview/request-parameters
>
> notice this piece of code from here https://github.com/
> apache/bahir/blob/master/streaming-twitter/src/main/
> scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala
>
> val query = new FilterQuery
> if (filters.size > 0) {
> query.track(filters.mkString(","))
> newTwitterStream.filter(query)
> } else {
> newTwitterStream.sample()
> }
>
>
>
>
>
> On Thu, Oct 13, 2016 at 3:52 PM, Luciano Resende <lu...@gmail.com>
> wrote:
>
>> The TwitterInputDStream supports using filters, which is then mapped to
>> the query filters you linked below without much manipulation on the
>> connector side.
>>
>> What filters are you using ? Are they just being ignored or are you
>> getting specific errors ?
>>
>>
>>
>> On Thu, Oct 13, 2016 at 2:40 PM, Nidhal Selmi <ns...@asu.edu> wrote:
>>
>>> Greetings,
>>>
>>> I am trying to get a twitter stream for a research project using :
>>> TwitterUtils.createStream(ssc, None, filters)
>>>
>>> But it seems that not all possible parameters are implemented :
>>> https://dev.twitter.com/streaming/overview/request-parameters
>>>
>>> I checked the repo but I don't have a grasp of what is going on :
>>> https://github.com/apache/bahir/blob/master/streaming-twitte
>>> r/src/main/scala/org/apache/spark/streaming/twitter/TwitterI
>>> nputDStream.scala
>>>
>>> How can I fix this so that I can have a stream that is bounded
>>> geographically for example ?
>>>
>>> Thank you!
>>> Nidhal.
>>>
>>>
>>
>>
>> --
>> Luciano Resende
>> http://twitter.com/lresende1975
>> http://lresende.blogspot.com/
>>
>
>
Re: Twitter API request parameters
Posted by Nidhal Selmi <ns...@asu.edu>.
I think it's a twitter4j issue although I am not sure.
I am not getting any errors but the API allows for more types of filters,
while only the "track"one is implemented, which takes a list of strings as
input.
https://dev.twitter.com/streaming/overview/request-parameters
notice this piece of code from here
https://github.com/apache/bahir/blob/master/streaming-twitter/src/main/scala/org/apache/spark/streaming/twitter/TwitterInputDStream.scala
val query = new FilterQuery
if (filters.size > 0) {
query.track(filters.mkString(","))
newTwitterStream.filter(query)
} else {
newTwitterStream.sample()
}
On Thu, Oct 13, 2016 at 3:52 PM, Luciano Resende <lu...@gmail.com>
wrote:
> The TwitterInputDStream supports using filters, which is then mapped to
> the query filters you linked below without much manipulation on the
> connector side.
>
> What filters are you using ? Are they just being ignored or are you
> getting specific errors ?
>
>
>
> On Thu, Oct 13, 2016 at 2:40 PM, Nidhal Selmi <ns...@asu.edu> wrote:
>
>> Greetings,
>>
>> I am trying to get a twitter stream for a research project using :
>> TwitterUtils.createStream(ssc, None, filters)
>>
>> But it seems that not all possible parameters are implemented :
>> https://dev.twitter.com/streaming/overview/request-parameters
>>
>> I checked the repo but I don't have a grasp of what is going on :
>> https://github.com/apache/bahir/blob/master/streaming-twitte
>> r/src/main/scala/org/apache/spark/streaming/twitter/TwitterI
>> nputDStream.scala
>>
>> How can I fix this so that I can have a stream that is bounded
>> geographically for example ?
>>
>> Thank you!
>> Nidhal.
>>
>>
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>
Re: Twitter API request parameters
Posted by Luciano Resende <lu...@gmail.com>.
The TwitterInputDStream supports using filters, which is then mapped to the
query filters you linked below without much manipulation on the connector
side.
What filters are you using ? Are they just being ignored or are you getting
specific errors ?
On Thu, Oct 13, 2016 at 2:40 PM, Nidhal Selmi <ns...@asu.edu> wrote:
> Greetings,
>
> I am trying to get a twitter stream for a research project using :
> TwitterUtils.createStream(ssc, None, filters)
>
> But it seems that not all possible parameters are implemented :
> https://dev.twitter.com/streaming/overview/request-parameters
>
> I checked the repo but I don't have a grasp of what is going on :
> https://github.com/apache/bahir/blob/master/streaming-twitte
> r/src/main/scala/org/apache/spark/streaming/twitter/
> TwitterInputDStream.scala
>
> How can I fix this so that I can have a stream that is bounded
> geographically for example ?
>
> Thank you!
> Nidhal.
>
>
--
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/