RE: [EXTERNAL] - Re: GetTwitter to stream tweets instead pull

[Jamie Wang <ja...@opentext.com>]

Hi Joey,

Thanks for the information. The name actually plays only a small part as you indicated. But I sort of got convinced it is pulling after reading the help documentation for GetTwitter. You can see it here: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.twitter.GetTwitter/. The line "Pulls status changes from Twitter's streaming API" sort of got me to believe it is a pull instead streaming. Also it'd be a good idea if possible to add a line to explicitly document it is actually streaming. Thanks again for your note and my apology for the late response.

Jamie

From: Joey Frazee [mailto:joey.frazee@icloud.com]
Sent: Wednesday, May 03, 2017 12:51 PM
To: users@nifi.apache.org
Subject: [EXTERNAL] - Re: GetTwitter to stream tweets instead pull

Jamie, can you explain a little bit more about what you’re looking for?

The GetTwitter processor is accessing the spritzer/decahose/firehouse, what have you, via Twitter’s Hosebird library. This library is indeed streaming the Tweets from their sample and filter APIs in the usual way with a persistent, chunk-encoded HTTP connection to https://stream.twitter.com/1.1/statuses/sample.json<https://urldefense.proofpoint.com/v2/url?u=https-3A__stream.twitter.com_1.1_statuses_sample.json&d=DwMFaQ&c=ZgVRmm3mf2P1-XDAyDsu4A&r=TauVD_Op4rvIkArzdRrvTf-yuf4tmnM8R0LshdIDMbA&m=6M-GKh1Kc9QCdlSt4lVirUcS1suuGRF4MMEfHvjR45A&s=a8suuN1dyAV6Q9X5Z0tEns7WODrBD8VcmH0ctYLhJto&e=> and https://stream.twitter.com/1.1/statuses/filter.json<https://urldefense.proofpoint.com/v2/url?u=https-3A__stream.twitter.com_1.1_statuses_filter.json&d=DwMFaQ&c=ZgVRmm3mf2P1-XDAyDsu4A&r=TauVD_Op4rvIkArzdRrvTf-yuf4tmnM8R0LshdIDMbA&m=6M-GKh1Kc9QCdlSt4lVirUcS1suuGRF4MMEfHvjR45A&s=0TueQgZE0p1YdPjGiDCQ5IVHtDBL5h_C0DZsOFKpEsA&e=>.

I’ll admit the name might be a little confusing since the Get might suggest it’s hitting one of the REST https://api.twitter.com/1.1/statuses/<https://urldefense.proofpoint.com/v2/url?u=https-3A__api.twitter.com_1.1_statuses_&d=DwMFaQ&c=ZgVRmm3mf2P1-XDAyDsu4A&r=TauVD_Op4rvIkArzdRrvTf-yuf4tmnM8R0LshdIDMbA&m=6M-GKh1Kc9QCdlSt4lVirUcS1suuGRF4MMEfHvjR45A&s=-8N0_UEmgW8Yb56o43fS87OzYVjtoFlZmSBZzVTfa1Y&e=> resources periodically instead of using a long-term HTTP connection.

-joey

On May 3, 2017, at 2:00 PM, Jamie Wang <ja...@opentext.com>> wrote:

Hi,

I understand the built-in processor GetTwitter is a pull. Are there streaming based processor for getting Tweets available? If no, any suggestions on how would l go by to build one?

Thanks
Jamie