You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Mothilal M <mo...@gmail.com> on 2016/11/08 18:25:16 UTC
Need Information - Apache NiFi
Hi Apache NiFi team,
Greetings. I did my hands on experience with multiple tutorials around
sentimental analysis integrating social media like twitter and facebook and
also log analysis. Could you help me understand in what features Apache
NiFi would act as an alternative to Splunk.
I do have couple of technical clarifications in Apache NiFi - would it be
possible to have a initial technical call to clarify my questions.
Warm Regards,
M. Mothilal
Cell : +1(647)641-0910.
Re: Need Information - Apache NiFi
Posted by Joe Witt <jo...@gmail.com>.
Regarding comparison of NiFi and Splunk this is probably not the best
place to make those comparisons or claims and I'm not sure the two are
even in comparable space. What we can easily do though is help you in
talking through your use case and providing ideas on how Apache NiFi
can play a role in addressing it.
I'm hopeful others more knowledgable with those specific processors
and APIs (twitter, facebook) can assist.
Thanks
Joe
On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <mo...@gmail.com> wrote:
> Greetings Joe,
>
> Glad to connect with you from your reply in Apache NiFi community. I am
> integrating NiFi with social media - twitter and Facebook to achieve
> analysis but I am struck at a point. Request your assistance in solving the
> same.
>
> getTwitter processor - i am not able to get tweets filtered by twitter IDs.
> (IDs to follow field value is not working as expected. I did get the numeric
> value of twitter IDs and input in the value as comma separated but no
> results). Please advice as I look to filter my tweets on particular IDs and
> also by particular location by cities like Toronto, calgary.
>
> getHTTP processor for facebook integration works fine but i am unable to
> fetch data from my personal facebook ID. I tried accessing the Graph API and
> it mentions user permissions are not granted but I did provide the essential
> permissions in app settings. Please advice if you could help out in setting
> the facebook app permissions to ensure I can fetch user_status,
> user_location fields. On success, I would need to venture out few options
> further.
>
> General question - How could Apache NiFi be useful in replacing Splunk. I
> understand most of the current companies have a paid version of Splunk and
> NiFi is an open-source but please let me know features which are key in
> replacing Splunk.
>
> I appreciate your time and would be more helpful if you could let me know
> your comfortable timing for a technical call discussion. I am available and
> reachable on +1(647)641-0910. Look forwardly for your reply.
>
> Warm Regards,
> M. Mothilal
>
> ------------------------------------------------------------------------------------------
>
> Hello,
>
> I think as a community we're all quite happy to help address questions
> or ideas you might have for Apache NiFi. Please feel free to ask your
> questions here and if they're more from the perspective of a user
> please use the users@nifi.apache.org list.
>
> For the technical questions it is probably best to align them with
> some portion of the overview, administration, user, best practice, or
> developer guide so we can use that as a basis for further discussion.
> You can find those here: https://nifi.apache.org/docs.html
>
> Thanks
> Joe
>
>
> ---------- Forwarded message ----------
> From: Mothilal M <mo...@gmail.com>
> Date: Tue, Nov 8, 2016 at 1:25 PM
> Subject: Need Information - Apache NiFi
> To: dev@nifi.apache.org
>
>
> Hi Apache NiFi team,
>
> Greetings. I did my hands on experience with multiple tutorials around
> sentimental analysis integrating social media like twitter and facebook and
> also log analysis. Could you help me understand in what features Apache NiFi
> would act as an alternative to Splunk.
>
> I do have couple of technical clarifications in Apache NiFi - would it be
> possible to have a initial technical call to clarify my questions.
>
> Warm Regards,
> M. Mothilal
> Cell : +1(647)641-0910.
>
Re: Need Information - Apache NiFi
Posted by Bryan Bende <bb...@gmail.com>.
For GetTwitter, I can't really tell if the followers field is supposed to
be the numeric id, or just the twitter username that you would see in
tweets.
I believe it would be this from their API:
https://dev.twitter.com/streaming/overview/request-parameters#follow
It does say that IDs to Follow is only used when using the Filter
end-point, did you try changing the end-point to Filter instead of Sample?
-Bryan
On Tue, Nov 8, 2016 at 2:45 PM, Mothilal M <mo...@gmail.com> wrote:
> Hi Joe,
>
> Greetings and appreciate your reply. Totally agree with your opinion on
> comparisons.
>
> Look forward for other developers to comment on suggestions of getTwitter
> and getHTTP processor issues for twitter and facebook integration.
>
> Warm Regards,
> M. Mothilal
>
> On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <mo...@gmail.com> wrote:
>
> > Greetings Joe,
> >
> > Glad to connect with you from your reply in Apache NiFi community. I am
> > integrating NiFi with social media - twitter and Facebook to achieve
> > analysis but I am struck at a point. Request your assistance in solving
> the
> > same.
> >
> > getTwitter processor - i am not able to get tweets filtered by twitter
> > IDs. (IDs to follow field value is not working as expected. I did get the
> > numeric value of twitter IDs and input in the value as comma separated
> but
> > no results). Please advice as I look to filter my tweets on particular
> IDs
> > and also by particular location by cities like Toronto, calgary.
> >
> > getHTTP processor for facebook integration works fine but i am unable to
> > fetch data from my personal facebook ID. I tried accessing the Graph API
> > and it mentions user permissions are not granted but I did provide the
> > essential permissions in app settings. Please advice if you could help
> out
> > in setting the facebook app permissions to ensure I can fetch
> user_status,
> > user_location fields. On success, I would need to venture out few options
> > further.
> >
> > General question - How could Apache NiFi be useful in replacing Splunk. I
> > understand most of the current companies have a paid version of Splunk
> and
> > NiFi is an open-source but please let me know features which are key in
> > replacing Splunk.
> >
> > I appreciate your time and would be more helpful if you could let me know
> > your comfortable timing for a technical call discussion. I am available
> and
> > reachable on +1(647)641-0910. Look forwardly for your reply.
> >
> > Warm Regards,
> > M. Mothilal
> >
> > ------------------------------------------------------------
> > ------------------------------
> >
> > Hello,
> >
> > I think as a community we're all quite happy to help address questions
> > or ideas you might have for Apache NiFi. Please feel free to ask your
> > questions here and if they're more from the perspective of a user
> > please use the users@nifi.apache.org list.
> >
> > For the technical questions it is probably best to align them with
> > some portion of the overview, administration, user, best practice, or
> > developer guide so we can use that as a basis for further discussion.
> > You can find those here: https://nifi.apache.org/docs.html
> >
> > Thanks
> > Joe
> >
> >
> > ---------- Forwarded message ----------
> > From: Mothilal M <mo...@gmail.com>
> > Date: Tue, Nov 8, 2016 at 1:25 PM
> > Subject: Need Information - Apache NiFi
> > To: dev@nifi.apache.org
> >
> >
> > Hi Apache NiFi team,
> >
> > Greetings. I did my hands on experience with multiple tutorials around
> > sentimental analysis integrating social media like twitter and facebook
> and
> > also log analysis. Could you help me understand in what features Apache
> > NiFi would act as an alternative to Splunk.
> >
> > I do have couple of technical clarifications in Apache NiFi - would it be
> > possible to have a initial technical call to clarify my questions.
> >
> > Warm Regards,
> > M. Mothilal
> > Cell : +1(647)641-0910.
> >
> >
>
Re: Need Information - Apache NiFi
Posted by Mothi86 <mo...@gmail.com>.
Thanks @JoeWitt @BryanBende @AndrewPsaltis for your valuable comments.
Getting back active on this thread connections after a long time. I was able
to succeed long back in retrieving the tweets filtered by users (IDs needs
to be the numeric values and can be fetched from twitter or
http://gettwitterid.com/).
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Need-Information-Apache-NiFi-tp13842p16250.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Need Information - Apache NiFi
Posted by Andrew Psaltis <ps...@gmail.com>.
Hi M. Mothilal,
Attached is a template that filters based on a Twitter ID (my id, which I
retrieved from [1]). As I am sure you already know, you need to change the
tokens and keys to match your application. After setting up the
configuration, I went to Twitter and just re-tweeted a tweet and moments
later it appeared in my NiFi flow. The different actions that should cause
data to appear are identified here [2].
Hopefully this helps.
[1] http://gettwitterid.com/?user_name=itmdata&submit=GET+USER+ID
[2] https://dev.twitter.com/streaming/overview/request-parameters#follow
Thanks,
Andrew
On Tue, Nov 8, 2016 at 6:59 PM, Mothilal M <mo...@gmail.com> wrote:
> Hi Bryan Bende,
>
> Greetings. I did try filter endpoint and it certainly needs numeric ID and
> not twitter ID. Processor shows error if twitter ID - so I had converted it
> to respective numeric. Please advice if there is any sample worked on this
> use case - filtered tweets based on specific IDs.
>
> Warm Regards,
> M. Mothilal
>
> On Tue, Nov 8, 2016 at 2:45 PM, Mothilal M <mo...@gmail.com> wrote:
>
> > Hi Joe,
> >
> > Greetings and appreciate your reply. Totally agree with your opinion on
> > comparisons.
> >
> > Look forward for other developers to comment on suggestions of getTwitter
> > and getHTTP processor issues for twitter and facebook integration.
> >
> > Warm Regards,
> > M. Mothilal
> >
> > On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <mo...@gmail.com> wrote:
> >
> >> Greetings Joe,
> >>
> >> Glad to connect with you from your reply in Apache NiFi community. I am
> >> integrating NiFi with social media - twitter and Facebook to achieve
> >> analysis but I am struck at a point. Request your assistance in solving
> the
> >> same.
> >>
> >> getTwitter processor - i am not able to get tweets filtered by twitter
> >> IDs. (IDs to follow field value is not working as expected. I did get
> the
> >> numeric value of twitter IDs and input in the value as comma separated
> but
> >> no results). Please advice as I look to filter my tweets on particular
> IDs
> >> and also by particular location by cities like Toronto, calgary.
> >>
> >> getHTTP processor for facebook integration works fine but i am unable to
> >> fetch data from my personal facebook ID. I tried accessing the Graph API
> >> and it mentions user permissions are not granted but I did provide the
> >> essential permissions in app settings. Please advice if you could help
> out
> >> in setting the facebook app permissions to ensure I can fetch
> user_status,
> >> user_location fields. On success, I would need to venture out few
> options
> >> further.
> >>
> >> General question - How could Apache NiFi be useful in replacing Splunk.
> I
> >> understand most of the current companies have a paid version of Splunk
> and
> >> NiFi is an open-source but please let me know features which are key in
> >> replacing Splunk.
> >>
> >> I appreciate your time and would be more helpful if you could let me
> know
> >> your comfortable timing for a technical call discussion. I am available
> and
> >> reachable on +1(647)641-0910. Look forwardly for your reply.
> >>
> >> Warm Regards,
> >> M. Mothilal
> >>
> >> ------------------------------------------------------------
> >> ------------------------------
> >>
> >> Hello,
> >>
> >> I think as a community we're all quite happy to help address questions
> >> or ideas you might have for Apache NiFi. Please feel free to ask your
> >> questions here and if they're more from the perspective of a user
> >> please use the users@nifi.apache.org list.
> >>
> >> For the technical questions it is probably best to align them with
> >> some portion of the overview, administration, user, best practice, or
> >> developer guide so we can use that as a basis for further discussion.
> >> You can find those here: https://nifi.apache.org/docs.html
> >>
> >> Thanks
> >> Joe
> >>
> >>
> >> ---------- Forwarded message ----------
> >> From: Mothilal M <mo...@gmail.com>
> >> Date: Tue, Nov 8, 2016 at 1:25 PM
> >> Subject: Need Information - Apache NiFi
> >> To: dev@nifi.apache.org
> >>
> >>
> >> Hi Apache NiFi team,
> >>
> >> Greetings. I did my hands on experience with multiple tutorials around
> >> sentimental analysis integrating social media like twitter and facebook
> and
> >> also log analysis. Could you help me understand in what features Apache
> >> NiFi would act as an alternative to Splunk.
> >>
> >> I do have couple of technical clarifications in Apache NiFi - would it
> be
> >> possible to have a initial technical call to clarify my questions.
> >>
> >> Warm Regards,
> >> M. Mothilal
> >> Cell : +1(647)641-0910.
> >>
> >>
> >
>
--
Thanks,
Andrew
Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
Re: Need Information - Apache NiFi
Posted by Mothilal M <mo...@gmail.com>.
Hi Bryan Bende,
Greetings. I did try filter endpoint and it certainly needs numeric ID and
not twitter ID. Processor shows error if twitter ID - so I had converted it
to respective numeric. Please advice if there is any sample worked on this
use case - filtered tweets based on specific IDs.
Warm Regards,
M. Mothilal
On Tue, Nov 8, 2016 at 2:45 PM, Mothilal M <mo...@gmail.com> wrote:
> Hi Joe,
>
> Greetings and appreciate your reply. Totally agree with your opinion on
> comparisons.
>
> Look forward for other developers to comment on suggestions of getTwitter
> and getHTTP processor issues for twitter and facebook integration.
>
> Warm Regards,
> M. Mothilal
>
> On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <mo...@gmail.com> wrote:
>
>> Greetings Joe,
>>
>> Glad to connect with you from your reply in Apache NiFi community. I am
>> integrating NiFi with social media - twitter and Facebook to achieve
>> analysis but I am struck at a point. Request your assistance in solving the
>> same.
>>
>> getTwitter processor - i am not able to get tweets filtered by twitter
>> IDs. (IDs to follow field value is not working as expected. I did get the
>> numeric value of twitter IDs and input in the value as comma separated but
>> no results). Please advice as I look to filter my tweets on particular IDs
>> and also by particular location by cities like Toronto, calgary.
>>
>> getHTTP processor for facebook integration works fine but i am unable to
>> fetch data from my personal facebook ID. I tried accessing the Graph API
>> and it mentions user permissions are not granted but I did provide the
>> essential permissions in app settings. Please advice if you could help out
>> in setting the facebook app permissions to ensure I can fetch user_status,
>> user_location fields. On success, I would need to venture out few options
>> further.
>>
>> General question - How could Apache NiFi be useful in replacing Splunk. I
>> understand most of the current companies have a paid version of Splunk and
>> NiFi is an open-source but please let me know features which are key in
>> replacing Splunk.
>>
>> I appreciate your time and would be more helpful if you could let me know
>> your comfortable timing for a technical call discussion. I am available and
>> reachable on +1(647)641-0910. Look forwardly for your reply.
>>
>> Warm Regards,
>> M. Mothilal
>>
>> ------------------------------------------------------------
>> ------------------------------
>>
>> Hello,
>>
>> I think as a community we're all quite happy to help address questions
>> or ideas you might have for Apache NiFi. Please feel free to ask your
>> questions here and if they're more from the perspective of a user
>> please use the users@nifi.apache.org list.
>>
>> For the technical questions it is probably best to align them with
>> some portion of the overview, administration, user, best practice, or
>> developer guide so we can use that as a basis for further discussion.
>> You can find those here: https://nifi.apache.org/docs.html
>>
>> Thanks
>> Joe
>>
>>
>> ---------- Forwarded message ----------
>> From: Mothilal M <mo...@gmail.com>
>> Date: Tue, Nov 8, 2016 at 1:25 PM
>> Subject: Need Information - Apache NiFi
>> To: dev@nifi.apache.org
>>
>>
>> Hi Apache NiFi team,
>>
>> Greetings. I did my hands on experience with multiple tutorials around
>> sentimental analysis integrating social media like twitter and facebook and
>> also log analysis. Could you help me understand in what features Apache
>> NiFi would act as an alternative to Splunk.
>>
>> I do have couple of technical clarifications in Apache NiFi - would it be
>> possible to have a initial technical call to clarify my questions.
>>
>> Warm Regards,
>> M. Mothilal
>> Cell : +1(647)641-0910.
>>
>>
>
Re: Need Information - Apache NiFi
Posted by Mothilal M <mo...@gmail.com>.
Hi Joe,
Greetings and appreciate your reply. Totally agree with your opinion on
comparisons.
Look forward for other developers to comment on suggestions of getTwitter
and getHTTP processor issues for twitter and facebook integration.
Warm Regards,
M. Mothilal
On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <mo...@gmail.com> wrote:
> Greetings Joe,
>
> Glad to connect with you from your reply in Apache NiFi community. I am
> integrating NiFi with social media - twitter and Facebook to achieve
> analysis but I am struck at a point. Request your assistance in solving the
> same.
>
> getTwitter processor - i am not able to get tweets filtered by twitter
> IDs. (IDs to follow field value is not working as expected. I did get the
> numeric value of twitter IDs and input in the value as comma separated but
> no results). Please advice as I look to filter my tweets on particular IDs
> and also by particular location by cities like Toronto, calgary.
>
> getHTTP processor for facebook integration works fine but i am unable to
> fetch data from my personal facebook ID. I tried accessing the Graph API
> and it mentions user permissions are not granted but I did provide the
> essential permissions in app settings. Please advice if you could help out
> in setting the facebook app permissions to ensure I can fetch user_status,
> user_location fields. On success, I would need to venture out few options
> further.
>
> General question - How could Apache NiFi be useful in replacing Splunk. I
> understand most of the current companies have a paid version of Splunk and
> NiFi is an open-source but please let me know features which are key in
> replacing Splunk.
>
> I appreciate your time and would be more helpful if you could let me know
> your comfortable timing for a technical call discussion. I am available and
> reachable on +1(647)641-0910. Look forwardly for your reply.
>
> Warm Regards,
> M. Mothilal
>
> ------------------------------------------------------------
> ------------------------------
>
> Hello,
>
> I think as a community we're all quite happy to help address questions
> or ideas you might have for Apache NiFi. Please feel free to ask your
> questions here and if they're more from the perspective of a user
> please use the users@nifi.apache.org list.
>
> For the technical questions it is probably best to align them with
> some portion of the overview, administration, user, best practice, or
> developer guide so we can use that as a basis for further discussion.
> You can find those here: https://nifi.apache.org/docs.html
>
> Thanks
> Joe
>
>
> ---------- Forwarded message ----------
> From: Mothilal M <mo...@gmail.com>
> Date: Tue, Nov 8, 2016 at 1:25 PM
> Subject: Need Information - Apache NiFi
> To: dev@nifi.apache.org
>
>
> Hi Apache NiFi team,
>
> Greetings. I did my hands on experience with multiple tutorials around
> sentimental analysis integrating social media like twitter and facebook and
> also log analysis. Could you help me understand in what features Apache
> NiFi would act as an alternative to Splunk.
>
> I do have couple of technical clarifications in Apache NiFi - would it be
> possible to have a initial technical call to clarify my questions.
>
> Warm Regards,
> M. Mothilal
> Cell : +1(647)641-0910.
>
>
Re: Need Information - Apache NiFi
Posted by Mothilal M <mo...@gmail.com>.
Greetings Joe,
Glad to connect with you from your reply in Apache NiFi community. I am
integrating NiFi with social media - twitter and Facebook to achieve
analysis but I am struck at a point. Request your assistance in solving the
same.
getTwitter processor - i am not able to get tweets filtered by twitter IDs.
(IDs to follow field value is not working as expected. I did get the
numeric value of twitter IDs and input in the value as comma separated but
no results). Please advice as I look to filter my tweets on particular IDs
and also by particular location by cities like Toronto, calgary.
getHTTP processor for facebook integration works fine but i am unable to
fetch data from my personal facebook ID. I tried accessing the Graph API
and it mentions user permissions are not granted but I did provide the
essential permissions in app settings. Please advice if you could help out
in setting the facebook app permissions to ensure I can fetch user_status,
user_location fields. On success, I would need to venture out few options
further.
General question - How could Apache NiFi be useful in replacing Splunk. I
understand most of the current companies have a paid version of Splunk and
NiFi is an open-source but please let me know features which are key in
replacing Splunk.
I appreciate your time and would be more helpful if you could let me know
your comfortable timing for a technical call discussion. I am available and
reachable on +1(647)641-0910. Look forwardly for your reply.
Warm Regards,
M. Mothilal
------------------------------------------------------------------------------------------
Hello,
I think as a community we're all quite happy to help address questions
or ideas you might have for Apache NiFi. Please feel free to ask your
questions here and if they're more from the perspective of a user
please use the users@nifi.apache.org list.
For the technical questions it is probably best to align them with
some portion of the overview, administration, user, best practice, or
developer guide so we can use that as a basis for further discussion.
You can find those here: https://nifi.apache.org/docs.html
Thanks
Joe
---------- Forwarded message ----------
From: Mothilal M <mo...@gmail.com>
Date: Tue, Nov 8, 2016 at 1:25 PM
Subject: Need Information - Apache NiFi
To: dev@nifi.apache.org
Hi Apache NiFi team,
Greetings. I did my hands on experience with multiple tutorials around
sentimental analysis integrating social media like twitter and facebook and
also log analysis. Could you help me understand in what features Apache
NiFi would act as an alternative to Splunk.
I do have couple of technical clarifications in Apache NiFi - would it be
possible to have a initial technical call to clarify my questions.
Warm Regards,
M. Mothilal
Cell : +1(647)641-0910.
Re: Need Information - Apache NiFi
Posted by Joe Witt <jo...@gmail.com>.
Hello,
I think as a community we're all quite happy to help address questions
or ideas you might have for Apache NiFi. Please feel free to ask your
questions here and if they're more from the perspective of a user
please use the users@nifi.apache.org list.
For the technical questions it is probably best to align them with
some portion of the overview, administration, user, best practice, or
developer guide so we can use that as a basis for further discussion.
You can find those here: https://nifi.apache.org/docs.html
Thanks
Joe
On Tue, Nov 8, 2016 at 1:25 PM, Mothilal M <mo...@gmail.com> wrote:
> Hi Apache NiFi team,
>
> Greetings. I did my hands on experience with multiple tutorials around
> sentimental analysis integrating social media like twitter and facebook and
> also log analysis. Could you help me understand in what features Apache
> NiFi would act as an alternative to Splunk.
>
> I do have couple of technical clarifications in Apache NiFi - would it be
> possible to have a initial technical call to clarify my questions.
>
> Warm Regards,
> M. Mothilal
> Cell : +1(647)641-0910.