You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Rick Braddy <rb...@softnas.com> on 2015/08/22 07:09:59 UTC
ExecuteXQuery only returns single item when there are multiple
matching items in FlowFile
I have an XQuery processor test app configured to filter for <item> entries from an RSS feed where the <title> tag contains the word "Trump". The XQuery is properly formed, based upon testing it outside of NiFi using XPathTester<http://www.xpathtester.com/xquery>. Is this a bug in XQuery processor or am I doing something wrong?
Here's the XQuery: //item[descendant::title[contains(text(),'Trump')]]
It properly matches only the first entry instead of matching all of the entries with "Trump" in the title.
[cid:image001.png@01D0DC6E.E29C1EE0]
I'm attaching a sample RSS news feed file that has numerous Trump articles in it (this file was created by a PutFile that wrote the entire file out ahead of the XQuery block)
Thanks
Rick
Re: ExecuteXQuery only returns single item when there are multiple
matching items in FlowFile
Posted by Joe Witt <jo...@gmail.com>.
Rick
Excellent. Just route the output to an UpdateAttribute processor.
Set a property on that processor with a name of 'filename' and give it a
value of (for example):
${filename}-${uuid}
That would append a hyphen and a uuid to the name. You can read more about
all the wacky stuff the expression language will let you do in the
help->Expression Language guide. And of course you can always ask us here.
Thanks
Joe
On Sat, Aug 22, 2015 at 4:28 AM, Rick Braddy <rb...@softnas.com> wrote:
> Joe,
>
>
>
> Thanks for the quick responses.
>
>
>
> I figured it out. ExecuteXQuery is indeed spitting out 9 separate
> FlowFiles, but since it flows into PutFile (and originated from a GetFile),
> it has a single static filename, so each one is overwriting the final
> output file. Doh!
>
>
>
> Looks like a NiFi noob trick. Sorry to waste your time chasing this one.
>
>
>
> Now I just need to figure out a way to generate a unique output filename
> attribute for each PutFile.
>
>
>
> Best,
>
> Rick
>
>
>
> *From:* Joe Witt [mailto:joe.witt@gmail.com]
> *Sent:* Saturday, August 22, 2015 12:45 AM
> *To:* users@nifi.apache.org
> *Subject:* Re: ExecuteXQuery only returns single item when there are
> multiple matching items in FlowFile
>
>
>
> Based on reading the docs for that processor I'm now with you on expected
> behavior...
>
>
>
> On Sat, Aug 22, 2015 at 1:33 AM, Joe Witt <jo...@gmail.com> wrote:
>
> Rick,
>
>
>
> Quick initial response...
>
>
>
> I think the behavior makes sense because if it did just match all the
> items with desired titles you're left with blocks of valid XML but as a
> whole an invalid XML document.
>
>
>
> What I believe is the expected behavior then is to use SplitXML to split
> out the item elements. Then you can run each of those through what you
> have configured here.
>
>
>
> With this approach you end up with valid XML in every case.
>
>
>
> I'll put together a template to verify.
>
>
>
> Thanks
>
> Joe
>
>
>
> On Sat, Aug 22, 2015 at 1:09 AM, Rick Braddy <rb...@softnas.com> wrote:
>
> I have an XQuery processor test app configured to filter for <item>
> entries from an RSS feed where the <title> tag contains the word “Trump”.
> The XQuery is properly formed, based upon testing it outside of NiFi using
> XPathTester <http://www.xpathtester.com/xquery>. Is this a bug in XQuery
> processor or am I doing something wrong?
>
>
>
> Here’s the XQuery: *//item[descendant::title[contains(text(),'Trump')]]*
>
>
>
> It properly matches only the first entry instead of matching all of the
> entries with “Trump” in the title.
>
>
>
>
>
> I’m attaching a sample RSS news feed file that has numerous Trump articles
> in it (this file was created by a PutFile that wrote the entire file out
> ahead of the XQuery block)
>
>
>
> Thanks
>
> Rick
>
>
>
>
>
>
>
RE: ExecuteXQuery only returns single item when there are multiple
matching items in FlowFile
Posted by Rick Braddy <rb...@softnas.com>.
Joe,
Thanks for the quick responses.
I figured it out. ExecuteXQuery is indeed spitting out 9 separate FlowFiles, but since it flows into PutFile (and originated from a GetFile), it has a single static filename, so each one is overwriting the final output file. Doh!
Looks like a NiFi noob trick. Sorry to waste your time chasing this one.
Now I just need to figure out a way to generate a unique output filename attribute for each PutFile.
Best,
Rick
From: Joe Witt [mailto:joe.witt@gmail.com]
Sent: Saturday, August 22, 2015 12:45 AM
To: users@nifi.apache.org
Subject: Re: ExecuteXQuery only returns single item when there are multiple matching items in FlowFile
Based on reading the docs for that processor I'm now with you on expected behavior...
On Sat, Aug 22, 2015 at 1:33 AM, Joe Witt <jo...@gmail.com>> wrote:
Rick,
Quick initial response...
I think the behavior makes sense because if it did just match all the items with desired titles you're left with blocks of valid XML but as a whole an invalid XML document.
What I believe is the expected behavior then is to use SplitXML to split out the item elements. Then you can run each of those through what you have configured here.
With this approach you end up with valid XML in every case.
I'll put together a template to verify.
Thanks
Joe
On Sat, Aug 22, 2015 at 1:09 AM, Rick Braddy <rb...@softnas.com>> wrote:
I have an XQuery processor test app configured to filter for <item> entries from an RSS feed where the <title> tag contains the word “Trump”. The XQuery is properly formed, based upon testing it outside of NiFi using XPathTester<http://www.xpathtester.com/xquery>. Is this a bug in XQuery processor or am I doing something wrong?
Here’s the XQuery: //item[descendant::title[contains(text(),'Trump')]]
It properly matches only the first entry instead of matching all of the entries with “Trump” in the title.
[cid:image001.png@01D0DC8A.A8BCD6D0]
I’m attaching a sample RSS news feed file that has numerous Trump articles in it (this file was created by a PutFile that wrote the entire file out ahead of the XQuery block)
Thanks
Rick
RE: ExecuteXQuery only returns single item when there are multiple
matching items in FlowFile
Posted by Rick Braddy <rb...@softnas.com>.
I was able to resolve the issue by adding an UpdateAttribute processor after ExecuteXQuery block and before the PutFile to update “filename” attribute to “${filename}.${uuid}”, which works perfectly.
Issue resolved. Lessons learned.
Best,
Rick
From: Rick Braddy
Sent: Saturday, August 22, 2015 3:29 AM
To: users@nifi.apache.org
Subject: RE: ExecuteXQuery only returns single item when there are multiple matching items in FlowFile
Joe,
Thanks for the quick responses.
I figured it out. ExecuteXQuery is indeed spitting out 9 separate FlowFiles, but since it flows into PutFile (and originated from a GetFile), it has a single static filename, so each one is overwriting the final output file. Doh!
Looks like a NiFi noob trick. Sorry to waste your time chasing this one.
Now I just need to figure out a way to generate a unique output filename attribute for each PutFile.
Best,
Rick
From: Joe Witt [mailto:joe.witt@gmail.com]
Sent: Saturday, August 22, 2015 12:45 AM
To: users@nifi.apache.org<ma...@nifi.apache.org>
Subject: Re: ExecuteXQuery only returns single item when there are multiple matching items in FlowFile
Based on reading the docs for that processor I'm now with you on expected behavior...
On Sat, Aug 22, 2015 at 1:33 AM, Joe Witt <jo...@gmail.com>> wrote:
Rick,
Quick initial response...
I think the behavior makes sense because if it did just match all the items with desired titles you're left with blocks of valid XML but as a whole an invalid XML document.
What I believe is the expected behavior then is to use SplitXML to split out the item elements. Then you can run each of those through what you have configured here.
With this approach you end up with valid XML in every case.
I'll put together a template to verify.
Thanks
Joe
On Sat, Aug 22, 2015 at 1:09 AM, Rick Braddy <rb...@softnas.com>> wrote:
I have an XQuery processor test app configured to filter for <item> entries from an RSS feed where the <title> tag contains the word “Trump”. The XQuery is properly formed, based upon testing it outside of NiFi using XPathTester<http://www.xpathtester.com/xquery>. Is this a bug in XQuery processor or am I doing something wrong?
Here’s the XQuery: //item[descendant::title[contains(text(),'Trump')]]
It properly matches only the first entry instead of matching all of the entries with “Trump” in the title.
[cid:image001.png@01D0DC8C.26401350]
I’m attaching a sample RSS news feed file that has numerous Trump articles in it (this file was created by a PutFile that wrote the entire file out ahead of the XQuery block)
Thanks
Rick
Re: ExecuteXQuery only returns single item when there are multiple
matching items in FlowFile
Posted by Joe Witt <jo...@gmail.com>.
Based on reading the docs for that processor I'm now with you on expected
behavior...
On Sat, Aug 22, 2015 at 1:33 AM, Joe Witt <jo...@gmail.com> wrote:
> Rick,
>
> Quick initial response...
>
> I think the behavior makes sense because if it did just match all the
> items with desired titles you're left with blocks of valid XML but as a
> whole an invalid XML document.
>
> What I believe is the expected behavior then is to use SplitXML to split
> out the item elements. Then you can run each of those through what you
> have configured here.
>
> With this approach you end up with valid XML in every case.
>
> I'll put together a template to verify.
>
> Thanks
> Joe
>
> On Sat, Aug 22, 2015 at 1:09 AM, Rick Braddy <rb...@softnas.com> wrote:
>
>> I have an XQuery processor test app configured to filter for <item>
>> entries from an RSS feed where the <title> tag contains the word “Trump”.
>> The XQuery is properly formed, based upon testing it outside of NiFi using
>> XPathTester <http://www.xpathtester.com/xquery>. Is this a bug in
>> XQuery processor or am I doing something wrong?
>>
>>
>>
>> Here’s the XQuery: *//item[descendant::title[contains(text(),'Trump')]]*
>>
>>
>>
>> It properly matches only the first entry instead of matching all of the
>> entries with “Trump” in the title.
>>
>>
>>
>>
>>
>> I’m attaching a sample RSS news feed file that has numerous Trump
>> articles in it (this file was created by a PutFile that wrote the entire
>> file out ahead of the XQuery block)
>>
>>
>>
>> Thanks
>>
>> Rick
>>
>>
>>
>
>
Re: ExecuteXQuery only returns single item when there are multiple
matching items in FlowFile
Posted by Joe Witt <jo...@gmail.com>.
Rick,
Quick initial response...
I think the behavior makes sense because if it did just match all the items
with desired titles you're left with blocks of valid XML but as a whole an
invalid XML document.
What I believe is the expected behavior then is to use SplitXML to split
out the item elements. Then you can run each of those through what you
have configured here.
With this approach you end up with valid XML in every case.
I'll put together a template to verify.
Thanks
Joe
On Sat, Aug 22, 2015 at 1:09 AM, Rick Braddy <rb...@softnas.com> wrote:
> I have an XQuery processor test app configured to filter for <item>
> entries from an RSS feed where the <title> tag contains the word “Trump”.
> The XQuery is properly formed, based upon testing it outside of NiFi using
> XPathTester <http://www.xpathtester.com/xquery>. Is this a bug in XQuery
> processor or am I doing something wrong?
>
>
>
> Here’s the XQuery: *//item[descendant::title[contains(text(),'Trump')]]*
>
>
>
> It properly matches only the first entry instead of matching all of the
> entries with “Trump” in the title.
>
>
>
>
>
> I’m attaching a sample RSS news feed file that has numerous Trump articles
> in it (this file was created by a PutFile that wrote the entire file out
> ahead of the XQuery block)
>
>
>
> Thanks
>
> Rick
>
>
>