You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Selvam Raman <se...@gmail.com> on 2016/09/13 12:22:35 UTC
Nifi - GetHTTP and EvaluateXquery
Hi ,
Need your help techies.
I am using GetHttp to pull the data and it is coming as xml file.(have
mentioned the url and filename).
want to extract field value from the xml using Evaluate Xquery.
I have tried this two but nothing is working
1) for $x in doc("selmna.xml")/OAI-PMH/ListSets/set
return data($x/setSpec)
2)//setSpec or /OAI-PMH/ListSets/set/setSpec/text()
Can you please help me, what am i missing here.
--
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
Re: Nifi - GetHTTP and EvaluateXquery
Posted by Selvam Raman <se...@gmail.com>.
Hi Mark and matt,
thanks for the help.
How can i get all the values in single file.
Nifi flow:
GetHttp, EvaluateXquery and Putfile
In putfile i am getting error because the above query executes the number
of childs presents. In our example 2 times. So each time it trying to
create same filename.
How can i get the all the records into a single file.
GetHttp:
Url :*********
Filename: test.xml
EvaluateXquery:
/*:OAI-PMH/*:ListSets/*:set/*:setSpec/text()
putFile:
C:\Downloads\Nifi\
i am getting file as
C:\Downloads\Nifi\test.xml: this file currently contains only one record.
On Wed, Sep 14, 2016 at 1:56 AM, Matt Burgess <ma...@gmail.com> wrote:
> Selvam,
>
> Looks like you're using a default namespace
> (xmlns="http://www.openarchives.org/OAI/2.0/"), this makes the XPath
> querying more difficult as you have to use local-name(), as described
> in a NiFi email thread here:
>
> http://mail-archives.us.apache.org/mod_mbox/incubator-
> nifi-users/201605.mbox/%3CCAEV8zdX5VDOZHyaNi_
> pB1mLGpmA4dDSEnczwd62k1QkN8Ag3xQ@mail.gmail.com%3E
>
> Regards,
> Matt
>
> On Tue, Sep 13, 2016 at 4:09 PM, Selvam Raman <se...@gmail.com> wrote:
> > This is the sample xml
> >
> > <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="
> > http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
> > http://www.openarchives.org/OAI/2.0/
> > http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
> > <responseDate>2016-09-13T20:05:49Z</responseDate>
> > <request verb="ListSets">http://testing</request>
> > <ListSets>
> > <set>
> > <setSpec>h10255_dryad.148</setSpec>
> > <setName>test1</setName>
> > </set>
> > <set>
> > <setSpec>10255_2</setSpec>
> > <setName>test2</setName>
> > </set>
> > </ListSets>
> > </OAI-PMH>
> >
> > Thanks,
> > RS
> >
> > On Tue, Sep 13, 2016 at 6:44 PM, Mark Payne <ma...@hotmail.com>
> wrote:
> >
> >> Selvam,
> >>
> >> Can you share a sample of your XML? It's hard to tell why it's not
> working
> >> without the actual
> >> data that you're trying to run this against. I can tell you though, that
> >> you won't have much luck
> >> using the doc() function. You'd instead want to just omit it all
> together
> >> and use /OAI-PMH/...
> >>
> >> I tried using the XQuery "/OAI-PMH/ListSets/set/setSpec/text()" against
> >> the following XML:
> >>
> >> <OAI-PMH>
> >> <ListSets>
> >> <set>
> >> <setSpec>test</setSpec>
> >> </set>
> >> </ListSets>
> >> </OAI-PMH>
> >>
> >> I got back the expected result of simply "test".
> >>
> >> With a sample that is more representative of your actual XML I may be
> able
> >> to offer more assistance.
> >>
> >> Thanks
> >> -Mark
> >>
> >> > On Sep 13, 2016, at 8:22 AM, Selvam Raman <se...@gmail.com> wrote:
> >> >
> >> > Hi ,
> >> >
> >> > Need your help techies.
> >> >
> >> > I am using GetHttp to pull the data and it is coming as xml file.(have
> >> > mentioned the url and filename).
> >> >
> >> > want to extract field value from the xml using Evaluate Xquery.
> >> >
> >> > I have tried this two but nothing is working
> >> >
> >> > 1) for $x in doc("selmna.xml")/OAI-PMH/ListSets/set
> >> > return data($x/setSpec)
> >> >
> >> > 2)//setSpec or /OAI-PMH/ListSets/set/setSpec/text()
> >> >
> >> > Can you please help me, what am i missing here.
> >> >
> >> > --
> >> > Selvam Raman
> >> > "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
> >>
> >>
> >
> >
> > --
> > Selvam Raman
> > "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>
--
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
Re: Nifi - GetHTTP and EvaluateXquery
Posted by Matt Burgess <ma...@gmail.com>.
Selvam,
Looks like you're using a default namespace
(xmlns="http://www.openarchives.org/OAI/2.0/"), this makes the XPath
querying more difficult as you have to use local-name(), as described
in a NiFi email thread here:
http://mail-archives.us.apache.org/mod_mbox/incubator-nifi-users/201605.mbox/%3CCAEV8zdX5VDOZHyaNi_pB1mLGpmA4dDSEnczwd62k1QkN8Ag3xQ@mail.gmail.com%3E
Regards,
Matt
On Tue, Sep 13, 2016 at 4:09 PM, Selvam Raman <se...@gmail.com> wrote:
> This is the sample xml
>
> <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
> http://www.openarchives.org/OAI/2.0/
> http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
> <responseDate>2016-09-13T20:05:49Z</responseDate>
> <request verb="ListSets">http://testing</request>
> <ListSets>
> <set>
> <setSpec>h10255_dryad.148</setSpec>
> <setName>test1</setName>
> </set>
> <set>
> <setSpec>10255_2</setSpec>
> <setName>test2</setName>
> </set>
> </ListSets>
> </OAI-PMH>
>
> Thanks,
> RS
>
> On Tue, Sep 13, 2016 at 6:44 PM, Mark Payne <ma...@hotmail.com> wrote:
>
>> Selvam,
>>
>> Can you share a sample of your XML? It's hard to tell why it's not working
>> without the actual
>> data that you're trying to run this against. I can tell you though, that
>> you won't have much luck
>> using the doc() function. You'd instead want to just omit it all together
>> and use /OAI-PMH/...
>>
>> I tried using the XQuery "/OAI-PMH/ListSets/set/setSpec/text()" against
>> the following XML:
>>
>> <OAI-PMH>
>> <ListSets>
>> <set>
>> <setSpec>test</setSpec>
>> </set>
>> </ListSets>
>> </OAI-PMH>
>>
>> I got back the expected result of simply "test".
>>
>> With a sample that is more representative of your actual XML I may be able
>> to offer more assistance.
>>
>> Thanks
>> -Mark
>>
>> > On Sep 13, 2016, at 8:22 AM, Selvam Raman <se...@gmail.com> wrote:
>> >
>> > Hi ,
>> >
>> > Need your help techies.
>> >
>> > I am using GetHttp to pull the data and it is coming as xml file.(have
>> > mentioned the url and filename).
>> >
>> > want to extract field value from the xml using Evaluate Xquery.
>> >
>> > I have tried this two but nothing is working
>> >
>> > 1) for $x in doc("selmna.xml")/OAI-PMH/ListSets/set
>> > return data($x/setSpec)
>> >
>> > 2)//setSpec or /OAI-PMH/ListSets/set/setSpec/text()
>> >
>> > Can you please help me, what am i missing here.
>> >
>> > --
>> > Selvam Raman
>> > "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>>
>>
>
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
Re: Nifi - GetHTTP and EvaluateXquery
Posted by Selvam Raman <se...@gmail.com>.
Hi Mark,
Thanks for the help.
The above query executes each and every child node ,i guess. Suppose if
were 6 records it executes 6 times. Nifi flow ( Gethttp, evaluatexquery,
putfile).
GetHTTP(URL,FileName) , EvaluateXquery(result =
/*:OAI-PMH/*:ListSets/*:set/*:setSpec/text()) and finally i want to store
the data in local file system and using putfile(Directory :
C:/Users/selvam_r01/Downloads/Nifi/${Filename}/${result}).
Now it throws filename exists already it receives gethttp filename as a
filename.
Case 1:
How can i get the all the values in a single putfile
how can i get each record in a single putfile .
On Wed, Sep 14, 2016 at 1:55 AM, Mark Payne <ma...@hotmail.com> wrote:
> Thanks, Selvam.
>
> The issue that I think you're having here is that your XML is using
> namespace, but your
> XQuery does not use any namespaces. The XQuery does support using a
> wildcard for
> the namespace. So when I changed the XQuery to use the wildcard:
>
> /*:OAI-PMH/*:ListSets/*:set/*:setSpec/text()
>
> I get back two results:
>
> h10255_dryad.148
> 10255_2
>
> Does this give you what you need?
>
> Thanks
> -Mark
>
> > On Sep 13, 2016, at 4:09 PM, Selvam Raman <se...@gmail.com> wrote:
> >
> > This is the sample xml
> >
> > <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="
> > http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
> > http://www.openarchives.org/OAI/2.0/
> > http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
> > <responseDate>2016-09-13T20:05:49Z</responseDate>
> > <request verb="ListSets">http://testing</request>
> > <ListSets>
> > <set>
> > <setSpec>h10255_dryad.148</setSpec>
> > <setName>test1</setName>
> > </set>
> > <set>
> > <setSpec>10255_2</setSpec>
> > <setName>test2</setName>
> > </set>
> > </ListSets>
> > </OAI-PMH>
> >
> > Thanks,
> > RS
> >
> > On Tue, Sep 13, 2016 at 6:44 PM, Mark Payne <ma...@hotmail.com>
> wrote:
> >
> >> Selvam,
> >>
> >> Can you share a sample of your XML? It's hard to tell why it's not
> working
> >> without the actual
> >> data that you're trying to run this against. I can tell you though, that
> >> you won't have much luck
> >> using the doc() function. You'd instead want to just omit it all
> together
> >> and use /OAI-PMH/...
> >>
> >> I tried using the XQuery "/OAI-PMH/ListSets/set/setSpec/text()" against
> >> the following XML:
> >>
> >> <OAI-PMH>
> >> <ListSets>
> >> <set>
> >> <setSpec>test</setSpec>
> >> </set>
> >> </ListSets>
> >> </OAI-PMH>
> >>
> >> I got back the expected result of simply "test".
> >>
> >> With a sample that is more representative of your actual XML I may be
> able
> >> to offer more assistance.
> >>
> >> Thanks
> >> -Mark
> >>
> >>> On Sep 13, 2016, at 8:22 AM, Selvam Raman <se...@gmail.com> wrote:
> >>>
> >>> Hi ,
> >>>
> >>> Need your help techies.
> >>>
> >>> I am using GetHttp to pull the data and it is coming as xml file.(have
> >>> mentioned the url and filename).
> >>>
> >>> want to extract field value from the xml using Evaluate Xquery.
> >>>
> >>> I have tried this two but nothing is working
> >>>
> >>> 1) for $x in doc("selmna.xml")/OAI-PMH/ListSets/set
> >>> return data($x/setSpec)
> >>>
> >>> 2)//setSpec or /OAI-PMH/ListSets/set/setSpec/text()
> >>>
> >>> Can you please help me, what am i missing here.
> >>>
> >>> --
> >>> Selvam Raman
> >>> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
> >>
> >>
> >
> >
> > --
> > Selvam Raman
> > "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>
>
--
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
Re: Nifi - GetHTTP and EvaluateXquery
Posted by Mark Payne <ma...@hotmail.com>.
Thanks, Selvam.
The issue that I think you're having here is that your XML is using namespace, but your
XQuery does not use any namespaces. The XQuery does support using a wildcard for
the namespace. So when I changed the XQuery to use the wildcard:
/*:OAI-PMH/*:ListSets/*:set/*:setSpec/text()
I get back two results:
h10255_dryad.148
10255_2
Does this give you what you need?
Thanks
-Mark
> On Sep 13, 2016, at 4:09 PM, Selvam Raman <se...@gmail.com> wrote:
>
> This is the sample xml
>
> <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
> http://www.openarchives.org/OAI/2.0/
> http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
> <responseDate>2016-09-13T20:05:49Z</responseDate>
> <request verb="ListSets">http://testing</request>
> <ListSets>
> <set>
> <setSpec>h10255_dryad.148</setSpec>
> <setName>test1</setName>
> </set>
> <set>
> <setSpec>10255_2</setSpec>
> <setName>test2</setName>
> </set>
> </ListSets>
> </OAI-PMH>
>
> Thanks,
> RS
>
> On Tue, Sep 13, 2016 at 6:44 PM, Mark Payne <ma...@hotmail.com> wrote:
>
>> Selvam,
>>
>> Can you share a sample of your XML? It's hard to tell why it's not working
>> without the actual
>> data that you're trying to run this against. I can tell you though, that
>> you won't have much luck
>> using the doc() function. You'd instead want to just omit it all together
>> and use /OAI-PMH/...
>>
>> I tried using the XQuery "/OAI-PMH/ListSets/set/setSpec/text()" against
>> the following XML:
>>
>> <OAI-PMH>
>> <ListSets>
>> <set>
>> <setSpec>test</setSpec>
>> </set>
>> </ListSets>
>> </OAI-PMH>
>>
>> I got back the expected result of simply "test".
>>
>> With a sample that is more representative of your actual XML I may be able
>> to offer more assistance.
>>
>> Thanks
>> -Mark
>>
>>> On Sep 13, 2016, at 8:22 AM, Selvam Raman <se...@gmail.com> wrote:
>>>
>>> Hi ,
>>>
>>> Need your help techies.
>>>
>>> I am using GetHttp to pull the data and it is coming as xml file.(have
>>> mentioned the url and filename).
>>>
>>> want to extract field value from the xml using Evaluate Xquery.
>>>
>>> I have tried this two but nothing is working
>>>
>>> 1) for $x in doc("selmna.xml")/OAI-PMH/ListSets/set
>>> return data($x/setSpec)
>>>
>>> 2)//setSpec or /OAI-PMH/ListSets/set/setSpec/text()
>>>
>>> Can you please help me, what am i missing here.
>>>
>>> --
>>> Selvam Raman
>>> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>>
>>
>
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
Re: Nifi - GetHTTP and EvaluateXquery
Posted by Selvam Raman <se...@gmail.com>.
This is the sample xml
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.openarchives.org/OAI/2.0/
http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
<responseDate>2016-09-13T20:05:49Z</responseDate>
<request verb="ListSets">http://testing</request>
<ListSets>
<set>
<setSpec>h10255_dryad.148</setSpec>
<setName>test1</setName>
</set>
<set>
<setSpec>10255_2</setSpec>
<setName>test2</setName>
</set>
</ListSets>
</OAI-PMH>
Thanks,
RS
On Tue, Sep 13, 2016 at 6:44 PM, Mark Payne <ma...@hotmail.com> wrote:
> Selvam,
>
> Can you share a sample of your XML? It's hard to tell why it's not working
> without the actual
> data that you're trying to run this against. I can tell you though, that
> you won't have much luck
> using the doc() function. You'd instead want to just omit it all together
> and use /OAI-PMH/...
>
> I tried using the XQuery "/OAI-PMH/ListSets/set/setSpec/text()" against
> the following XML:
>
> <OAI-PMH>
> <ListSets>
> <set>
> <setSpec>test</setSpec>
> </set>
> </ListSets>
> </OAI-PMH>
>
> I got back the expected result of simply "test".
>
> With a sample that is more representative of your actual XML I may be able
> to offer more assistance.
>
> Thanks
> -Mark
>
> > On Sep 13, 2016, at 8:22 AM, Selvam Raman <se...@gmail.com> wrote:
> >
> > Hi ,
> >
> > Need your help techies.
> >
> > I am using GetHttp to pull the data and it is coming as xml file.(have
> > mentioned the url and filename).
> >
> > want to extract field value from the xml using Evaluate Xquery.
> >
> > I have tried this two but nothing is working
> >
> > 1) for $x in doc("selmna.xml")/OAI-PMH/ListSets/set
> > return data($x/setSpec)
> >
> > 2)//setSpec or /OAI-PMH/ListSets/set/setSpec/text()
> >
> > Can you please help me, what am i missing here.
> >
> > --
> > Selvam Raman
> > "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>
>
--
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
Re: Nifi - GetHTTP and EvaluateXquery
Posted by Mark Payne <ma...@hotmail.com>.
Selvam,
Can you share a sample of your XML? It's hard to tell why it's not working without the actual
data that you're trying to run this against. I can tell you though, that you won't have much luck
using the doc() function. You'd instead want to just omit it all together and use /OAI-PMH/...
I tried using the XQuery "/OAI-PMH/ListSets/set/setSpec/text()" against the following XML:
<OAI-PMH>
<ListSets>
<set>
<setSpec>test</setSpec>
</set>
</ListSets>
</OAI-PMH>
I got back the expected result of simply "test".
With a sample that is more representative of your actual XML I may be able to offer more assistance.
Thanks
-Mark
> On Sep 13, 2016, at 8:22 AM, Selvam Raman <se...@gmail.com> wrote:
>
> Hi ,
>
> Need your help techies.
>
> I am using GetHttp to pull the data and it is coming as xml file.(have
> mentioned the url and filename).
>
> want to extract field value from the xml using Evaluate Xquery.
>
> I have tried this two but nothing is working
>
> 1) for $x in doc("selmna.xml")/OAI-PMH/ListSets/set
> return data($x/setSpec)
>
> 2)//setSpec or /OAI-PMH/ListSets/set/setSpec/text()
>
> Can you please help me, what am i missing here.
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"