You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Hong Li <ho...@centricconsulting.com> on 2016/03/29 10:30:30 UTC
EvaluateJsonPath and Json Field Name Starting with @ as the First Character
How can I assign the value of a Json field that has a name starting with @
as the first character of its name to a user-defined-property in
EvaluateJsonPath?
For example, I have the following sample Json message:
{
"effectiveTimestamp": "2015-12-03T23:17:29.874Z",
"event": {
> "@class": "events.policy.PolicyCreated",
"id": "1e9b91398160471f8b6197ad974e2464",
"ipAddress": "10.8.30.145",
"policy": {
"additionalListedInsureds": [],
"address": {
"city": "Skokie",
"county": "Cook",
"id": "b863190a5bf846858eb372fb5f532fe7",
"latitude": 42.0101,
"longitude": -87.75354,
"state": "IL",
"street": "5014 Estes Ave",
"zip": "60077-3520"
},
"applicant": {
"age": 36,
"birthDate": "1979-01-12",
"clientId": "191",
"creditReport": {
"id": "ca5ec932d33d444b880c9a43a6eb7c50",
"reasons": [],
"referenceNumber": "15317191300474",
"status": "NoHit"
},
"firstName": "Kathy",
"gender": "Female",
"id": "1f4a3862fab54e058305e3c73cc13dd3",
"lastName": "Bockett",
"maritalStatus": "Single",
"middleName": "Sue",
"ssn": "*******"
},
"channelOfOrigin": "PublicWebsite",
... ...
Under EvaluateJsonPath, I can make
eventID = $.event.id
But I found out that if I make
eventClass = $.event.@class
the value of @class will never be assigned to eventClass.
How should we capture the value of such a Json field?
Thanks.
Hong
*Hong Li*
*Centric Consulting*
*In Balance*
(888) 781-7567 office
(614) 296-7644 mobile
www.centricconsulting.com | @Centric <https://twitter.com/centric>
Re: EvaluateJsonPath and Json Field Name Starting with @ as the First Character
Posted by Hong Li <ho...@centricconsulting.com>.
Thanks, Matt. We installed Nifi 0.6.0.
Hong
*Hong Li*
*Centric Consulting*
*In Balance*
(888) 781-7567 office
(614) 296-7644 mobile
www.centricconsulting.com | @Centric <https://twitter.com/centric>
On Sun, Apr 3, 2016 at 9:19 PM, Matt Burgess <ma...@gmail.com> wrote:
> Hong,
>
> At this point you have the choice of going to Apache NiFi 0.6.0 or
> upgrading to HDF 1.2 which has NiFi 0.6.0 as well:
> http://docs.hortonworks.com/HDPDocuments/HDF1/HDF-1.2/bk_HDF_RelNotes/content/ch_hdf_relnotes.html#release_summary
>
> Regards,
> Matt
>
> On Tue, Mar 29, 2016 at 9:26 AM, Hong Li <ho...@centricconsulting.com>
> wrote:
>
>> Thanks, Matt. Out version is 0.5.1. We have been using HDF. But we are
>> going to upgrade to 0.6.0 from Apache directly this time.
>>
>> Hong
>>
>>
>>
>> *Hong Li*
>>
>> *Centric Consulting*
>>
>> *In Balance*
>> (888) 781-7567 office
>> (614) 296-7644 mobile
>> www.centricconsulting.com | @Centric <https://twitter.com/centric>
>>
>> On Tue, Mar 29, 2016 at 8:44 AM, Matt Burgess <ma...@gmail.com>
>> wrote:
>>
>>> Hong,
>>>
>>> I was able to use EvaluateJsonPath with eventClass $.event.@class and
>>> the attribute had the correct value (see output from LogAttribute below):
>>>
>>> --------------------------------------------------
>>> Standard FlowFile Attributes
>>> Key: 'entryDate'
>>> Value: 'Tue Mar 29 08:41:36 EDT 2016'
>>> Key: 'lineageStartDate'
>>> Value: 'Tue Mar 29 08:41:36 EDT 2016'
>>> Key: 'fileSize'
>>> Value: '932'
>>> FlowFile Attribute Map Content
>>> Key: 'eventClass'
>>> Value: 'events.policy.PolicyCreated'
>>> Key: 'filename'
>>> Value: 'test.json'
>>> Key: 'path'
>>> Value: './'
>>> Key: 'uuid'
>>> Value: '174e1add-0411-41a4-b7f5-7a43360ad104'
>>> --------------------------------------------------
>>>
>>> What version of NiFi are you using? I used 0.6.0 to get the above
>>> results, perhaps there was a bug that was fixed?
>>>
>>> Regards,
>>> Matt
>>>
>>>
>>> On Tue, Mar 29, 2016 at 4:30 AM, Hong Li <ho...@centricconsulting.com>
>>> wrote:
>>>
>>>> How can I assign the value of a Json field that has a name starting
>>>> with @ as the first character of its name to a user-defined-property in
>>>> EvaluateJsonPath?
>>>>
>>>> For example, I have the following sample Json message:
>>>>
>>>> {
>>>> "effectiveTimestamp": "2015-12-03T23:17:29.874Z",
>>>> "event": {
>>>>
>>>>> "@class": "events.policy.PolicyCreated",
>>>>
>>>> "id": "1e9b91398160471f8b6197ad974e2464",
>>>> "ipAddress": "10.8.30.145",
>>>> "policy": {
>>>> "additionalListedInsureds": [],
>>>> "address": {
>>>> "city": "Skokie",
>>>> "county": "Cook",
>>>> "id": "b863190a5bf846858eb372fb5f532fe7",
>>>> "latitude": 42.0101,
>>>> "longitude": -87.75354,
>>>> "state": "IL",
>>>> "street": "5014 Estes Ave",
>>>> "zip": "60077-3520"
>>>> },
>>>> "applicant": {
>>>> "age": 36,
>>>> "birthDate": "1979-01-12",
>>>> "clientId": "191",
>>>> "creditReport": {
>>>> "id": "ca5ec932d33d444b880c9a43a6eb7c50",
>>>> "reasons": [],
>>>> "referenceNumber": "15317191300474",
>>>> "status": "NoHit"
>>>> },
>>>> "firstName": "Kathy",
>>>> "gender": "Female",
>>>> "id": "1f4a3862fab54e058305e3c73cc13dd3",
>>>> "lastName": "Bockett",
>>>> "maritalStatus": "Single",
>>>> "middleName": "Sue",
>>>> "ssn": "*******"
>>>> },
>>>> "channelOfOrigin": "PublicWebsite",
>>>> ... ...
>>>>
>>>> Under EvaluateJsonPath, I can make
>>>>
>>>> eventID = $.event.id
>>>>
>>>>
>>>> But I found out that if I make
>>>>
>>>> eventClass = $.event.@class
>>>>
>>>> the value of @class will never be assigned to eventClass.
>>>>
>>>> How should we capture the value of such a Json field?
>>>>
>>>> Thanks.
>>>> Hong
>>>>
>>>>
>>>> *Hong Li*
>>>>
>>>> *Centric Consulting*
>>>>
>>>> *In Balance*
>>>> (888) 781-7567 office
>>>> (614) 296-7644 mobile
>>>> www.centricconsulting.com | @Centric <https://twitter.com/centric>
>>>>
>>>
>>>
>>
>
Re: EvaluateJsonPath and Json Field Name Starting with @ as the First Character
Posted by Matt Burgess <ma...@gmail.com>.
Hong,
At this point you have the choice of going to Apache NiFi 0.6.0 or
upgrading to HDF 1.2 which has NiFi 0.6.0 as well:
http://docs.hortonworks.com/HDPDocuments/HDF1/HDF-1.2/bk_HDF_RelNotes/content/ch_hdf_relnotes.html#release_summary
Regards,
Matt
On Tue, Mar 29, 2016 at 9:26 AM, Hong Li <ho...@centricconsulting.com>
wrote:
> Thanks, Matt. Out version is 0.5.1. We have been using HDF. But we are
> going to upgrade to 0.6.0 from Apache directly this time.
>
> Hong
>
>
>
> *Hong Li*
>
> *Centric Consulting*
>
> *In Balance*
> (888) 781-7567 office
> (614) 296-7644 mobile
> www.centricconsulting.com | @Centric <https://twitter.com/centric>
>
> On Tue, Mar 29, 2016 at 8:44 AM, Matt Burgess <ma...@gmail.com> wrote:
>
>> Hong,
>>
>> I was able to use EvaluateJsonPath with eventClass $.event.@class and
>> the attribute had the correct value (see output from LogAttribute below):
>>
>> --------------------------------------------------
>> Standard FlowFile Attributes
>> Key: 'entryDate'
>> Value: 'Tue Mar 29 08:41:36 EDT 2016'
>> Key: 'lineageStartDate'
>> Value: 'Tue Mar 29 08:41:36 EDT 2016'
>> Key: 'fileSize'
>> Value: '932'
>> FlowFile Attribute Map Content
>> Key: 'eventClass'
>> Value: 'events.policy.PolicyCreated'
>> Key: 'filename'
>> Value: 'test.json'
>> Key: 'path'
>> Value: './'
>> Key: 'uuid'
>> Value: '174e1add-0411-41a4-b7f5-7a43360ad104'
>> --------------------------------------------------
>>
>> What version of NiFi are you using? I used 0.6.0 to get the above
>> results, perhaps there was a bug that was fixed?
>>
>> Regards,
>> Matt
>>
>>
>> On Tue, Mar 29, 2016 at 4:30 AM, Hong Li <ho...@centricconsulting.com>
>> wrote:
>>
>>> How can I assign the value of a Json field that has a name starting with
>>> @ as the first character of its name to a user-defined-property in
>>> EvaluateJsonPath?
>>>
>>> For example, I have the following sample Json message:
>>>
>>> {
>>> "effectiveTimestamp": "2015-12-03T23:17:29.874Z",
>>> "event": {
>>>
>>>> "@class": "events.policy.PolicyCreated",
>>>
>>> "id": "1e9b91398160471f8b6197ad974e2464",
>>> "ipAddress": "10.8.30.145",
>>> "policy": {
>>> "additionalListedInsureds": [],
>>> "address": {
>>> "city": "Skokie",
>>> "county": "Cook",
>>> "id": "b863190a5bf846858eb372fb5f532fe7",
>>> "latitude": 42.0101,
>>> "longitude": -87.75354,
>>> "state": "IL",
>>> "street": "5014 Estes Ave",
>>> "zip": "60077-3520"
>>> },
>>> "applicant": {
>>> "age": 36,
>>> "birthDate": "1979-01-12",
>>> "clientId": "191",
>>> "creditReport": {
>>> "id": "ca5ec932d33d444b880c9a43a6eb7c50",
>>> "reasons": [],
>>> "referenceNumber": "15317191300474",
>>> "status": "NoHit"
>>> },
>>> "firstName": "Kathy",
>>> "gender": "Female",
>>> "id": "1f4a3862fab54e058305e3c73cc13dd3",
>>> "lastName": "Bockett",
>>> "maritalStatus": "Single",
>>> "middleName": "Sue",
>>> "ssn": "*******"
>>> },
>>> "channelOfOrigin": "PublicWebsite",
>>> ... ...
>>>
>>> Under EvaluateJsonPath, I can make
>>>
>>> eventID = $.event.id
>>>
>>>
>>> But I found out that if I make
>>>
>>> eventClass = $.event.@class
>>>
>>> the value of @class will never be assigned to eventClass.
>>>
>>> How should we capture the value of such a Json field?
>>>
>>> Thanks.
>>> Hong
>>>
>>>
>>> *Hong Li*
>>>
>>> *Centric Consulting*
>>>
>>> *In Balance*
>>> (888) 781-7567 office
>>> (614) 296-7644 mobile
>>> www.centricconsulting.com | @Centric <https://twitter.com/centric>
>>>
>>
>>
>
Re: EvaluateJsonPath and Json Field Name Starting with @ as the First Character
Posted by Hong Li <ho...@centricconsulting.com>.
Thanks, Matt. Out version is 0.5.1. We have been using HDF. But we are
going to upgrade to 0.6.0 from Apache directly this time.
Hong
*Hong Li*
*Centric Consulting*
*In Balance*
(888) 781-7567 office
(614) 296-7644 mobile
www.centricconsulting.com | @Centric <https://twitter.com/centric>
On Tue, Mar 29, 2016 at 8:44 AM, Matt Burgess <ma...@gmail.com> wrote:
> Hong,
>
> I was able to use EvaluateJsonPath with eventClass $.event.@class and the
> attribute had the correct value (see output from LogAttribute below):
>
> --------------------------------------------------
> Standard FlowFile Attributes
> Key: 'entryDate'
> Value: 'Tue Mar 29 08:41:36 EDT 2016'
> Key: 'lineageStartDate'
> Value: 'Tue Mar 29 08:41:36 EDT 2016'
> Key: 'fileSize'
> Value: '932'
> FlowFile Attribute Map Content
> Key: 'eventClass'
> Value: 'events.policy.PolicyCreated'
> Key: 'filename'
> Value: 'test.json'
> Key: 'path'
> Value: './'
> Key: 'uuid'
> Value: '174e1add-0411-41a4-b7f5-7a43360ad104'
> --------------------------------------------------
>
> What version of NiFi are you using? I used 0.6.0 to get the above results,
> perhaps there was a bug that was fixed?
>
> Regards,
> Matt
>
>
> On Tue, Mar 29, 2016 at 4:30 AM, Hong Li <ho...@centricconsulting.com>
> wrote:
>
>> How can I assign the value of a Json field that has a name starting with
>> @ as the first character of its name to a user-defined-property in
>> EvaluateJsonPath?
>>
>> For example, I have the following sample Json message:
>>
>> {
>> "effectiveTimestamp": "2015-12-03T23:17:29.874Z",
>> "event": {
>>
>>> "@class": "events.policy.PolicyCreated",
>>
>> "id": "1e9b91398160471f8b6197ad974e2464",
>> "ipAddress": "10.8.30.145",
>> "policy": {
>> "additionalListedInsureds": [],
>> "address": {
>> "city": "Skokie",
>> "county": "Cook",
>> "id": "b863190a5bf846858eb372fb5f532fe7",
>> "latitude": 42.0101,
>> "longitude": -87.75354,
>> "state": "IL",
>> "street": "5014 Estes Ave",
>> "zip": "60077-3520"
>> },
>> "applicant": {
>> "age": 36,
>> "birthDate": "1979-01-12",
>> "clientId": "191",
>> "creditReport": {
>> "id": "ca5ec932d33d444b880c9a43a6eb7c50",
>> "reasons": [],
>> "referenceNumber": "15317191300474",
>> "status": "NoHit"
>> },
>> "firstName": "Kathy",
>> "gender": "Female",
>> "id": "1f4a3862fab54e058305e3c73cc13dd3",
>> "lastName": "Bockett",
>> "maritalStatus": "Single",
>> "middleName": "Sue",
>> "ssn": "*******"
>> },
>> "channelOfOrigin": "PublicWebsite",
>> ... ...
>>
>> Under EvaluateJsonPath, I can make
>>
>> eventID = $.event.id
>>
>>
>> But I found out that if I make
>>
>> eventClass = $.event.@class
>>
>> the value of @class will never be assigned to eventClass.
>>
>> How should we capture the value of such a Json field?
>>
>> Thanks.
>> Hong
>>
>>
>> *Hong Li*
>>
>> *Centric Consulting*
>>
>> *In Balance*
>> (888) 781-7567 office
>> (614) 296-7644 mobile
>> www.centricconsulting.com | @Centric <https://twitter.com/centric>
>>
>
>
Re: EvaluateJsonPath and Json Field Name Starting with @ as the First Character
Posted by Matt Burgess <ma...@gmail.com>.
Hong,
I was able to use EvaluateJsonPath with eventClass $.event.@class and the
attribute had the correct value (see output from LogAttribute below):
--------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Tue Mar 29 08:41:36 EDT 2016'
Key: 'lineageStartDate'
Value: 'Tue Mar 29 08:41:36 EDT 2016'
Key: 'fileSize'
Value: '932'
FlowFile Attribute Map Content
Key: 'eventClass'
Value: 'events.policy.PolicyCreated'
Key: 'filename'
Value: 'test.json'
Key: 'path'
Value: './'
Key: 'uuid'
Value: '174e1add-0411-41a4-b7f5-7a43360ad104'
--------------------------------------------------
What version of NiFi are you using? I used 0.6.0 to get the above results,
perhaps there was a bug that was fixed?
Regards,
Matt
On Tue, Mar 29, 2016 at 4:30 AM, Hong Li <ho...@centricconsulting.com>
wrote:
> How can I assign the value of a Json field that has a name starting with @
> as the first character of its name to a user-defined-property in
> EvaluateJsonPath?
>
> For example, I have the following sample Json message:
>
> {
> "effectiveTimestamp": "2015-12-03T23:17:29.874Z",
> "event": {
>
>> "@class": "events.policy.PolicyCreated",
>
> "id": "1e9b91398160471f8b6197ad974e2464",
> "ipAddress": "10.8.30.145",
> "policy": {
> "additionalListedInsureds": [],
> "address": {
> "city": "Skokie",
> "county": "Cook",
> "id": "b863190a5bf846858eb372fb5f532fe7",
> "latitude": 42.0101,
> "longitude": -87.75354,
> "state": "IL",
> "street": "5014 Estes Ave",
> "zip": "60077-3520"
> },
> "applicant": {
> "age": 36,
> "birthDate": "1979-01-12",
> "clientId": "191",
> "creditReport": {
> "id": "ca5ec932d33d444b880c9a43a6eb7c50",
> "reasons": [],
> "referenceNumber": "15317191300474",
> "status": "NoHit"
> },
> "firstName": "Kathy",
> "gender": "Female",
> "id": "1f4a3862fab54e058305e3c73cc13dd3",
> "lastName": "Bockett",
> "maritalStatus": "Single",
> "middleName": "Sue",
> "ssn": "*******"
> },
> "channelOfOrigin": "PublicWebsite",
> ... ...
>
> Under EvaluateJsonPath, I can make
>
> eventID = $.event.id
>
>
> But I found out that if I make
>
> eventClass = $.event.@class
>
> the value of @class will never be assigned to eventClass.
>
> How should we capture the value of such a Json field?
>
> Thanks.
> Hong
>
>
> *Hong Li*
>
> *Centric Consulting*
>
> *In Balance*
> (888) 781-7567 office
> (614) 296-7644 mobile
> www.centricconsulting.com | @Centric <https://twitter.com/centric>
>