You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Igor Kravzov <ig...@gmail.com> on 2016/04/23 23:11:29 UTC
EvaluateJsonPath processor
Hi,
I am using EvaluateJsonPath to extract some properties from tweet JSON
returned by GetTwitter processor and later convert to another JSON using
AttributesToJSON processor. One of the properties I want to extract is
"entities" which is structure of arrays by itself.
Unfortunately EvaluateJsonPath fails with "Unable to return scalar value
for expression $.[entities]. I don't want to evaluate each and every
possible value of entities structure. is there a way to get entities or
other similar properties as it is?
Thanks in advance.
Re: EvaluateJsonPath processor
Posted by Igor Kravzov <ig...@gmail.com>.
Hi Pierre,
I'll try. But need some help. I am not so good at Regular expressions.
What be regex to remove backslash? I also need to replace
"{ -> {
}" -> }
On Sun, Apr 24, 2016 at 3:18 AM, Pierre Villard <pierre.villard.fr@gmail.com
> wrote:
> Igor, in this case, you may try to add a ReplaceText processor at the end
> of your flow to use regular expressions and remove the backslash characters.
>
> Let me know if it does not help.
> Pierre
>
> 2016-04-24 0:24 GMT+02:00 Igor Kravzov <ig...@gmail.com>:
>
>> Hi Pierre,
>>
>> From some reason it escapes values
>>
>> {
>> "twitter.unixtime": "1461449854014",
>> "twitter.user": "find super deal",
>> "RouteOnAttribute.Route": "tweet",
>> "twitter.msg": "DELL T7610 WORKSTATION W8 E5-2687WV2 3.4GHZ 32GB 2 X
>> 1TB SATA NVS295 https://t.co/UAyLCtoikM https://t.co/6QJoWPjXGB",
>> "twitter.tweet_id": "723999278948081664",
>> "twitter.handle": "super_find",
>> "language": "en",
>> "twitter.hashtags": "{\"hashtags\":[],\"urls\":[{\"url\":\"
>> https://t.co/UAyLCtoikM\",\"expanded_url\":\"
>> http://computers-and-tablets.info/cmptrsn/dtblts013/?query=http://rover.ebay.com/rover/1/711-53200-19255-0/1?ff3=2&toolid=10039&campid=5337797091&item=191854657494&vectorid=229466&lgeo=1\
>> ",\"display_url\":\"computers-and-tablets.info/cmptrsn/dtblts
>> …\",\"indices\":[69,92]}],\"user_mentions\":[],\"symbols\":[],\"media\":[{\"id\":723999278717370369,\"id_str\":\"723999278717370369\",\"indices\":[93,116],\"media_url\":\"
>> http://pbs.twimg.com/media/CgwpghGWgAEow2Z.jpg\",\"media_url_https\":\"
>> https://pbs.twimg.com/media/CgwpghGWgAEow2Z.jpg\",\"url\":\"
>> https://t.co/6QJoWPjXGB\",\"display_url\":\"pic.twitter.com/6QJoWPjXGB\
>> <http://pic.twitter.com/6QJoWPjXGB%5C>",\"expanded_url\":\"
>> http://twitter.com/super_find/status/723999278948081664/photo/1\
>> ",\"type\":\"photo\",\"sizes\":{\"small\":{\"w\":340,\"h\":255,\"resize\":\"fit\"},\"medium\":{\"w\":400,\"h\":300,\"resize\":\"fit\"},\"thumb\":{\"w\":150,\"h\":150,\"resize\":\"crop\"},\"large\":{\"w\":400,\"h\":300,\"resize\":\"fit\"}}}]}",
>> "user": "{\"id\":4715465700,\"id_str\":\"4715465700\",\"name\":\"find
>> super
>> deal\",\"screen_name\":\"super_find\",\"location\":null,\"url\":null,\"description\":null,\"protected\":false,\"verified\":false,\"followers_count\":18,\"friends_count\":40,\"listed_count\":21,\"favourites_count\":0,\"statuses_count\":23731,\"created_at\":\"Tue
>> Jan 05 22:19:11 +0000
>> 2016\",\"utc_offset\":null,\"time_zone\":null,\"geo_enabled\":false,\"lang\":\"es\",\"contributors_enabled\":false,\"is_translator\":false,\"profile_background_color\":\"F5F8FA\",\"profile_background_image_url\":\"\",\"profile_background_image_url_https\":\"\",\"profile_background_tile\":false,\"profile_link_color\":\"2B7BB9\",\"profile_sidebar_border_color\":\"C0DEED\",\"profile_sidebar_fill_color\":\"DDEEF6\",\"profile_text_color\":\"333333\",\"profile_use_background_image\":true,\"profile_image_url\":\"
>> http://pbs.twimg.com/profile_images/684499862197563392/x72zf3WB_normal.jpg\
>> ",\"profile_image_url_https\":\"
>> https://pbs.twimg.com/profile_images/684499862197563392/x72zf3WB_normal.jpg\
>> ",\"default_profile\":true,\"default_profile_image\":false,\"following\":null,\"follow_request_sent\":null,\"notifications\":null}",
>> "twitter.time": "Sat Apr 23 22:17:34 +0000 2016"
>> }
>>
>> On Sat, Apr 23, 2016 at 5:57 PM, Pierre Villard <
>> pierre.villard.fr@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> In this case, in your EvaluateJsonPath processor, I would try to change
>>> the property "return type" to "json".
>>>
>>> Hope that helps!
>>> Pierre
>>>
>>>
>>> 2016-04-23 23:11 GMT+02:00 Igor Kravzov <ig...@gmail.com>:
>>>
>>>> Hi,
>>>>
>>>> I am using EvaluateJsonPath to extract some properties from tweet JSON
>>>> returned by GetTwitter processor and later convert to another JSON using
>>>> AttributesToJSON processor. One of the properties I want to extract is
>>>> "entities" which is structure of arrays by itself.
>>>> Unfortunately EvaluateJsonPath fails with "Unable to return scalar
>>>> value for expression $.[entities]. I don't want to evaluate each and every
>>>> possible value of entities structure. is there a way to get entities or
>>>> other similar properties as it is?
>>>>
>>>> Thanks in advance.
>>>>
>>>
>>>
>>
>
Re: EvaluateJsonPath processor
Posted by Pierre Villard <pi...@gmail.com>.
Igor, in this case, you may try to add a ReplaceText processor at the end
of your flow to use regular expressions and remove the backslash characters.
Let me know if it does not help.
Pierre
2016-04-24 0:24 GMT+02:00 Igor Kravzov <ig...@gmail.com>:
> Hi Pierre,
>
> From some reason it escapes values
>
> {
> "twitter.unixtime": "1461449854014",
> "twitter.user": "find super deal",
> "RouteOnAttribute.Route": "tweet",
> "twitter.msg": "DELL T7610 WORKSTATION W8 E5-2687WV2 3.4GHZ 32GB 2 X 1TB
> SATA NVS295 https://t.co/UAyLCtoikM https://t.co/6QJoWPjXGB",
> "twitter.tweet_id": "723999278948081664",
> "twitter.handle": "super_find",
> "language": "en",
> "twitter.hashtags": "{\"hashtags\":[],\"urls\":[{\"url\":\"
> https://t.co/UAyLCtoikM\",\"expanded_url\":\"
> http://computers-and-tablets.info/cmptrsn/dtblts013/?query=http://rover.ebay.com/rover/1/711-53200-19255-0/1?ff3=2&toolid=10039&campid=5337797091&item=191854657494&vectorid=229466&lgeo=1\
> ",\"display_url\":\"computers-and-tablets.info/cmptrsn/dtblts
> …\",\"indices\":[69,92]}],\"user_mentions\":[],\"symbols\":[],\"media\":[{\"id\":723999278717370369,\"id_str\":\"723999278717370369\",\"indices\":[93,116],\"media_url\":\"
> http://pbs.twimg.com/media/CgwpghGWgAEow2Z.jpg\",\"media_url_https\":\"
> https://pbs.twimg.com/media/CgwpghGWgAEow2Z.jpg\",\"url\":\"
> https://t.co/6QJoWPjXGB\",\"display_url\":\"pic.twitter.com/6QJoWPjXGB\
> <http://pic.twitter.com/6QJoWPjXGB%5C>",\"expanded_url\":\"
> http://twitter.com/super_find/status/723999278948081664/photo/1\
> ",\"type\":\"photo\",\"sizes\":{\"small\":{\"w\":340,\"h\":255,\"resize\":\"fit\"},\"medium\":{\"w\":400,\"h\":300,\"resize\":\"fit\"},\"thumb\":{\"w\":150,\"h\":150,\"resize\":\"crop\"},\"large\":{\"w\":400,\"h\":300,\"resize\":\"fit\"}}}]}",
> "user": "{\"id\":4715465700,\"id_str\":\"4715465700\",\"name\":\"find
> super
> deal\",\"screen_name\":\"super_find\",\"location\":null,\"url\":null,\"description\":null,\"protected\":false,\"verified\":false,\"followers_count\":18,\"friends_count\":40,\"listed_count\":21,\"favourites_count\":0,\"statuses_count\":23731,\"created_at\":\"Tue
> Jan 05 22:19:11 +0000
> 2016\",\"utc_offset\":null,\"time_zone\":null,\"geo_enabled\":false,\"lang\":\"es\",\"contributors_enabled\":false,\"is_translator\":false,\"profile_background_color\":\"F5F8FA\",\"profile_background_image_url\":\"\",\"profile_background_image_url_https\":\"\",\"profile_background_tile\":false,\"profile_link_color\":\"2B7BB9\",\"profile_sidebar_border_color\":\"C0DEED\",\"profile_sidebar_fill_color\":\"DDEEF6\",\"profile_text_color\":\"333333\",\"profile_use_background_image\":true,\"profile_image_url\":\"
> http://pbs.twimg.com/profile_images/684499862197563392/x72zf3WB_normal.jpg\
> ",\"profile_image_url_https\":\"
> https://pbs.twimg.com/profile_images/684499862197563392/x72zf3WB_normal.jpg\
> ",\"default_profile\":true,\"default_profile_image\":false,\"following\":null,\"follow_request_sent\":null,\"notifications\":null}",
> "twitter.time": "Sat Apr 23 22:17:34 +0000 2016"
> }
>
> On Sat, Apr 23, 2016 at 5:57 PM, Pierre Villard <
> pierre.villard.fr@gmail.com> wrote:
>
>> Hi,
>>
>> In this case, in your EvaluateJsonPath processor, I would try to change
>> the property "return type" to "json".
>>
>> Hope that helps!
>> Pierre
>>
>>
>> 2016-04-23 23:11 GMT+02:00 Igor Kravzov <ig...@gmail.com>:
>>
>>> Hi,
>>>
>>> I am using EvaluateJsonPath to extract some properties from tweet JSON
>>> returned by GetTwitter processor and later convert to another JSON using
>>> AttributesToJSON processor. One of the properties I want to extract is
>>> "entities" which is structure of arrays by itself.
>>> Unfortunately EvaluateJsonPath fails with "Unable to return scalar value
>>> for expression $.[entities]. I don't want to evaluate each and every
>>> possible value of entities structure. is there a way to get entities or
>>> other similar properties as it is?
>>>
>>> Thanks in advance.
>>>
>>
>>
>
Re: EvaluateJsonPath processor
Posted by Igor Kravzov <ig...@gmail.com>.
Hi Pierre,
>From some reason it escapes values
{
"twitter.unixtime": "1461449854014",
"twitter.user": "find super deal",
"RouteOnAttribute.Route": "tweet",
"twitter.msg": "DELL T7610 WORKSTATION W8 E5-2687WV2 3.4GHZ 32GB 2 X 1TB
SATA NVS295 https://t.co/UAyLCtoikM https://t.co/6QJoWPjXGB",
"twitter.tweet_id": "723999278948081664",
"twitter.handle": "super_find",
"language": "en",
"twitter.hashtags": "{\"hashtags\":[],\"urls\":[{\"url\":\"
https://t.co/UAyLCtoikM\",\"expanded_url\":\"
http://computers-and-tablets.info/cmptrsn/dtblts013/?query=http://rover.ebay.com/rover/1/711-53200-19255-0/1?ff3=2&toolid=10039&campid=5337797091&item=191854657494&vectorid=229466&lgeo=1\
",\"display_url\":\"computers-and-tablets.info/cmptrsn/dtblts
…\",\"indices\":[69,92]}],\"user_mentions\":[],\"symbols\":[],\"media\":[{\"id\":723999278717370369,\"id_str\":\"723999278717370369\",\"indices\":[93,116],\"media_url\":\"
http://pbs.twimg.com/media/CgwpghGWgAEow2Z.jpg\",\"media_url_https\":\"
https://pbs.twimg.com/media/CgwpghGWgAEow2Z.jpg\",\"url\":\"
https://t.co/6QJoWPjXGB\",\"display_url\":\"pic.twitter.com/6QJoWPjXGB\
",\"expanded_url\":\"
http://twitter.com/super_find/status/723999278948081664/photo/1\
",\"type\":\"photo\",\"sizes\":{\"small\":{\"w\":340,\"h\":255,\"resize\":\"fit\"},\"medium\":{\"w\":400,\"h\":300,\"resize\":\"fit\"},\"thumb\":{\"w\":150,\"h\":150,\"resize\":\"crop\"},\"large\":{\"w\":400,\"h\":300,\"resize\":\"fit\"}}}]}",
"user": "{\"id\":4715465700,\"id_str\":\"4715465700\",\"name\":\"find
super
deal\",\"screen_name\":\"super_find\",\"location\":null,\"url\":null,\"description\":null,\"protected\":false,\"verified\":false,\"followers_count\":18,\"friends_count\":40,\"listed_count\":21,\"favourites_count\":0,\"statuses_count\":23731,\"created_at\":\"Tue
Jan 05 22:19:11 +0000
2016\",\"utc_offset\":null,\"time_zone\":null,\"geo_enabled\":false,\"lang\":\"es\",\"contributors_enabled\":false,\"is_translator\":false,\"profile_background_color\":\"F5F8FA\",\"profile_background_image_url\":\"\",\"profile_background_image_url_https\":\"\",\"profile_background_tile\":false,\"profile_link_color\":\"2B7BB9\",\"profile_sidebar_border_color\":\"C0DEED\",\"profile_sidebar_fill_color\":\"DDEEF6\",\"profile_text_color\":\"333333\",\"profile_use_background_image\":true,\"profile_image_url\":\"
http://pbs.twimg.com/profile_images/684499862197563392/x72zf3WB_normal.jpg\
",\"profile_image_url_https\":\"
https://pbs.twimg.com/profile_images/684499862197563392/x72zf3WB_normal.jpg\
",\"default_profile\":true,\"default_profile_image\":false,\"following\":null,\"follow_request_sent\":null,\"notifications\":null}",
"twitter.time": "Sat Apr 23 22:17:34 +0000 2016"
}
On Sat, Apr 23, 2016 at 5:57 PM, Pierre Villard <pierre.villard.fr@gmail.com
> wrote:
> Hi,
>
> In this case, in your EvaluateJsonPath processor, I would try to change
> the property "return type" to "json".
>
> Hope that helps!
> Pierre
>
>
> 2016-04-23 23:11 GMT+02:00 Igor Kravzov <ig...@gmail.com>:
>
>> Hi,
>>
>> I am using EvaluateJsonPath to extract some properties from tweet JSON
>> returned by GetTwitter processor and later convert to another JSON using
>> AttributesToJSON processor. One of the properties I want to extract is
>> "entities" which is structure of arrays by itself.
>> Unfortunately EvaluateJsonPath fails with "Unable to return scalar value
>> for expression $.[entities]. I don't want to evaluate each and every
>> possible value of entities structure. is there a way to get entities or
>> other similar properties as it is?
>>
>> Thanks in advance.
>>
>
>
Re: EvaluateJsonPath processor
Posted by Pierre Villard <pi...@gmail.com>.
Hi,
In this case, in your EvaluateJsonPath processor, I would try to change the
property "return type" to "json".
Hope that helps!
Pierre
2016-04-23 23:11 GMT+02:00 Igor Kravzov <ig...@gmail.com>:
> Hi,
>
> I am using EvaluateJsonPath to extract some properties from tweet JSON
> returned by GetTwitter processor and later convert to another JSON using
> AttributesToJSON processor. One of the properties I want to extract is
> "entities" which is structure of arrays by itself.
> Unfortunately EvaluateJsonPath fails with "Unable to return scalar value
> for expression $.[entities]. I don't want to evaluate each and every
> possible value of entities structure. is there a way to get entities or
> other similar properties as it is?
>
> Thanks in advance.
>