You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by scriptnull <vi...@gmail.com> on 2020/04/24 09:08:26 UTC
Inconsistency in data stored via the Redis layer
Hi,
We are trying to use Apache Ignite via the Redis layer (
https://apacheignite.readme.io/docs/redis ). While trying to store a string
from a ruby redis client and retrieving back, we notice some inconsistency
in the data. We believe that this has something to do with how Apache Ignite
handles encoding. Would be great to learn about why this is happening and
possible mitigation for the problem.
Here is the problem in more detail.
First we have a ruby object which we marshal to get the string
representation of.
```
=> {:id=>7833548, :ad_group_id=>"91254654888",
:adwords_campaign_id=>548351, :name=>"mcdonald's_ (e)",
:configured_status=>"ENABLED", :adwords_ad_account_id=>4798,
:created_at=>Fri, 06 Dec 2019 08:18:34 UTC +00:00, :updated_at=>Mon, 20 Apr
2020 18:51:01 UTC +00:00, :targeting=>{"targeting_type"=>["Keyword"]},
:tracking_url_template=>nil, :ad_group_type=>"Search-Standard",
:content_bid_criterion_type_group=>nil,
:final_urls=>["https://www.foodora.se/restaurant/s7fx/mcdonald-s-kungsgatan?"],
:bidding_strategy_configuration=>{"bidding_strategy_type"=>"TARGET_CPA",
"bids"=>[{"bids_type"=>"CpcBid", "bid"=>{"comparable_value_type"=>"Money",
"micro_amount"=>1000000}, "cpc_bid_source"=>"ADGROUP",
"xsi_type"=>"CpcBid"}, {"bids_type"=>"CpaBid",
"bid"=>{"comparable_value_type"=>"Money", "micro_amount"=>15000000},
"xsi_type"=>"CpaBid"}]}, :labels=>"", :audiences=>nil,
:mongo_core_object_updated_at=>Sat, 14 Dec 2019 13:38:30 UTC +00:00,
:ad_rotation_mode=>nil, :system_dimensions_last_run_at=>Fri, 06 Mar 2020
00:31:07 UTC +00:00}
```
We marshal this object to a string representation (encoding of this string
is ASCII-8BIT)
```
redis_client.set('key', Marshal.dump(obj))
```
and here is the contents of the string stored in Apache Ignite (as seen via
the redis-cli)
```
"\x04\b{\x18:\aidi\x03\xcc\x87w:\x10ad_group_idI\"\x1091254654888\x06:\x06ET:\x18adwords_campaign_idi\x03\xef\xbf\xbd]\b:\tnameI\"\x14mcdonald's_
(e)\x06;\aT:\x16configured_statusI\"\x0cENABLED\x06;\aT:\x1aadwords_ad_account_idi\x02\xef\xbf\xbd\x12:\x0fcreated_atU:
ActiveSupport::TimeWithZone[\bIu:\tTime\r\xef\xbf\xbd\xef\xbf\xbd\x1d\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd.J\x06:\tzoneI\"\bUTC\x06;\aFI\"\bUTC\x06;\aT@\x0c:\x0fupdated_atU;\r[\bIu;\x0e\r\xef\xbf\xbd\x0e\x1e\xef\xbf\xbd\x0bX\x1c\xef\xbf\xbd\x06;\x0f@\x0b@\r@\x10:\x0etargeting{\x06I\"\x13targeting_type\x06;\aT[\x06I\"\x0cKeyword\x06;\aT:\x1atracking_url_template0:\x12ad_group_typeI\"\x14Search-Standard\x06;\aT:%content_bid_criterion_type_group0:\x0ffinal_urls[\x06I\"Bhttps://www.foodora.se/restaurant/s7fx/mcdonald-s-kungsgatan?\x06;\aT:#bidding_strategy_configuration{\aI\"\x1abidding_strategy_type\x06;\aTI\"\x0fTARGET_CPA\x06;\aTI\"\tbids\x06;\aT[\a{\tI\"\x0ebids_type\x06;\aTI\"\x0bCpcBid\x06;\aTI\"\bbid\x06;\aT{\aI\"\x1acomparable_value_type\x06;\aTI\"\nMoney\x06;\aTI\"\x11micro_amount\x06;\aTi\x03@B\x0fI\"\x13cpc_bid_source\x06;\aTI\"\x0cADGROUP\x06;\aTI\"\rxsi_type\x06;\aTI\"\x0bCpcBid\x06;\aT{\b@\x1eI\"\x0bCpaBid\x06;\aT@
{\a@\"I\"\nMoney\x06;\aT@$i\x03\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd@'I\"\x0bCpaBid\x06;\aT:\x0blabelsI\"\x00\x06;\aF:\x0eaudiences0:!mongo_core_object_updated_atU;\r[\bIu;\x0e\r\xef\xbf\xbd\xef\xbf\xbd\x1d\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\x06;\x0f@\x0b@\r@1:\x15ad_rotation_mode0:\"system_dimensions_last_run_atU;\r[\bIu;\x0e\r\xef\xbf\xbd\b\x1e\xef\xbf\xbd\x00\x00p|\x06;\x0f@\x0b@\r@4"
```
If we try to store the same data in Redis, we can see the following contents
via the redis-cli
```
"\x04\b{\x18:\aidi\x03\xcc\x87w:\x10ad_group_idI\"\x1091254654888\x06:\x06ET:\x18adwords_campaign_idi\x03\xff]\b:\tnameI\"\x14mcdonald's_
(e)\x06;\aT:\x16configured_statusI\"\x0cENABLED\x06;\aT:\x1aadwords_ad_account_idi\x02\xbe\x12:\x0fcreated_atU:
ActiveSupport::TimeWithZone[\bIu:\tTime\r\xc8\xec\x1d\xc0\x92\xd3.J\x06:\tzoneI\"\bUTC\x06;\aFI\"\bUTC\x06;\aT@\x0c:\x0fupdated_atU;\r[\bIu;\x0e\r\x92\x0e\x1e\xc0\x0bX\x1c\xcc\x06;\x0f@\x0b@\r@\x10:\x0etargeting{\x06I\"\x13targeting_type\x06;\aT[\x06I\"\x0cKeyword\x06;\aT:\x1atracking_url_template0:\x12ad_group_typeI\"\x14Search-Standard\x06;\aT:%content_bid_criterion_type_group0:\x0ffinal_urls[\x06I\"Bhttps://www.foodora.se/restaurant/s7fx/mcdonald-s-kungsgatan?\x06;\aT:#bidding_strategy_configuration{\aI\"\x1abidding_strategy_type\x06;\aTI\"\x0fTARGET_CPA\x06;\aTI\"\tbids\x06;\aT[\a{\tI\"\x0ebids_type\x06;\aTI\"\x0bCpcBid\x06;\aTI\"\bbid\x06;\aT{\aI\"\x1acomparable_value_type\x06;\aTI\"\nMoney\x06;\aTI\"\x11micro_amount\x06;\aTi\x03@B\x0fI\"\x13cpc_bid_source\x06;\aTI\"\x0cADGROUP\x06;\aTI\"\rxsi_type\x06;\aTI\"\x0bCpcBid\x06;\aT{\b@\x1eI\"\x0bCpaBid\x06;\aT@
{\a@\"I\"\nMoney\x06;\aT@$i\x03\xc0\xe1\xe4@'I\"\x0bCpaBid\x06;\aT:\x0blabelsI\"\x00\x06;\aF:\x0eaudiences0:!mongo_core_object_updated_atU;\r[\bIu;\x0e\r\xcd\xed\x1d\xc0\xab\xb2\xee\x99\x06;\x0f@\x0b@\r@1:\x15ad_rotation_mode0:\"system_dimensions_last_run_atU;\r[\bIu;\x0e\r\xc0\b\x1e\xc0\x00\x00p|\x06;\x0f@\x0b@\r@4"
```
Due to this, when we retrieve back this data in the ruby code and try
unmarshalling (Marshal.load), we are unable to reconstruct the object in
case of Apache Ignite, whereas for redis the reconstruction of object is
successful.
```
Marshal.load(redis_client.get('key'))
```
So what might possibly be happening differently in each of those cases?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Inconsistency in data stored via the Redis layer
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
My recommendation is to use REST API instead of redis/memcached.
Regards,
--
Ilya Kasnacheev
пт, 24 апр. 2020 г. в 12:08, scriptnull <vi...@gmail.com>:
> Hi,
>
> We are trying to use Apache Ignite via the Redis layer (
> https://apacheignite.readme.io/docs/redis ). While trying to store a
> string
> from a ruby redis client and retrieving back, we notice some inconsistency
> in the data. We believe that this has something to do with how Apache
> Ignite
> handles encoding. Would be great to learn about why this is happening and
> possible mitigation for the problem.
>
> Here is the problem in more detail.
>
> First we have a ruby object which we marshal to get the string
> representation of.
>
> ```
> => {:id=>7833548, :ad_group_id=>"91254654888",
> :adwords_campaign_id=>548351, :name=>"mcdonald's_ (e)",
> :configured_status=>"ENABLED", :adwords_ad_account_id=>4798,
> :created_at=>Fri, 06 Dec 2019 08:18:34 UTC +00:00, :updated_at=>Mon, 20 Apr
> 2020 18:51:01 UTC +00:00, :targeting=>{"targeting_type"=>["Keyword"]},
> :tracking_url_template=>nil, :ad_group_type=>"Search-Standard",
> :content_bid_criterion_type_group=>nil,
> :final_urls=>["
> https://www.foodora.se/restaurant/s7fx/mcdonald-s-kungsgatan?"],
> :bidding_strategy_configuration=>{"bidding_strategy_type"=>"TARGET_CPA",
> "bids"=>[{"bids_type"=>"CpcBid", "bid"=>{"comparable_value_type"=>"Money",
> "micro_amount"=>1000000}, "cpc_bid_source"=>"ADGROUP",
> "xsi_type"=>"CpcBid"}, {"bids_type"=>"CpaBid",
> "bid"=>{"comparable_value_type"=>"Money", "micro_amount"=>15000000},
> "xsi_type"=>"CpaBid"}]}, :labels=>"", :audiences=>nil,
> :mongo_core_object_updated_at=>Sat, 14 Dec 2019 13:38:30 UTC +00:00,
> :ad_rotation_mode=>nil, :system_dimensions_last_run_at=>Fri, 06 Mar 2020
> 00:31:07 UTC +00:00}
> ```
>
> We marshal this object to a string representation (encoding of this string
> is ASCII-8BIT)
> ```
> redis_client.set('key', Marshal.dump(obj))
> ```
>
> and here is the contents of the string stored in Apache Ignite (as seen via
> the redis-cli)
> ```
>
> "\x04\b{\x18:\aidi\x03\xcc\x87w:\x10ad_group_idI\"\x1091254654888\x06:\x06ET:\x18adwords_campaign_idi\x03\xef\xbf\xbd]\b:\tnameI\"\x14mcdonald's_
>
> (e)\x06;\aT:\x16configured_statusI\"\x0cENABLED\x06;\aT:\x1aadwords_ad_account_idi\x02\xef\xbf\xbd\x12:\x0fcreated_atU:
>
> ActiveSupport::TimeWithZone[\bIu:\tTime\r\xef\xbf\xbd\xef\xbf\xbd\x1d\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd.J\x06:\tzoneI\"\bUTC\x06;\aFI\"\bUTC\x06;\aT@
> \x0c:\x0fupdated_atU;\r[\bIu;\x0e\r\xef\xbf\xbd\x0e\x1e\xef\xbf\xbd\x0bX\x1c\xef\xbf\xbd\x06;\x0f@
> \x0b@\r@
> \x10:\x0etargeting{\x06I\"\x13targeting_type\x06;\aT[\x06I\"\x0cKeyword\x06;\aT:\x1atracking_url_template0:\x12ad_group_typeI\"\x14Search-Standard\x06;\aT:%content_bid_criterion_type_group0:\x0ffinal_urls[\x06I\"Bhttps://
> www.foodora.se/restaurant/s7fx/mcdonald-s-kungsgatan
> ?\x06;\aT:#bidding_strategy_configuration{\aI\"\x1abidding_strategy_type\x06;\aTI\"\x0fTARGET_CPA\x06;\aTI\"\tbids\x06;\aT[\a{\tI\"\x0ebids_type\x06;\aTI\"\x0bCpcBid\x06;\aTI\"\bbid\x06;\aT{\aI\"\x1acomparable_value_type\x06;\aTI\"\nMoney\x06;\aTI\"\x11micro_amount\x06;\aTi\x03@B
> \x0fI\"\x13cpc_bid_source\x06;\aTI\"\x0cADGROUP\x06;\aTI\"\rxsi_type\x06;\aTI\"\x0bCpcBid\x06;\aT{\b@
> \x1eI\"\x0bCpaBid\x06;\aT@
> {\a@\"I\"\nMoney\x06;\aT@$i\x03\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd@
> 'I\"\x0bCpaBid\x06;\aT:\x0blabelsI\"\x00\x06;\aF:\x0eaudiences0:!mongo_core_object_updated_atU;\r[\bIu;\x0e\r\xef\xbf\xbd\xef\xbf\xbd\x1d\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\x06;\x0f@
> \x0b@\r@1
> :\x15ad_rotation_mode0:\"system_dimensions_last_run_atU;\r[\bIu;\x0e\r\xef\xbf\xbd\b\x1e\xef\xbf\xbd\x00\x00p|\x06;\x0f@
> \x0b@\r@4"
> ```
>
> If we try to store the same data in Redis, we can see the following
> contents
> via the redis-cli
> ```
>
> "\x04\b{\x18:\aidi\x03\xcc\x87w:\x10ad_group_idI\"\x1091254654888\x06:\x06ET:\x18adwords_campaign_idi\x03\xff]\b:\tnameI\"\x14mcdonald's_
>
> (e)\x06;\aT:\x16configured_statusI\"\x0cENABLED\x06;\aT:\x1aadwords_ad_account_idi\x02\xbe\x12:\x0fcreated_atU:
>
> ActiveSupport::TimeWithZone[\bIu:\tTime\r\xc8\xec\x1d\xc0\x92\xd3.J\x06:\tzoneI\"\bUTC\x06;\aFI\"\bUTC\x06;\aT@
> \x0c:\x0fupdated_atU;\r[\bIu;\x0e\r\x92\x0e\x1e\xc0\x0bX\x1c\xcc\x06;\x0f@
> \x0b@\r@
> \x10:\x0etargeting{\x06I\"\x13targeting_type\x06;\aT[\x06I\"\x0cKeyword\x06;\aT:\x1atracking_url_template0:\x12ad_group_typeI\"\x14Search-Standard\x06;\aT:%content_bid_criterion_type_group0:\x0ffinal_urls[\x06I\"Bhttps://
> www.foodora.se/restaurant/s7fx/mcdonald-s-kungsgatan
> ?\x06;\aT:#bidding_strategy_configuration{\aI\"\x1abidding_strategy_type\x06;\aTI\"\x0fTARGET_CPA\x06;\aTI\"\tbids\x06;\aT[\a{\tI\"\x0ebids_type\x06;\aTI\"\x0bCpcBid\x06;\aTI\"\bbid\x06;\aT{\aI\"\x1acomparable_value_type\x06;\aTI\"\nMoney\x06;\aTI\"\x11micro_amount\x06;\aTi\x03@B
> \x0fI\"\x13cpc_bid_source\x06;\aTI\"\x0cADGROUP\x06;\aTI\"\rxsi_type\x06;\aTI\"\x0bCpcBid\x06;\aT{\b@
> \x1eI\"\x0bCpaBid\x06;\aT@
> {\a@\"I\"\nMoney\x06;\aT@$i\x03\xc0\xe1\xe4@
> 'I\"\x0bCpaBid\x06;\aT:\x0blabelsI\"\x00\x06;\aF:\x0eaudiences0:!mongo_core_object_updated_atU;\r[\bIu;\x0e\r\xcd\xed\x1d\xc0\xab\xb2\xee\x99\x06;\x0f@
> \x0b@\r@1
> :\x15ad_rotation_mode0:\"system_dimensions_last_run_atU;\r[\bIu;\x0e\r\xc0\b\x1e\xc0\x00\x00p|\x06;\x0f@
> \x0b@\r@4"
> ```
>
> Due to this, when we retrieve back this data in the ruby code and try
> unmarshalling (Marshal.load), we are unable to reconstruct the object in
> case of Apache Ignite, whereas for redis the reconstruction of object is
> successful.
>
> ```
> Marshal.load(redis_client.get('key'))
> ```
>
> So what might possibly be happening differently in each of those cases?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>