You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Tunas <si...@msci.com> on 2020/01/30 13:32:08 UTC
How to do wildcard search by key in ignite?
I have key composed of "A"+"B"+"C"+Id.
I want to do search by "ABC*" or"AB*" or "A*".
Require very efficient query as i am fetching huge number of rows from cache
based on above query.
Also my values are serialized customize big user objects?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to do wildcard search by key in ignite?
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
If you like, you can use meaningful bits from the first two characters of
your key to form a partition id. You will need to inherit from
RendezvousAffinityFunction to do that.
In this case, you can just scan one partition (out of e.g. 1024).
However, this may pose a problem if data is not distributed evenly between
such partitions: some nodes will be virtually empty while other nodes will
be quickly overwhelmed.
Regards,
--
Ilya Kasnacheev
пт, 31 янв. 2020 г. в 10:28, Tunas <si...@msci.com>:
> Sorry, I have not provided full information or not able to understand your
> answer as i am newbie.
>
> I am storing my keys and values as string.
> For e.g. Key "ABC"+"id" value : serialized object in string form.
>
> My objects/entities has no sql attributed like "QuerySqlField" and its not
> possible (not a requirement) for me to annotate each properties of all
> entities.
>
> I was using ScanQuery operator to get all data from cache
> .......ScanQuery<string, string>(null)).GetAll()
>
> so my questions is how can i do wild card search for key 'ABC*' ? Is there
> any operator like scanQuery or is there any way without using SQL query.
>
> Thanks.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: How to do wildcard search by key in ignite?
Posted by Pavel Tupitsyn <pt...@apache.org>.
Another possibility is full-text search, which provides indexed search with
wildcards:
https://apacheignite.readme.io/v1.1/docs/cache-queries#text-queries
On Fri, Jan 31, 2020 at 2:29 PM Stephen Darlington <
stephen.darlington@gridgain.com> wrote:
> If you don’t use SQL, Ignite is basically a key-value store. That is, if
> you don’t know the key you have to look at *every* record to see if it
> matches.
>
> You can specify a filter on the ScanQuery:
>
> ScanQuery<String,Integer> q = new ScanQuery<>((k,v) ->
> k.equals("Stephen"));
>
> That wouldn’t be indexed, though. If you have a lot of records this isn’t
> going to be very efficient.
>
> On 31 Jan 2020, at 07:28, Tunas <si...@msci.com> wrote:
>
> Sorry, I have not provided full information or not able to understand your
> answer as i am newbie.
>
> I am storing my keys and values as string.
> For e.g. Key "ABC"+"id" value : serialized object in string form.
>
> My objects/entities has no sql attributed like "QuerySqlField" and its not
> possible (not a requirement) for me to annotate each properties of all
> entities.
>
> I was using ScanQuery operator to get all data from cache
> .......ScanQuery<string, string>(null)).GetAll()
>
> so my questions is how can i do wild card search for key 'ABC*' ? Is there
> any operator like scanQuery or is there any way without using SQL query.
>
> Thanks.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
>
>
Re: How to do wildcard search by key in ignite?
Posted by Stephen Darlington <st...@gridgain.com>.
If you don’t use SQL, Ignite is basically a key-value store. That is, if you don’t know the key you have to look at every record to see if it matches.
You can specify a filter on the ScanQuery:
ScanQuery<String,Integer> q = new ScanQuery<>((k,v) -> k.equals("Stephen"));
That wouldn’t be indexed, though. If you have a lot of records this isn’t going to be very efficient.
> On 31 Jan 2020, at 07:28, Tunas <si...@msci.com> wrote:
>
> Sorry, I have not provided full information or not able to understand your
> answer as i am newbie.
>
> I am storing my keys and values as string.
> For e.g. Key "ABC"+"id" value : serialized object in string form.
>
> My objects/entities has no sql attributed like "QuerySqlField" and its not
> possible (not a requirement) for me to annotate each properties of all
> entities.
>
> I was using ScanQuery operator to get all data from cache
> .......ScanQuery<string, string>(null)).GetAll()
>
> so my questions is how can i do wild card search for key 'ABC*' ? Is there
> any operator like scanQuery or is there any way without using SQL query.
>
> Thanks.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to do wildcard search by key in ignite?
Posted by Tunas <si...@msci.com>.
Sorry, I have not provided full information or not able to understand your
answer as i am newbie.
I am storing my keys and values as string.
For e.g. Key "ABC"+"id" value : serialized object in string form.
My objects/entities has no sql attributed like "QuerySqlField" and its not
possible (not a requirement) for me to annotate each properties of all
entities.
I was using ScanQuery operator to get all data from cache
.......ScanQuery<string, string>(null)).GetAll()
so my questions is how can i do wild card search for key 'ABC*' ? Is there
any operator like scanQuery or is there any way without using SQL query.
Thanks.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to do wildcard search by key in ignite?
Posted by Stephen Darlington <st...@gridgain.com>.
If you create an index on (A,B,C), SQL queries for all three variants you note should work and use the index.
Having said that, “returning a huge number of rows” doesn’t seem like a good usage pattern with Ignite. You might be better distributing your query around the cluster rather than returning lots of records to a client. And depending on what proportion of records you’re returning, a scan query may be more efficient.
> On 30 Jan 2020, at 13:32, Tunas <si...@msci.com> wrote:
>
> I have key composed of "A"+"B"+"C"+Id.
>
> I want to do search by "ABC*" or"AB*" or "A*".
> Require very efficient query as i am fetching huge number of rows from cache
> based on above query.
> Also my values are serialized customize big user objects?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/