You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by luqmanahmad <lu...@gmail.com> on 2017/11/20 15:45:27 UTC
SQL warning for partitioned caches with setLocal
Hi there,
Working with SQL queries with setLocal(true) with partitioned cache, it is
very easy for someone to run SQL queries without affinityRun or affinityCall
computations which are the preferred ways of running queries on partition
cache, as described in [1].
Now what I was thinking whenever a SQL is about to execute against
partitioned caches it should check for a check whether the call for this SQL
is made through an affinityRun or affinityCall function. If the call to SQL
is not part of affinityRun or affinityCall then by default it should log a
WARNING message or throw an exception which should be configurable in
CacheConfiguration. The advantage would be it won't break others code
instantly and allow them some time to fix it.
This can be achieved when the affinityCall or affinityRun method is called
we can set something specifically for SQL queries in the context which can
be read before executing the queries. If the SQL processor cannot find the
value in the given context for partitioned caches we can either log the
warning or throw an exception based on the cache configuration.
Let me know if it makes sense?
Thanks,
Luqman
[1]. https://apacheignite-sql.readme.io/docs/local-queries
<https://apacheignite-sql.readme.io/docs/local-queries>
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: SQL warning for partitioned caches with setLocal
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Hi Luqman,
Required caches are already derived from SQL query through Ignite SQL
internals. We should just re-use this code for local queries. I filed a
ticket to fix this [1].
[1] https://issues.apache.org/jira/browse/IGNITE-7039
On Wed, Nov 22, 2017 at 12:05 AM, Luqman Ahmad <lu...@gmail.com>
wrote:
> Hi Vladmir,
>
> Agree - they shouldnt be coupled togethor but what if we can set something
> in affinity api which can be read in sql api.
>
> Please correct me if I am wrong but in the affinityCall/Run we have to
> provide all the cache names and rebalancing will skip if there is already
> an operation in process. If we go with your approach not sure whether we
> can calculate all the related partitioned caches to be locked dynamically.
>
> Ofcourse you would be in a better position to comment on it but cant we
> introduce something in affinity api which can be set/read through each
> affinityCall/Run, and the affinity api can be used inside SQL api - just
> like the same way calculating partition id for a specific key or an finding
> an atomic reference.
>
> Thanks,
> Luqman
>
>
>
> On 21 Nov 2017 20:17, "Vladimir Ozerov" <vo...@gridgain.com> wrote:
>
> Hi Luqman,
>
> I do not think SQL and compute should be coupled in the product. Instead,
> we should fix local query execution and pin partitions in the same way it
> is done for affinityCall/Run and distributed SQL.
>
> On Tue, Nov 21, 2017 at 6:25 PM, luqmanahmad <lu...@gmail.com>
> wrote:
>
> > Thanks dsetrakyan,
> >
> > I would like to add a few more things over here which should be
> applicable
> > to partitioned caches.
> >
> > This context variable which is set through affinityCall or affinityRun
> > should be available through either a helper class or cache configuration.
> > There could be other advantages as well for example:
> >
> > 1. We can check the context variable in all the partitioned cache
> > operations. In department and employee example if an employee is accessed
> > without an affinityRun or affinityCall computation it should also log a
> > WARNING message or through an exception based on the cache configuration.
> >
> > 2. The user would be able to implement their own custom checks using it.
> > For
> > example, if we want to have some abstract level checks to restrict
> > developers to use specific functionality related to partitioned caches.
> >
> > Luqman
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>
Re: SQL warning for partitioned caches with setLocal
Posted by Luqman Ahmad <lu...@gmail.com>.
Hi Vladmir,
Agree - they shouldnt be coupled togethor but what if we can set something
in affinity api which can be read in sql api.
Please correct me if I am wrong but in the affinityCall/Run we have to
provide all the cache names and rebalancing will skip if there is already
an operation in process. If we go with your approach not sure whether we
can calculate all the related partitioned caches to be locked dynamically.
Ofcourse you would be in a better position to comment on it but cant we
introduce something in affinity api which can be set/read through each
affinityCall/Run, and the affinity api can be used inside SQL api - just
like the same way calculating partition id for a specific key or an finding
an atomic reference.
Thanks,
Luqman
On 21 Nov 2017 20:17, "Vladimir Ozerov" <vo...@gridgain.com> wrote:
Hi Luqman,
I do not think SQL and compute should be coupled in the product. Instead,
we should fix local query execution and pin partitions in the same way it
is done for affinityCall/Run and distributed SQL.
On Tue, Nov 21, 2017 at 6:25 PM, luqmanahmad <lu...@gmail.com> wrote:
> Thanks dsetrakyan,
>
> I would like to add a few more things over here which should be applicable
> to partitioned caches.
>
> This context variable which is set through affinityCall or affinityRun
> should be available through either a helper class or cache configuration.
> There could be other advantages as well for example:
>
> 1. We can check the context variable in all the partitioned cache
> operations. In department and employee example if an employee is accessed
> without an affinityRun or affinityCall computation it should also log a
> WARNING message or through an exception based on the cache configuration.
>
> 2. The user would be able to implement their own custom checks using it.
> For
> example, if we want to have some abstract level checks to restrict
> developers to use specific functionality related to partitioned caches.
>
> Luqman
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>
Re: SQL warning for partitioned caches with setLocal
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Hi Luqman,
I do not think SQL and compute should be coupled in the product. Instead,
we should fix local query execution and pin partitions in the same way it
is done for affinityCall/Run and distributed SQL.
On Tue, Nov 21, 2017 at 6:25 PM, luqmanahmad <lu...@gmail.com> wrote:
> Thanks dsetrakyan,
>
> I would like to add a few more things over here which should be applicable
> to partitioned caches.
>
> This context variable which is set through affinityCall or affinityRun
> should be available through either a helper class or cache configuration.
> There could be other advantages as well for example:
>
> 1. We can check the context variable in all the partitioned cache
> operations. In department and employee example if an employee is accessed
> without an affinityRun or affinityCall computation it should also log a
> WARNING message or through an exception based on the cache configuration.
>
> 2. The user would be able to implement their own custom checks using it.
> For
> example, if we want to have some abstract level checks to restrict
> developers to use specific functionality related to partitioned caches.
>
> Luqman
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>
Re: SQL warning for partitioned caches with setLocal
Posted by luqmanahmad <lu...@gmail.com>.
Thanks dsetrakyan,
I would like to add a few more things over here which should be applicable
to partitioned caches.
This context variable which is set through affinityCall or affinityRun
should be available through either a helper class or cache configuration.
There could be other advantages as well for example:
1. We can check the context variable in all the partitioned cache
operations. In department and employee example if an employee is accessed
without an affinityRun or affinityCall computation it should also log a
WARNING message or through an exception based on the cache configuration.
2. The user would be able to implement their own custom checks using it. For
example, if we want to have some abstract level checks to restrict
developers to use specific functionality related to partitioned caches.
Luqman
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: SQL warning for partitioned caches with setLocal
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Sounds like a good idea. Vladimir, would be nice to hear your thoughts.
D.
On Mon, Nov 20, 2017 at 7:45 AM, luqmanahmad <lu...@gmail.com> wrote:
> Hi there,
>
> Working with SQL queries with setLocal(true) with partitioned cache, it is
> very easy for someone to run SQL queries without affinityRun or
> affinityCall
> computations which are the preferred ways of running queries on partition
> cache, as described in [1].
>
> Now what I was thinking whenever a SQL is about to execute against
> partitioned caches it should check for a check whether the call for this
> SQL
> is made through an affinityRun or affinityCall function. If the call to SQL
> is not part of affinityRun or affinityCall then by default it should log a
> WARNING message or throw an exception which should be configurable in
> CacheConfiguration. The advantage would be it won't break others code
> instantly and allow them some time to fix it.
>
> This can be achieved when the affinityCall or affinityRun method is called
> we can set something specifically for SQL queries in the context which can
> be read before executing the queries. If the SQL processor cannot find the
> value in the given context for partitioned caches we can either log the
> warning or throw an exception based on the cache configuration.
>
> Let me know if it makes sense?
>
> Thanks,
> Luqman
>
> [1]. https://apacheignite-sql.readme.io/docs/local-queries
> <https://apacheignite-sql.readme.io/docs/local-queries>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>