You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Yanna elina <ya...@gmail.com> on 2019/10/16 11:57:34 UTC
CassandraAdapter (Add Type) and WHERE statement.
Hi guys ,
I study Calcite the benefits that a Cassandra-Calcite Adapter can bring ,
as for example brings the possibility of join.
the problem type defined into CassandraSchema.getRelDataType(..) is very
limited
some important type are missing boolean / array ect...
I thought inherited from the class CassandraSchema for Override this
method and add more type but this method is used inside CassandraTable too.
i would like to avoid to re-write fully this adapter :)
do you have suggestions?
My second question is : Cassandra is not optimized to have WHERE on key
not defined on cluster/partition key. I was wondering if calcite could play
a role without this mechanism to improve performance
Thank !
Yanna
Re: CassandraAdapter (Add Type) and WHERE statement.
Posted by Yanna elina <ya...@gmail.com>.
I think it's me who does not have to understand all the subtlety.
I thought that STREAM works more like a in-memory- relational database
but i missed something thank for your help :)
Le jeu. 17 oct. 2019 à 15:53, Michael Mior <mm...@apache.org> a écrit :
> Perhaps I'm missing something, but I don't see why this would be any
> more efficient. Selecting all data is also not an efficient operation
> in Cassandra. Using ALLOW FILTERING will likely be more efficient
> since it's basically the same as doing a table scan, but it avoids
> returning data which would later be filtered by Calcite anyway.
> --
> Michael Mior
> mmior@apache.org
>
> Le jeu. 17 oct. 2019 à 09:13, Yanna elina <ya...@gmail.com> a
> écrit :
> >
> > Thank for reply Michael.
> >
> > yes i understood this on the documentation for example with "WHERE"
> > statement calcite i force the . "ALLOW FILTERING; "
> > and this can be expensive.
> >
> > I think there may be an interesting approach using STREAM.
> >
> > for example maintain a regular update between a cassandra TABLE and a
> > STREAM TABLE.
> >
> > CASSANDRA_TABLE_A . (SELECT * FROM TABLE_A) --------> STREAM_TABLE_A .
> > SELECT STREAM * FROM STREAM_TABLE_A WHERE username = 'JmuhsAaMdw'
> >
> > i guess it will be more efficient to directly make the WHERE from the
> > STREAM than the cassandra_adapter using "allow filtering"
> > a synchronization strategy can be set up between the cassandra table and
> > the STREAM table
> > what is your opinion about this approach ?
> > Thanks !
> > Yana
> >
> >
> > Le mer. 16 oct. 2019 à 17:08, Michael Mior <mm...@apache.org> a écrit :
> >
> > > You're right that there are several types which are not supported by
> > > the Cassandra adapter. We would happily accept pull requests to add
> > > support for new types.
> > >
> > > You're also correct that Cassandra cannot efficiently execute queries
> > > which do not specify the partition key. Calcite will make those
> > > queries more efficient, but it can make it easier to execute queries
> > > that CQL does not directly support. Ultimately data is still stored
> > > based on the partition key, so if your query does not specify a
> > > partition key, Calcite will still need to issue an expensive
> > > cross-partition query to Cassandra.
> > > --
> > > Michael Mior
> > > mmior@apache.org
> > >
> > > Le mer. 16 oct. 2019 à 07:57, Yanna elina <ya...@gmail.com>
> a
> > > écrit :
> > > >
> > > > Hi guys ,
> > > >
> > > > I study Calcite the benefits that a Cassandra-Calcite Adapter can
> bring ,
> > > > as for example brings the possibility of join.
> > > >
> > > > the problem type defined into CassandraSchema.getRelDataType(..) is
> very
> > > > limited
> > > > some important type are missing boolean / array ect...
> > > >
> > > > I thought inherited from the class CassandraSchema for Override this
> > > > method and add more type but this method is used inside
> CassandraTable
> > > too.
> > > >
> > > > i would like to avoid to re-write fully this adapter :)
> > > >
> > > > do you have suggestions?
> > > >
> > > > My second question is : Cassandra is not optimized to have WHERE on
> key
> > > > not defined on cluster/partition key. I was wondering if calcite
> could
> > > play
> > > > a role without this mechanism to improve performance
> > > >
> > > >
> > > > Thank !
> > > >
> > > > Yanna
> > >
>
Re: CassandraAdapter (Add Type) and WHERE statement.
Posted by Michael Mior <mm...@apache.org>.
Perhaps I'm missing something, but I don't see why this would be any
more efficient. Selecting all data is also not an efficient operation
in Cassandra. Using ALLOW FILTERING will likely be more efficient
since it's basically the same as doing a table scan, but it avoids
returning data which would later be filtered by Calcite anyway.
--
Michael Mior
mmior@apache.org
Le jeu. 17 oct. 2019 à 09:13, Yanna elina <ya...@gmail.com> a écrit :
>
> Thank for reply Michael.
>
> yes i understood this on the documentation for example with "WHERE"
> statement calcite i force the . "ALLOW FILTERING; "
> and this can be expensive.
>
> I think there may be an interesting approach using STREAM.
>
> for example maintain a regular update between a cassandra TABLE and a
> STREAM TABLE.
>
> CASSANDRA_TABLE_A . (SELECT * FROM TABLE_A) --------> STREAM_TABLE_A .
> SELECT STREAM * FROM STREAM_TABLE_A WHERE username = 'JmuhsAaMdw'
>
> i guess it will be more efficient to directly make the WHERE from the
> STREAM than the cassandra_adapter using "allow filtering"
> a synchronization strategy can be set up between the cassandra table and
> the STREAM table
> what is your opinion about this approach ?
> Thanks !
> Yana
>
>
> Le mer. 16 oct. 2019 à 17:08, Michael Mior <mm...@apache.org> a écrit :
>
> > You're right that there are several types which are not supported by
> > the Cassandra adapter. We would happily accept pull requests to add
> > support for new types.
> >
> > You're also correct that Cassandra cannot efficiently execute queries
> > which do not specify the partition key. Calcite will make those
> > queries more efficient, but it can make it easier to execute queries
> > that CQL does not directly support. Ultimately data is still stored
> > based on the partition key, so if your query does not specify a
> > partition key, Calcite will still need to issue an expensive
> > cross-partition query to Cassandra.
> > --
> > Michael Mior
> > mmior@apache.org
> >
> > Le mer. 16 oct. 2019 à 07:57, Yanna elina <ya...@gmail.com> a
> > écrit :
> > >
> > > Hi guys ,
> > >
> > > I study Calcite the benefits that a Cassandra-Calcite Adapter can bring ,
> > > as for example brings the possibility of join.
> > >
> > > the problem type defined into CassandraSchema.getRelDataType(..) is very
> > > limited
> > > some important type are missing boolean / array ect...
> > >
> > > I thought inherited from the class CassandraSchema for Override this
> > > method and add more type but this method is used inside CassandraTable
> > too.
> > >
> > > i would like to avoid to re-write fully this adapter :)
> > >
> > > do you have suggestions?
> > >
> > > My second question is : Cassandra is not optimized to have WHERE on key
> > > not defined on cluster/partition key. I was wondering if calcite could
> > play
> > > a role without this mechanism to improve performance
> > >
> > >
> > > Thank !
> > >
> > > Yanna
> >
Re: CassandraAdapter (Add Type) and WHERE statement.
Posted by Yanna elina <ya...@gmail.com>.
Thank for reply Michael.
yes i understood this on the documentation for example with "WHERE"
statement calcite i force the . "ALLOW FILTERING; "
and this can be expensive.
I think there may be an interesting approach using STREAM.
for example maintain a regular update between a cassandra TABLE and a
STREAM TABLE.
CASSANDRA_TABLE_A . (SELECT * FROM TABLE_A) --------> STREAM_TABLE_A .
SELECT STREAM * FROM STREAM_TABLE_A WHERE username = 'JmuhsAaMdw'
i guess it will be more efficient to directly make the WHERE from the
STREAM than the cassandra_adapter using "allow filtering"
a synchronization strategy can be set up between the cassandra table and
the STREAM table
what is your opinion about this approach ?
Thanks !
Yana
Le mer. 16 oct. 2019 à 17:08, Michael Mior <mm...@apache.org> a écrit :
> You're right that there are several types which are not supported by
> the Cassandra adapter. We would happily accept pull requests to add
> support for new types.
>
> You're also correct that Cassandra cannot efficiently execute queries
> which do not specify the partition key. Calcite will make those
> queries more efficient, but it can make it easier to execute queries
> that CQL does not directly support. Ultimately data is still stored
> based on the partition key, so if your query does not specify a
> partition key, Calcite will still need to issue an expensive
> cross-partition query to Cassandra.
> --
> Michael Mior
> mmior@apache.org
>
> Le mer. 16 oct. 2019 à 07:57, Yanna elina <ya...@gmail.com> a
> écrit :
> >
> > Hi guys ,
> >
> > I study Calcite the benefits that a Cassandra-Calcite Adapter can bring ,
> > as for example brings the possibility of join.
> >
> > the problem type defined into CassandraSchema.getRelDataType(..) is very
> > limited
> > some important type are missing boolean / array ect...
> >
> > I thought inherited from the class CassandraSchema for Override this
> > method and add more type but this method is used inside CassandraTable
> too.
> >
> > i would like to avoid to re-write fully this adapter :)
> >
> > do you have suggestions?
> >
> > My second question is : Cassandra is not optimized to have WHERE on key
> > not defined on cluster/partition key. I was wondering if calcite could
> play
> > a role without this mechanism to improve performance
> >
> >
> > Thank !
> >
> > Yanna
>
Re: CassandraAdapter (Add Type) and WHERE statement.
Posted by Alessandro Solimando <al...@gmail.com>.
Hello,
I have logged a Jira ticket for this:
https://issues.apache.org/jira/browse/CALCITE-3465
I have listed all the data types for Cassandra 3.x, and I have tried to
compile a table with the current status, but there are few entries for
which I am not sure.
If you have time to contribute to the table or material I can use to fill
the blanks, it would be great.
Best regards,
Alessandro
On Wed, 16 Oct 2019 at 17:08, Michael Mior <mm...@apache.org> wrote:
> You're right that there are several types which are not supported by
> the Cassandra adapter. We would happily accept pull requests to add
> support for new types.
>
> You're also correct that Cassandra cannot efficiently execute queries
> which do not specify the partition key. Calcite will make those
> queries more efficient, but it can make it easier to execute queries
> that CQL does not directly support. Ultimately data is still stored
> based on the partition key, so if your query does not specify a
> partition key, Calcite will still need to issue an expensive
> cross-partition query to Cassandra.
> --
> Michael Mior
> mmior@apache.org
>
> Le mer. 16 oct. 2019 à 07:57, Yanna elina <ya...@gmail.com> a
> écrit :
> >
> > Hi guys ,
> >
> > I study Calcite the benefits that a Cassandra-Calcite Adapter can bring ,
> > as for example brings the possibility of join.
> >
> > the problem type defined into CassandraSchema.getRelDataType(..) is very
> > limited
> > some important type are missing boolean / array ect...
> >
> > I thought inherited from the class CassandraSchema for Override this
> > method and add more type but this method is used inside CassandraTable
> too.
> >
> > i would like to avoid to re-write fully this adapter :)
> >
> > do you have suggestions?
> >
> > My second question is : Cassandra is not optimized to have WHERE on key
> > not defined on cluster/partition key. I was wondering if calcite could
> play
> > a role without this mechanism to improve performance
> >
> >
> > Thank !
> >
> > Yanna
>
Re: CassandraAdapter (Add Type) and WHERE statement.
Posted by Michael Mior <mm...@apache.org>.
You're right that there are several types which are not supported by
the Cassandra adapter. We would happily accept pull requests to add
support for new types.
You're also correct that Cassandra cannot efficiently execute queries
which do not specify the partition key. Calcite will make those
queries more efficient, but it can make it easier to execute queries
that CQL does not directly support. Ultimately data is still stored
based on the partition key, so if your query does not specify a
partition key, Calcite will still need to issue an expensive
cross-partition query to Cassandra.
--
Michael Mior
mmior@apache.org
Le mer. 16 oct. 2019 à 07:57, Yanna elina <ya...@gmail.com> a écrit :
>
> Hi guys ,
>
> I study Calcite the benefits that a Cassandra-Calcite Adapter can bring ,
> as for example brings the possibility of join.
>
> the problem type defined into CassandraSchema.getRelDataType(..) is very
> limited
> some important type are missing boolean / array ect...
>
> I thought inherited from the class CassandraSchema for Override this
> method and add more type but this method is used inside CassandraTable too.
>
> i would like to avoid to re-write fully this adapter :)
>
> do you have suggestions?
>
> My second question is : Cassandra is not optimized to have WHERE on key
> not defined on cluster/partition key. I was wondering if calcite could play
> a role without this mechanism to improve performance
>
>
> Thank !
>
> Yanna