You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metamodel.apache.org by Alberto Rodriguez <ar...@gmail.com> on 2015/07/23 12:28:58 UTC
Scalar functions implementation
Hi all,
I've just started having a look at how to add scalar functions like
SUBSTRING, CONCAT, UPPER... to MetaModel.
I started by adding these functions to the MetaModel parser. There is
already a FunctionType enum that takes care of all the aggregate functions
(SUM, COUNT, AVG...) This FunctionType is used in the SelectItemParser to
find out the function type and parse it accordingly.
Not sure if I should add a new "ScalarFunctionType" enum or modify somehow
the existing one to take into account the new scalar functions. I do not
like the idea of having quite different "semantic" elements within the same
enum type but neither like the idea of having to ask everywhere whether the
function is scalar or not.
I'm thinking of creating kinda enum hierarchical data structure, but still
not sure how to do it because the FunctionType has several methods that are
very coupled to the aggregate functions (abstract AggregateBuilder,
evaluate...)
Any ideas?
Kind regards,
Alberto
Re: Scalar functions implementation
Posted by Alberto Rodriguez <ar...@gmail.com>.
Hi all,
sorry for not getting back earlier (quite busy at work).
Your first guidelines sound like a plan Kasper! I would start with a pretty
simple one, maybe SUBSTRING function is a good candidate?
Hopefully I'll be able to start implementing something around these ideas
in the next weeks.
Kind regards,
Alberto
2015-09-13 23:01 GMT+02:00 Kasper Sørensen <i....@gmail.com>:
> Hi Alberto,
>
> Wanted to revive this thread/topic a bit. Maybe we should give it a shot to
> implement some basic scalar functions in MetaModel?
>
> I guess obvious question will be "how?"... I think the basis we did with
> replacing the enum with an interface was good, but now we have to start
> supporting the branches in the code where it should deal differently with
> scalar functions as compared to aggregate functions.
>
> I guess there are two general scenarios to cover:
> * What to do when the backing datastore supports the scalar function
> natively, e.g. by rewriting a query. From my recollection only the JDBC
> module forces us to think about this while all other modules could fall
> back to the next scenario...
> * What to do when the backing datastore does not support the scala
> function natively. I would suggest here to ensure that
> QueryPostprocessDataContext can wrap an existing DataSet into a new DataSet
> implementation which evaluates the scalar functions when necesary.
>
> That's of course very high-level. But just to fuel a bit of energy on the
> subject :-)
>
> Best regards,
> Kasper
>
> 2015-07-23 13:06 GMT+02:00 Alberto Rodriguez <ar...@gmail.com>:
>
> > Hi Kasper,
> >
> > thank you for your quick response. I was having a look at some
> > stackoverflow threads [1][2] and was start thinking of moving to an
> > interface solution. With your response I've got a much clear idea.
> >
> > [1]http://stackoverflow.com/questions/6511453/extending-a-enum-in-java
> > [2]
> >
> >
> http://stackoverflow.com/questions/2642281/is-it-possible-to-extend-java-enums
> >
> > Kind regards,
> >
> > Alberto
> >
> > 2015-07-23 12:36 GMT+02:00 Kasper Sørensen <
> i.am.kasper.sorensen@gmail.com
> > >:
> >
> > > Hi Alberto,
> > >
> > > I am thinking that this reminds me a bit of the time (some releases
> back)
> > > when we converted ColumnType from an enum into an interface. That
> change
> > > opened up a lot of doors IMO. And a similar change to the FunctionType
> > > could do the trick.
> > >
> > > What I have in mind is:
> > >
> > > * Add an interface called AggregateFunction. This interface will have
> > the
> > > "aggregation specific" methods related to AggregateBuilder etc.
> > > * Add an interface called ScalarFunction. This interface will have
> some
> > > method that could be used to do the scalar transformation in memory
> (for
> > > those DataContext implementations that don't support the function
> > > natively).
> > > * Rename the current FunctionType enum into DefaultAggregateFunction
> and
> > > let it implement AggregateFunction
> > > * Add a new FunctionType _interface_ which has static final references
> > to
> > > the aggregate functions. Thereby we facilitate that existing code will
> > > work, although it will need to be recompiled.
> > >
> > > Once these changes have been done then we can start looking at parsers,
> > how
> > > to execute the different types of functions in different kinds of
> > > DataContext implementations etc. But having the split in the interfaces
> > > will allow us to much better generalize that I think.
> > >
> > > Best regards,
> > > Kasper
> > >
> > > 2015-07-23 12:28 GMT+02:00 Alberto Rodriguez <ar...@gmail.com>:
> > >
> > > > Hi all,
> > > >
> > > > I've just started having a look at how to add scalar functions like
> > > > SUBSTRING, CONCAT, UPPER... to MetaModel.
> > > >
> > > > I started by adding these functions to the MetaModel parser. There is
> > > > already a FunctionType enum that takes care of all the aggregate
> > > functions
> > > > (SUM, COUNT, AVG...) This FunctionType is used in the
> SelectItemParser
> > to
> > > > find out the function type and parse it accordingly.
> > > >
> > > > Not sure if I should add a new "ScalarFunctionType" enum or modify
> > > somehow
> > > > the existing one to take into account the new scalar functions. I do
> > not
> > > > like the idea of having quite different "semantic" elements within
> the
> > > same
> > > > enum type but neither like the idea of having to ask everywhere
> whether
> > > the
> > > > function is scalar or not.
> > > >
> > > > I'm thinking of creating kinda enum hierarchical data structure, but
> > > still
> > > > not sure how to do it because the FunctionType has several methods
> that
> > > are
> > > > very coupled to the aggregate functions (abstract AggregateBuilder,
> > > > evaluate...)
> > > >
> > > > Any ideas?
> > > >
> > > > Kind regards,
> > > >
> > > > Alberto
> > > >
> > >
> >
>
Re: Scalar functions implementation
Posted by Kasper Sørensen <i....@gmail.com>.
Hi Alberto,
Wanted to revive this thread/topic a bit. Maybe we should give it a shot to
implement some basic scalar functions in MetaModel?
I guess obvious question will be "how?"... I think the basis we did with
replacing the enum with an interface was good, but now we have to start
supporting the branches in the code where it should deal differently with
scalar functions as compared to aggregate functions.
I guess there are two general scenarios to cover:
* What to do when the backing datastore supports the scalar function
natively, e.g. by rewriting a query. From my recollection only the JDBC
module forces us to think about this while all other modules could fall
back to the next scenario...
* What to do when the backing datastore does not support the scala
function natively. I would suggest here to ensure that
QueryPostprocessDataContext can wrap an existing DataSet into a new DataSet
implementation which evaluates the scalar functions when necesary.
That's of course very high-level. But just to fuel a bit of energy on the
subject :-)
Best regards,
Kasper
2015-07-23 13:06 GMT+02:00 Alberto Rodriguez <ar...@gmail.com>:
> Hi Kasper,
>
> thank you for your quick response. I was having a look at some
> stackoverflow threads [1][2] and was start thinking of moving to an
> interface solution. With your response I've got a much clear idea.
>
> [1]http://stackoverflow.com/questions/6511453/extending-a-enum-in-java
> [2]
>
> http://stackoverflow.com/questions/2642281/is-it-possible-to-extend-java-enums
>
> Kind regards,
>
> Alberto
>
> 2015-07-23 12:36 GMT+02:00 Kasper Sørensen <i.am.kasper.sorensen@gmail.com
> >:
>
> > Hi Alberto,
> >
> > I am thinking that this reminds me a bit of the time (some releases back)
> > when we converted ColumnType from an enum into an interface. That change
> > opened up a lot of doors IMO. And a similar change to the FunctionType
> > could do the trick.
> >
> > What I have in mind is:
> >
> > * Add an interface called AggregateFunction. This interface will have
> the
> > "aggregation specific" methods related to AggregateBuilder etc.
> > * Add an interface called ScalarFunction. This interface will have some
> > method that could be used to do the scalar transformation in memory (for
> > those DataContext implementations that don't support the function
> > natively).
> > * Rename the current FunctionType enum into DefaultAggregateFunction and
> > let it implement AggregateFunction
> > * Add a new FunctionType _interface_ which has static final references
> to
> > the aggregate functions. Thereby we facilitate that existing code will
> > work, although it will need to be recompiled.
> >
> > Once these changes have been done then we can start looking at parsers,
> how
> > to execute the different types of functions in different kinds of
> > DataContext implementations etc. But having the split in the interfaces
> > will allow us to much better generalize that I think.
> >
> > Best regards,
> > Kasper
> >
> > 2015-07-23 12:28 GMT+02:00 Alberto Rodriguez <ar...@gmail.com>:
> >
> > > Hi all,
> > >
> > > I've just started having a look at how to add scalar functions like
> > > SUBSTRING, CONCAT, UPPER... to MetaModel.
> > >
> > > I started by adding these functions to the MetaModel parser. There is
> > > already a FunctionType enum that takes care of all the aggregate
> > functions
> > > (SUM, COUNT, AVG...) This FunctionType is used in the SelectItemParser
> to
> > > find out the function type and parse it accordingly.
> > >
> > > Not sure if I should add a new "ScalarFunctionType" enum or modify
> > somehow
> > > the existing one to take into account the new scalar functions. I do
> not
> > > like the idea of having quite different "semantic" elements within the
> > same
> > > enum type but neither like the idea of having to ask everywhere whether
> > the
> > > function is scalar or not.
> > >
> > > I'm thinking of creating kinda enum hierarchical data structure, but
> > still
> > > not sure how to do it because the FunctionType has several methods that
> > are
> > > very coupled to the aggregate functions (abstract AggregateBuilder,
> > > evaluate...)
> > >
> > > Any ideas?
> > >
> > > Kind regards,
> > >
> > > Alberto
> > >
> >
>
Re: Scalar functions implementation
Posted by Alberto Rodriguez <ar...@gmail.com>.
Hi Kasper,
thank you for your quick response. I was having a look at some
stackoverflow threads [1][2] and was start thinking of moving to an
interface solution. With your response I've got a much clear idea.
[1]http://stackoverflow.com/questions/6511453/extending-a-enum-in-java
[2]
http://stackoverflow.com/questions/2642281/is-it-possible-to-extend-java-enums
Kind regards,
Alberto
2015-07-23 12:36 GMT+02:00 Kasper Sørensen <i....@gmail.com>:
> Hi Alberto,
>
> I am thinking that this reminds me a bit of the time (some releases back)
> when we converted ColumnType from an enum into an interface. That change
> opened up a lot of doors IMO. And a similar change to the FunctionType
> could do the trick.
>
> What I have in mind is:
>
> * Add an interface called AggregateFunction. This interface will have the
> "aggregation specific" methods related to AggregateBuilder etc.
> * Add an interface called ScalarFunction. This interface will have some
> method that could be used to do the scalar transformation in memory (for
> those DataContext implementations that don't support the function
> natively).
> * Rename the current FunctionType enum into DefaultAggregateFunction and
> let it implement AggregateFunction
> * Add a new FunctionType _interface_ which has static final references to
> the aggregate functions. Thereby we facilitate that existing code will
> work, although it will need to be recompiled.
>
> Once these changes have been done then we can start looking at parsers, how
> to execute the different types of functions in different kinds of
> DataContext implementations etc. But having the split in the interfaces
> will allow us to much better generalize that I think.
>
> Best regards,
> Kasper
>
> 2015-07-23 12:28 GMT+02:00 Alberto Rodriguez <ar...@gmail.com>:
>
> > Hi all,
> >
> > I've just started having a look at how to add scalar functions like
> > SUBSTRING, CONCAT, UPPER... to MetaModel.
> >
> > I started by adding these functions to the MetaModel parser. There is
> > already a FunctionType enum that takes care of all the aggregate
> functions
> > (SUM, COUNT, AVG...) This FunctionType is used in the SelectItemParser to
> > find out the function type and parse it accordingly.
> >
> > Not sure if I should add a new "ScalarFunctionType" enum or modify
> somehow
> > the existing one to take into account the new scalar functions. I do not
> > like the idea of having quite different "semantic" elements within the
> same
> > enum type but neither like the idea of having to ask everywhere whether
> the
> > function is scalar or not.
> >
> > I'm thinking of creating kinda enum hierarchical data structure, but
> still
> > not sure how to do it because the FunctionType has several methods that
> are
> > very coupled to the aggregate functions (abstract AggregateBuilder,
> > evaluate...)
> >
> > Any ideas?
> >
> > Kind regards,
> >
> > Alberto
> >
>
Re: Scalar functions implementation
Posted by Kasper Sørensen <i....@gmail.com>.
Hi Alberto,
I am thinking that this reminds me a bit of the time (some releases back)
when we converted ColumnType from an enum into an interface. That change
opened up a lot of doors IMO. And a similar change to the FunctionType
could do the trick.
What I have in mind is:
* Add an interface called AggregateFunction. This interface will have the
"aggregation specific" methods related to AggregateBuilder etc.
* Add an interface called ScalarFunction. This interface will have some
method that could be used to do the scalar transformation in memory (for
those DataContext implementations that don't support the function natively).
* Rename the current FunctionType enum into DefaultAggregateFunction and
let it implement AggregateFunction
* Add a new FunctionType _interface_ which has static final references to
the aggregate functions. Thereby we facilitate that existing code will
work, although it will need to be recompiled.
Once these changes have been done then we can start looking at parsers, how
to execute the different types of functions in different kinds of
DataContext implementations etc. But having the split in the interfaces
will allow us to much better generalize that I think.
Best regards,
Kasper
2015-07-23 12:28 GMT+02:00 Alberto Rodriguez <ar...@gmail.com>:
> Hi all,
>
> I've just started having a look at how to add scalar functions like
> SUBSTRING, CONCAT, UPPER... to MetaModel.
>
> I started by adding these functions to the MetaModel parser. There is
> already a FunctionType enum that takes care of all the aggregate functions
> (SUM, COUNT, AVG...) This FunctionType is used in the SelectItemParser to
> find out the function type and parse it accordingly.
>
> Not sure if I should add a new "ScalarFunctionType" enum or modify somehow
> the existing one to take into account the new scalar functions. I do not
> like the idea of having quite different "semantic" elements within the same
> enum type but neither like the idea of having to ask everywhere whether the
> function is scalar or not.
>
> I'm thinking of creating kinda enum hierarchical data structure, but still
> not sure how to do it because the FunctionType has several methods that are
> very coupled to the aggregate functions (abstract AggregateBuilder,
> evaluate...)
>
> Any ideas?
>
> Kind regards,
>
> Alberto
>