You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Bryan Pendleton <bp...@amberpoint.com> on 2007/07/18 17:58:30 UTC
DERBY-581 and OLAP operations
In the context of https://issues.apache.org/jira/browse/DERBY-581
I've been studying the SQL-99 standard's specification of
OLAP operations:
- Feature T611 specifies "Elementary OLAP operations"
- Feature T612 specifies "Advanced OLAP operations"
I'm interested in exploring an implementation of these features,
and to start with I'd like to get the community's reaction:
- Are these features that we're interested in seeing added to Derby?
- Are there others in the community who are interested in
working on these features?
- How could we approach this incrementally, building enough
functionality to be useful, arriving at a complete
implementation in pieces over time? For example, would it be
reasonable to build a subset of the T611 features at first,
and then expand that functionality over time? Would it be
reasonable to add new SQL syntax support, but have it work
only in a subset of cases, and then expand the execution
functionality over time to address more usages?
I was thinking that I might get started by trying to build
one or more Wiki pages that describe some of the materials
that might go into an implementation:
- background information about the features and the
concepts behind them
- notes about the current Derby implementation, and about
possible ways to extend the implementation in this area
- ideas for how to subdivide and stage the implementation,
in order to enable us to start talking about actual code.
Does this seem like a reasonable approach?
thanks,
bryan
Re: DERBY-581 and OLAP operations
Posted by Ivo Jimenez <iv...@gmail.com>.
Thanks Narayanan! I'll go through the contents of the link you pointed,
experiment a little bit and then will go the the JIRA issue as you
suggested.
Good to know that Thomas is also joining!
Regards,
Ivo
On 7/19/07, V.Narayanan@sun.com <V....@sun.com> wrote:
>
> Hi,
>
> Welcome to derby!
>
> If you are confident of the issue you want to work on feel free to start
> working on a JIRA issue.
>
> But if you think you need to start with some simple issues and then move
> on to the issues
> you think are a little bit more complicated this page contains lot of
> information on how to
> start-up with the development process.
>
> http://wiki.apache.org/db-derby/ForNewDevelopers.
>
> The same page also explains how you can choose simple issues to start
> working on.
>
> Also pls do post any queries or doubts in the development process to
> derby-dev@db.apache.org ,
> you are sure to get responses.
>
> Hope this helps,
>
> Narayanan
>
>
> Ivo Jimenez wrote:
>
> > Hi,
> >
> > I'm new to Derby and I'd like to contribute. Casually, my interests
> > are on OLAP, so its great to read your proposition of adding this kind
> > of features. I have experience in Java and Databases but I don't know
> > if there's a certain level of knowledge that a contributor has to get
> > in order to help with coding. Should I start by doing things less
> > sensible? Or doing another kind of work?
> >
> > I could start by helping with the literature exploration and adding
> > content to the Wiki, as Bryan mentioned. I'd also like to code! So I'd
> > be very happy to be the second person that Manish pointed.
> >
> > Just to complement, as far as I know the OLAP extensions were annexed
> > to SQL on the 1999 standard (http://www.jcc.com/sql.htm
> > <http://www.jcc.com/sql.htm>) by annexing the CUBE and ROLLUP
> > operators (paul.rutgers.edu/~aminabdu/cs541/
> > <http://paul.rutgers.edu/%7Eaminabdu/cs541/>*cube*_op.pdf).
> >
> > Cheers,
> > Ivo
> >
> > On 7/18/07, *Daniel John Debrunner* <djd@apache.org
> > <ma...@apache.org>> wrote:
> >
> > Bryan Pendleton wrote:
> > > In the context of
> > https://issues.apache.org/jira/browse/DERBY-581
> > <https://issues.apache.org/jira/browse/DERBY-581>
> > > I've been studying the SQL-99 standard's specification of
> > > OLAP operations:
> > > - Feature T611 specifies "Elementary OLAP operations"
> > > - Feature T612 specifies "Advanced OLAP operations"
> > >
> > > I'm interested in exploring an implementation of these features,
> > > and to start with I'd like to get the community's reaction:
> > > - Are these features that we're interested in seeing added to
> > Derby?
> >
> > If one person wants to scratch that itch then that's all that's
> > required. Go for it! :-)
> >
> > > - Are there others in the community who are interested in
> > > working on these features?
> > > - How could we approach this incrementally, building enough
> > > functionality to be useful, arriving at a complete
> > > implementation in pieces over time? For example, would it be
> > > reasonable to build a subset of the T611 features at first,
> > > and then expand that functionality over time? Would it be
> > > reasonable to add new SQL syntax support, but have it work
> > > only in a subset of cases, and then expand the execution
> > > functionality over time to address more usages?
> >
> > Sounds good, progress, not perfection. Just try to avoid committing
> > changes that result in working behaviour that might change in the
> > future, i.e. a subset of working functionality is a great way to
> > start.
> >
> > >
> > > I was thinking that I might get started by trying to build
> > > one or more Wiki pages that describe some of the materials
> > > that might go into an implementation:
> > > - background information about the features and the
> > > concepts behind them
> > > - notes about the current Derby implementation, and about
> > > possible ways to extend the implementation in this area
> > > - ideas for how to subdivide and stage the implementation,
> > > in order to enable us to start talking about actual code.
> >
> > I would encourage extending Derby in ways that enhance the code
> > clarity.
> > For example, code might be cleaner by creating a new QueryTreeNode
> for
> > any new grouping functionality rather than more 'if' statements or
> > other
> > complex logic in an existing GroupByNode. Or even cleaning up
> > existing
> > nodes to allow sharing between various related nodes. E.g. move
> common
> > code from existing GroupByNode into an abstract node, and then have
> > multiple nodes that handle grouping operations extend from that.
> >
> > > Does this seem like a reasonable approach?
> >
> > +1
> >
> > Dan.
> >
> >
>
>
Re: DERBY-581 and OLAP operations
Posted by V....@Sun.COM.
Hi,
Welcome to derby!
If you are confident of the issue you want to work on feel free to start
working on a JIRA issue.
But if you think you need to start with some simple issues and then move
on to the issues
you think are a little bit more complicated this page contains lot of
information on how to
start-up with the development process.
http://wiki.apache.org/db-derby/ForNewDevelopers.
The same page also explains how you can choose simple issues to start
working on.
Also pls do post any queries or doubts in the development process to
derby-dev@db.apache.org ,
you are sure to get responses.
Hope this helps,
Narayanan
Ivo Jimenez wrote:
> Hi,
>
> I'm new to Derby and I'd like to contribute. Casually, my interests
> are on OLAP, so its great to read your proposition of adding this kind
> of features. I have experience in Java and Databases but I don't know
> if there's a certain level of knowledge that a contributor has to get
> in order to help with coding. Should I start by doing things less
> sensible? Or doing another kind of work?
>
> I could start by helping with the literature exploration and adding
> content to the Wiki, as Bryan mentioned. I'd also like to code! So I'd
> be very happy to be the second person that Manish pointed.
>
> Just to complement, as far as I know the OLAP extensions were annexed
> to SQL on the 1999 standard (http://www.jcc.com/sql.htm
> <http://www.jcc.com/sql.htm>) by annexing the CUBE and ROLLUP
> operators (paul.rutgers.edu/~aminabdu/cs541/
> <http://paul.rutgers.edu/%7Eaminabdu/cs541/>*cube*_op.pdf).
>
> Cheers,
> Ivo
>
> On 7/18/07, *Daniel John Debrunner* <djd@apache.org
> <ma...@apache.org>> wrote:
>
> Bryan Pendleton wrote:
> > In the context of
> https://issues.apache.org/jira/browse/DERBY-581
> <https://issues.apache.org/jira/browse/DERBY-581>
> > I've been studying the SQL-99 standard's specification of
> > OLAP operations:
> > - Feature T611 specifies "Elementary OLAP operations"
> > - Feature T612 specifies "Advanced OLAP operations"
> >
> > I'm interested in exploring an implementation of these features,
> > and to start with I'd like to get the community's reaction:
> > - Are these features that we're interested in seeing added to
> Derby?
>
> If one person wants to scratch that itch then that's all that's
> required. Go for it! :-)
>
> > - Are there others in the community who are interested in
> > working on these features?
> > - How could we approach this incrementally, building enough
> > functionality to be useful, arriving at a complete
> > implementation in pieces over time? For example, would it be
> > reasonable to build a subset of the T611 features at first,
> > and then expand that functionality over time? Would it be
> > reasonable to add new SQL syntax support, but have it work
> > only in a subset of cases, and then expand the execution
> > functionality over time to address more usages?
>
> Sounds good, progress, not perfection. Just try to avoid committing
> changes that result in working behaviour that might change in the
> future, i.e. a subset of working functionality is a great way to
> start.
>
> >
> > I was thinking that I might get started by trying to build
> > one or more Wiki pages that describe some of the materials
> > that might go into an implementation:
> > - background information about the features and the
> > concepts behind them
> > - notes about the current Derby implementation, and about
> > possible ways to extend the implementation in this area
> > - ideas for how to subdivide and stage the implementation,
> > in order to enable us to start talking about actual code.
>
> I would encourage extending Derby in ways that enhance the code
> clarity.
> For example, code might be cleaner by creating a new QueryTreeNode for
> any new grouping functionality rather than more 'if' statements or
> other
> complex logic in an existing GroupByNode. Or even cleaning up
> existing
> nodes to allow sharing between various related nodes. E.g. move common
> code from existing GroupByNode into an abstract node, and then have
> multiple nodes that handle grouping operations extend from that.
>
> > Does this seem like a reasonable approach?
>
> +1
>
> Dan.
>
>
Re: DERBY-581 and OLAP operations
Posted by Ivo Jimenez <iv...@gmail.com>.
Hi,
I'm new to Derby and I'd like to contribute. Casually, my interests are on
OLAP, so its great to read your proposition of adding this kind of features.
I have experience in Java and Databases but I don't know if there's a
certain level of knowledge that a contributor has to get in order to help
with coding. Should I start by doing things less sensible? Or doing another
kind of work?
I could start by helping with the literature exploration and adding content
to the Wiki, as Bryan mentioned. I'd also like to code! So I'd be very happy
to be the second person that Manish pointed.
Just to complement, as far as I know the OLAP extensions were annexed to SQL
on the 1999 standard (http://www.jcc.com/sql.htm) by annexing the CUBE and
ROLLUP operators (paul.rutgers.edu/~aminabdu/cs541/*cube*_op.pdf).
Cheers,
Ivo
On 7/18/07, Daniel John Debrunner <dj...@apache.org> wrote:
>
> Bryan Pendleton wrote:
> > In the context of https://issues.apache.org/jira/browse/DERBY-581
> > I've been studying the SQL-99 standard's specification of
> > OLAP operations:
> > - Feature T611 specifies "Elementary OLAP operations"
> > - Feature T612 specifies "Advanced OLAP operations"
> >
> > I'm interested in exploring an implementation of these features,
> > and to start with I'd like to get the community's reaction:
> > - Are these features that we're interested in seeing added to Derby?
>
> If one person wants to scratch that itch then that's all that's
> required. Go for it! :-)
>
> > - Are there others in the community who are interested in
> > working on these features?
> > - How could we approach this incrementally, building enough
> > functionality to be useful, arriving at a complete
> > implementation in pieces over time? For example, would it be
> > reasonable to build a subset of the T611 features at first,
> > and then expand that functionality over time? Would it be
> > reasonable to add new SQL syntax support, but have it work
> > only in a subset of cases, and then expand the execution
> > functionality over time to address more usages?
>
> Sounds good, progress, not perfection. Just try to avoid committing
> changes that result in working behaviour that might change in the
> future, i.e. a subset of working functionality is a great way to start.
>
> >
> > I was thinking that I might get started by trying to build
> > one or more Wiki pages that describe some of the materials
> > that might go into an implementation:
> > - background information about the features and the
> > concepts behind them
> > - notes about the current Derby implementation, and about
> > possible ways to extend the implementation in this area
> > - ideas for how to subdivide and stage the implementation,
> > in order to enable us to start talking about actual code.
>
> I would encourage extending Derby in ways that enhance the code clarity.
> For example, code might be cleaner by creating a new QueryTreeNode for
> any new grouping functionality rather than more 'if' statements or other
> complex logic in an existing GroupByNode. Or even cleaning up existing
> nodes to allow sharing between various related nodes. E.g. move common
> code from existing GroupByNode into an abstract node, and then have
> multiple nodes that handle grouping operations extend from that.
>
> > Does this seem like a reasonable approach?
>
> +1
>
> Dan.
>
Re: DERBY-581 and OLAP operations
Posted by Daniel John Debrunner <dj...@apache.org>.
Bryan Pendleton wrote:
> In the context of https://issues.apache.org/jira/browse/DERBY-581
> I've been studying the SQL-99 standard's specification of
> OLAP operations:
> - Feature T611 specifies "Elementary OLAP operations"
> - Feature T612 specifies "Advanced OLAP operations"
>
> I'm interested in exploring an implementation of these features,
> and to start with I'd like to get the community's reaction:
> - Are these features that we're interested in seeing added to Derby?
If one person wants to scratch that itch then that's all that's
required. Go for it! :-)
> - Are there others in the community who are interested in
> working on these features?
> - How could we approach this incrementally, building enough
> functionality to be useful, arriving at a complete
> implementation in pieces over time? For example, would it be
> reasonable to build a subset of the T611 features at first,
> and then expand that functionality over time? Would it be
> reasonable to add new SQL syntax support, but have it work
> only in a subset of cases, and then expand the execution
> functionality over time to address more usages?
Sounds good, progress, not perfection. Just try to avoid committing
changes that result in working behaviour that might change in the
future, i.e. a subset of working functionality is a great way to start.
>
> I was thinking that I might get started by trying to build
> one or more Wiki pages that describe some of the materials
> that might go into an implementation:
> - background information about the features and the
> concepts behind them
> - notes about the current Derby implementation, and about
> possible ways to extend the implementation in this area
> - ideas for how to subdivide and stage the implementation,
> in order to enable us to start talking about actual code.
I would encourage extending Derby in ways that enhance the code clarity.
For example, code might be cleaner by creating a new QueryTreeNode for
any new grouping functionality rather than more 'if' statements or other
complex logic in an existing GroupByNode. Or even cleaning up existing
nodes to allow sharing between various related nodes. E.g. move common
code from existing GroupByNode into an abstract node, and then have
multiple nodes that handle grouping operations extend from that.
> Does this seem like a reasonable approach?
+1
Dan.
Re: DERBY-581 and OLAP operations
Posted by Bryan Pendleton <bp...@amberpoint.com>.
Hi Manish, thank you very much for the encouraging words. I agree
that the more people who get familiar with the code and the
implementation, the better it is.
Regarding the SQL standard, I think the best thing I can do is
to refer you to http://en.wikipedia.org/wiki/SQL#Standardization
Also, I made a mistake in my earlier mail: it is the SQL 2003
standard that standardized the OLAP operations T611 and T612,
*not* the 1999 SQL standard. I will correct this in the Wiki pages.
thanks,
bryan
Re: DERBY-581 and OLAP operations
Posted by Manish Khettry <ma...@gmail.com>.
Bryan,
It sounds like a very reasonable and thoughtful way to get started. One
thing which is frustrating about working with Derby is that large portions
of the code are so very cryptic and brittle, sometimes unnceccsarily so.
Having more than one person try and understand relevant code, documenting it
and sharing this information would help in more ways than one.
The other thing-- is it possible to get a hold of the relevant portions of
the SQL standard that you refer to?
m
On 7/18/07, Bryan Pendleton <bp...@amberpoint.com> wrote:
>
> In the context of https://issues.apache.org/jira/browse/DERBY-581
> I've been studying the SQL-99 standard's specification of
> OLAP operations:
> - Feature T611 specifies "Elementary OLAP operations"
> - Feature T612 specifies "Advanced OLAP operations"
>
> I'm interested in exploring an implementation of these features,
> and to start with I'd like to get the community's reaction:
> - Are these features that we're interested in seeing added to Derby?
> - Are there others in the community who are interested in
> working on these features?
> - How could we approach this incrementally, building enough
> functionality to be useful, arriving at a complete
> implementation in pieces over time? For example, would it be
> reasonable to build a subset of the T611 features at first,
> and then expand that functionality over time? Would it be
> reasonable to add new SQL syntax support, but have it work
> only in a subset of cases, and then expand the execution
> functionality over time to address more usages?
>
> I was thinking that I might get started by trying to build
> one or more Wiki pages that describe some of the materials
> that might go into an implementation:
> - background information about the features and the
> concepts behind them
> - notes about the current Derby implementation, and about
> possible ways to extend the implementation in this area
> - ideas for how to subdivide and stage the implementation,
> in order to enable us to start talking about actual code.
>
> Does this seem like a reasonable approach?
>
> thanks,
>
> bryan
>
>
>
Re: DERBY-581 and OLAP operations
Posted by Thomas Nielsen <Th...@Sun.COM>.
Hi Bryan,
I'll join in on this effort with the limited Derby experience I have!
We should now be three, so there is truly some interest in this
functionallity.
I also agree that we should start with a subset of the functionallity as
suggested, and plug the rest in at a later stage.
Seems like you have a very sensible approach laid out :)
Cheers,
Thomas
Bryan Pendleton wrote:
> In the context of https://issues.apache.org/jira/browse/DERBY-581
> I've been studying the SQL-99 standard's specification of
> OLAP operations:
> - Feature T611 specifies "Elementary OLAP operations"
> - Feature T612 specifies "Advanced OLAP operations"
>
> I'm interested in exploring an implementation of these features,
> and to start with I'd like to get the community's reaction:
> - Are these features that we're interested in seeing added to Derby?
> - Are there others in the community who are interested in
> working on these features?
> - How could we approach this incrementally, building enough
> functionality to be useful, arriving at a complete
> implementation in pieces over time? For example, would it be
> reasonable to build a subset of the T611 features at first,
> and then expand that functionality over time? Would it be
> reasonable to add new SQL syntax support, but have it work
> only in a subset of cases, and then expand the execution
> functionality over time to address more usages?
> I was thinking that I might get started by trying to build
> one or more Wiki pages that describe some of the materials
> that might go into an implementation:
> - background information about the features and the
> concepts behind them
> - notes about the current Derby implementation, and about
> possible ways to extend the implementation in this area
> - ideas for how to subdivide and stage the implementation,
> in order to enable us to start talking about actual code.
>
> Does this seem like a reasonable approach?
Yes!
>
> thanks,
>
> bryan
>
>
--
Thomas Nielsen