You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Bryan Pendleton <bp...@gmail.com> on 2010/07/20 16:29:12 UTC
PlanExporter library requirements (DERBY-4587) (long message)
Hi all,
I wanted to share a few thoughts about the PlanExporter library handling.
I think that the core mission of PlanExporter is to enable the export
of query plan data in a complete and well-structured XML document, with
a documented schema, tested by our test suite.
I think that Nirmal's current patch in DERBY-4587 is extremely close
to meeting that goal. The Xalan requirements in the current patch are
primarily to do with how the XPlainStatisticsTest suite verifies the
correctness of the generated XML.
But it may be that the use of Xalan in the XplainStatisticsTest was
not ideal, and we should re-evaluate that technique. Perhaps there
is some other way to easily incorporate assertions about the correctness
of the XML documents that are generated during the test? Or, if we
can re-write the assertion logic in XplainStatisticsTest to use the
older version of Xalan that is already available in the tools/java
directory of the Derby trunk, that would be fine too. In fact, I think
that Nirmal is already considering various approaches to these problems.
Then, there are the more advanced usages of PlanExporter, which involve
further techniques for taking that generated XML data and processing
and formatting it for display and visualization of query plan behaviors.
I think we need to be cautious about how we handle external libraries
such as Xalan or JQuery in the PlanExporter functionality. Derby needs
to run on an extremely broad spectrum of JVM implementations, and it
also needs to retain its low footprint, both in code size and in
runtime resource requirements.
So I think we need to retain the basic principle that use of libraries
such as Xalan or JQuery must be as optional external libraries that
users can include if they wish for additional functionality, but also
can omit with no ill effects (except perhaps the loss of some PlanExporter
functionality). This is the way the other Derby XML-related features work.
Regarding JQuery, my expectation for dynamic HTML behaviors was that the
transformation of XML into well-formatted and dynamic HTML would be
performed entirely by the web browser, and would not involve any Java
code at all. I think that modern web browsers can do all the processing
of the XML entirely in the client.
I thought that we might provide one or more XSL stylesheets,
which were automatically invoked by the browser using a mechanism
such as <?xml-stylesheet> ( http://www.w3.org/TR/xml-stylesheet/ ), and
I thought that a user who wished to combine JQuery or any other external
AJAX-style library for dynamic browser behaviors would be providing and
hosting the JavaScript libraries in their own web browser environment,
not in something provided by us.
If is is possible to provide a Java-hosted mechanism for doing this, such
as the "-html" technique that Nirmal's latest patch provides, I think this
would be worth doing, but I think that we can provide a very rich and
full-featured visualization environment inside the web browser, using
just the XML export features of PlanExporter, and so I think that focusing
on making that code path solid and complete is of the most benefit to
the Derby community.
thanks,
bryan
Re: PlanExporter library requirements (DERBY-4587) (long message)
Posted by Nirmal Fernando <ni...@gmail.com>.
Hi All,
I like to add my thoughts as well. I understand now, that adding
external libraries such as Xalan or JQuery will affect the Derby's low
footprint, both in code size and in runtime resource requirements as
Bryan clearly explained. So, I think it's better for our users to
avoid those external libraries as possible as we can.
I investigated of using an old Xalan version that is compatible with
JDK 1.4, and I'm happy to inform you that I've successfully managed to
do that, and my latest patch resolved it.
Secondly about using JQuery, I think it's enough to give our users few
XSL style sheets which are in different styles, so they can choose
either one of them as their preferences or an external style sheet
developed by them. For these style sheets I'm planning to use CSS and
JS, by writing my own code.
I would like to know will it be good enough?
Highly appreciate all of your ideas on this tool.
Thanks.
Best Regards,
Nirmal
C.S.Nirmal J. Fernando
Department of Computer Science & Engineering,
Faculty of Engineering,
University of Moratuwa,
Sri Lanka.
Re: PlanExporter library requirements (DERBY-4587) (long message)
Posted by Mike Matrigali <mi...@sbcglobal.net>.
Rick Hillegas wrote:
> Kathey Marsden wrote:
>> On 7/20/2010 7:03 PM, Kathey Marsden wrote:
>>>
>>> I agree wholeheartedly as well with these points. As for my
>>> admittedly gut level concerns for adding the JQuery (non-Apache
>>> license) build dependency and necessary NOTICE update, I sent an
>>> email query to someone that understands these issues better than I
>>> and hope to hear back soon. I will post as soon as I understand it
>>> better.
>>>
>>
>> Well Nirmal, I still don't understand things yet, but don't think I
>> should ask you to wait any longer. I think you should investigate
>> build options that are as loosely coupled as possible as Bryan
>> suggests and ideally a build that will succeed if JQuery is not
>> present, but not build the functionality that needs it. This is what
>> we used to do with the osgi.jar requirement before we checked in the
>> felix source for building.
>>
>> Thanks
>>
>> Kathey
>>
>>
>>
>>
>>
> We put considerable effort into making it possible to build the entire
> product code from what's checked into subversion. The previous
> arrangement forced developers to hunt around on the web for other assets
> before they could even get a complete build. We need to continue to have
> a product which builds completely out of the box. A complicated build
> setup is a real turnoff to newcomers. In fact, it's the number 1
> community killer on this list compiled by Postgres core team member Josh
> Berkus: http://blogs.sun.com/javaone2008/entry/ten_ways_to_destroy_your
>
> -1 to regressing this situation.
Given the added dependency I wonder if this project/code would be better
in another project. I would rather see supporting tools on top of derby
handled by another project rather than increasing the dependencies and
complexities of the server.
Maybe concentrate on building things into Derby that allow open
extensions by add on's that are built separate from the base database
server component.
Would it fit in ddutils?
>
> Regards,
> -Rick
>
Re: PlanExporter library requirements (DERBY-4587) (long message)
Posted by Rick Hillegas <ri...@oracle.com>.
Kathey Marsden wrote:
> On 7/23/2010 5:54 AM, Rick Hillegas wrote:
>> Kathey Marsden wrote:
>>> On 7/20/2010 7:03 PM, Kathey Marsden wrote:
>>>>
>>>> I agree wholeheartedly as well with these points. As for my
>>>> admittedly gut level concerns for adding the JQuery (non-Apache
>>>> license) build dependency and necessary NOTICE update, I sent an
>>>> email query to someone that understands these issues better than I
>>>> and hope to hear back soon. I will post as soon as I understand it
>>>> better.
>>>>
>>>
>>> Well Nirmal, I still don't understand things yet, but don't think I
>>> should ask you to wait any longer. I think you should investigate
>>> build options that are as loosely coupled as possible as Bryan
>>> suggests and ideally a build that will succeed if JQuery is not
>>> present, but not build the functionality that needs it. This is
>>> what we used to do with the osgi.jar requirement before we checked
>>> in the felix source for building.
>>>
>>> Thanks
>>>
>>> Kathey
>>>
>>>
>>>
>>>
>>>
>> We need to continue to have a product which builds completely out of
>> the box.
> I think appropriately licensed stubs for the build would be ok if we
> don't repackage code, but as I said I don't understand the licensing
> considerations in this case and won't be back on the list for a couple
> weeks. I think a loosely coupled build would not be a regression as
> this is adding new functionality and would not regress any existing
> functionality. Some better long term build strategy could be
> implemented later and still let Nirmal finish his summer project. I
> agree that repackaging source in the product would be really bad.
I agree that we should avoid repackaging the JQuery source. Creating
stubs is probably a big chore--it may not buy us much either since
JQuery will evolve over time and the stubs will have to track that
evolution. Some kind of plugin architecture may be what's needed here.
>
> As an aside I think the build still has the Junit dependency, but
> maybe just for the tests. I haven't tried building just the product,
> but it would be great if someday that was not required for ant all.
That's right. We still have that extra hurdle for new Derby developers
who want to test their changes. But we have made big progress for users
who want to build a patch distribution from the head of a branch. They
will get a complete product if they just build what's checked into
subversion.
Regards,
-Rick
>
> Thanks
>
> Kathey
>
>
Re: PlanExporter library requirements (DERBY-4587) (long message)
Posted by Kathey Marsden <km...@sbcglobal.net>.
On 7/23/2010 5:54 AM, Rick Hillegas wrote:
> Kathey Marsden wrote:
>> On 7/20/2010 7:03 PM, Kathey Marsden wrote:
>>>
>>> I agree wholeheartedly as well with these points. As for my
>>> admittedly gut level concerns for adding the JQuery (non-Apache
>>> license) build dependency and necessary NOTICE update, I sent an
>>> email query to someone that understands these issues better than I
>>> and hope to hear back soon. I will post as soon as I understand it
>>> better.
>>>
>>
>> Well Nirmal, I still don't understand things yet, but don't think I
>> should ask you to wait any longer. I think you should investigate
>> build options that are as loosely coupled as possible as Bryan
>> suggests and ideally a build that will succeed if JQuery is not
>> present, but not build the functionality that needs it. This is what
>> we used to do with the osgi.jar requirement before we checked in the
>> felix source for building.
>>
>> Thanks
>>
>> Kathey
>>
>>
>>
>>
>>
> We need to continue to have a product which builds completely out of
> the box.
I think appropriately licensed stubs for the build would be ok if we
don't repackage code, but as I said I don't understand the licensing
considerations in this case and won't be back on the list for a couple
weeks. I think a loosely coupled build would not be a regression as
this is adding new functionality and would not regress any existing
functionality. Some better long term build strategy could be implemented
later and still let Nirmal finish his summer project. I agree that
repackaging source in the product would be really bad.
As an aside I think the build still has the Junit dependency, but maybe
just for the tests. I haven't tried building just the product, but it
would be great if someday that was not required for ant all.
Thanks
Kathey
Re: PlanExporter library requirements (DERBY-4587) (long message)
Posted by Rick Hillegas <ri...@oracle.com>.
Kathey Marsden wrote:
> On 7/20/2010 7:03 PM, Kathey Marsden wrote:
>>
>> I agree wholeheartedly as well with these points. As for my
>> admittedly gut level concerns for adding the JQuery (non-Apache
>> license) build dependency and necessary NOTICE update, I sent an
>> email query to someone that understands these issues better than I
>> and hope to hear back soon. I will post as soon as I understand it
>> better.
>>
>
> Well Nirmal, I still don't understand things yet, but don't think I
> should ask you to wait any longer. I think you should investigate
> build options that are as loosely coupled as possible as Bryan
> suggests and ideally a build that will succeed if JQuery is not
> present, but not build the functionality that needs it. This is what
> we used to do with the osgi.jar requirement before we checked in the
> felix source for building.
>
> Thanks
>
> Kathey
>
>
>
>
>
We put considerable effort into making it possible to build the entire
product code from what's checked into subversion. The previous
arrangement forced developers to hunt around on the web for other assets
before they could even get a complete build. We need to continue to have
a product which builds completely out of the box. A complicated build
setup is a real turnoff to newcomers. In fact, it's the number 1
community killer on this list compiled by Postgres core team member Josh
Berkus: http://blogs.sun.com/javaone2008/entry/ten_ways_to_destroy_your
-1 to regressing this situation.
Regards,
-Rick
Re: PlanExporter library requirements (DERBY-4587) (long message)
Posted by Kathey Marsden <km...@sbcglobal.net>.
On 7/20/2010 7:03 PM, Kathey Marsden wrote:
>
> I agree wholeheartedly as well with these points. As for my
> admittedly gut level concerns for adding the JQuery (non-Apache
> license) build dependency and necessary NOTICE update, I sent an
> email query to someone that understands these issues better than I and
> hope to hear back soon. I will post as soon as I understand it better.
>
Well Nirmal, I still don't understand things yet, but don't think I
should ask you to wait any longer. I think you should investigate build
options that are as loosely coupled as possible as Bryan suggests and
ideally a build that will succeed if JQuery is not present, but not
build the functionality that needs it. This is what we used to do with
the osgi.jar requirement before we checked in the felix source for building.
Thanks
Kathey
Re: PlanExporter library requirements (DERBY-4587) (long message)
Posted by Kathey Marsden <km...@sbcglobal.net>.
On 7/20/2010 10:39 AM, Mike Matrigali wrote:
> +1
> I especially agree
> with the following:
>
> Bryan Pendleton wrote:
>
>> I think we need to be cautious about how we handle external libraries
>> such as Xalan or JQuery in the PlanExporter functionality. Derby needs
>> to run on an extremely broad spectrum of JVM implementations, and it
>> also needs to retain its low footprint, both in code size and in
>> runtime resource requirements.
>>
>> So I think we need to retain the basic principle that use of libraries
>> such as Xalan or JQuery must be as optional external libraries that
>> users can include if they wish for additional functionality, but also
>> can omit with no ill effects (except perhaps the loss of some
>> PlanExporter
>> functionality). This is the way the other Derby XML-related features
>> work.
>>
>
>
>
I agree wholeheartedly as well with these points. As for my
admittedly gut level concerns for adding the JQuery (non-Apache
license) build dependency and necessary NOTICE update, I sent an email
query to someone that understands these issues better than I and hope to
hear back soon. I will post as soon as I understand it better.
Thanks
Kathey
Re: PlanExporter library requirements (DERBY-4587) (long message)
Posted by Mike Matrigali <mi...@sbcglobal.net>.
+1
This is a much better thought out response to Derby's use of
external libraries for non core features. I especially agree
with the following:
Bryan Pendleton wrote:
> I think we need to be cautious about how we handle external libraries
> such as Xalan or JQuery in the PlanExporter functionality. Derby needs
> to run on an extremely broad spectrum of JVM implementations, and it
> also needs to retain its low footprint, both in code size and in
> runtime resource requirements.
>
> So I think we need to retain the basic principle that use of libraries
> such as Xalan or JQuery must be as optional external libraries that
> users can include if they wish for additional functionality, but also
> can omit with no ill effects (except perhaps the loss of some PlanExporter
> functionality). This is the way the other Derby XML-related features work.
>