You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Ryan McKinley <ry...@gmail.com> on 2011/04/06 20:12:13 UTC
[POLL] JTS compile/test dependency
Some may be following the thread on spatial development... here is a
quick summary, and a poll to help decide what may be the best next
move.
I'm hoping to introduce a high level spatial API that can be used for
a variety of indexing strategies and computational needs. For simple
point in BBox and point in WGS84 radius, this does not require any
external libraries. To support more complex queries -- point in
polygon, complex geometry intersections, etc -- we need an LGPL
library JTS. The LGPL dependency is only needed to compile/test,
there is no runtime requirement for JTS. To enable the more
complicated options you would need to add JTS to the classpath and
perhaps set a environment variable. This is essentially what we are
now doing with the (soon to be removed) bdb contrib.
I am trying to figure out the best home for this code and development
to live. I think it is essential for the JTS support to be part of
the core build/test -- splitting it into a separate module that is
tested elsewhere is not an option. This raises the basic question of
if people are willing to have the LGPL build dependency as part of the
main lucene build. I think it is, but am sympathetic to the idea that
it might not be.
If the JTS dependency is not acceptable, then we will look for a good
home elsewhere (maybe in apache, maybe at osgeo).
Before doing the work to actually integrate with the lucene build
system, it would be nice to see what the general feeling is about the
plan. If folks are OK with the idea, we can make a concrete
patch/branch to flush out the details
[] OK with JTS compile dependency. Spatial support should be a module
[] OK with JTS, but think this spatial stuff should happen elsewhere
[] Please, no LGPL dependencies in lucene build
thanks
ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by "Smiley, David W." <ds...@mitre.org>.
On Apr 6, 2011, at 2:12 PM, Ryan McKinley wrote:
> [ ] OK with JTS compile dependency. Spatial support should be a module
> [X] OK with JTS, but think this spatial stuff should happen elsewhere
> [ ] Please, no LGPL dependencies in lucene build
~ David Smiley
Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Ryan McKinley <ry...@gmail.com>.
On Wed, Apr 6, 2011 at 2:48 PM, Grant Ingersoll
<gr...@gmail.com> wrote:
>
> On Apr 6, 2011, at 2:44 PM, Ryan McKinley wrote:
>
>> On Wed, Apr 6, 2011 at 2:39 PM, Grant Ingersoll
>> <gr...@gmail.com> wrote:
>>>
>>> I don't see why we need a compile/test dependency is needed at all:
>>> We provide a factory based spatial module where one specifies a
>>> SpatialProvider. We have our own implementation of that which works for
>>> some set (or all) of the features. An external project (Apache Extras?)
>>
>> This is the non-starter for me. This would split the dev across
>> multiple places and mean that the implementations I use (JTS) would
>> not be a first class citizen in testing.
>>
>> This is the point of the whole debate... and why i think elsewhere may
>> be a better option.
>
>
> That's a bit contradictory, though, isn't it? By definition, elsewhere means split too,
I'm looking at the proposed spatial strategy stuff as a unit. It is
obviously related to existing stuff, but is a very different thing.
> b/c we have stated the point search stuff isn't going anywhere.
Agree -- i think the two would live happily together. Parts of
existing point stuff may be deprecated if that seems appropriate. But
other parts -- especially the general vector based function queries
would never map to a high level spatial API anyway.
>And even if it does, you will still need to have a separate factory based implementation and ship a non-JTS provider, otherwise none of it can be packaged into a L/S release, so it's still the same amount of work.
IIUC, we can distribute classes that were compiled against the JTS
API, but not JTS itself. People could register what provider should
get used and if JTS is available, it would load that one via
reflection.
ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Grant Ingersoll <gr...@gmail.com>.
On Apr 6, 2011, at 2:44 PM, Ryan McKinley wrote:
> On Wed, Apr 6, 2011 at 2:39 PM, Grant Ingersoll
> <gr...@gmail.com> wrote:
>>
>> I don't see why we need a compile/test dependency is needed at all:
>> We provide a factory based spatial module where one specifies a
>> SpatialProvider. We have our own implementation of that which works for
>> some set (or all) of the features. An external project (Apache Extras?)
>
> This is the non-starter for me. This would split the dev across
> multiple places and mean that the implementations I use (JTS) would
> not be a first class citizen in testing.
>
> This is the point of the whole debate... and why i think elsewhere may
> be a better option.
That's a bit contradictory, though, isn't it? By definition, elsewhere means split too, b/c we have stated the point search stuff isn't going anywhere. And even if it does, you will still need to have a separate factory based implementation and ship a non-JTS provider, otherwise none of it can be packaged into a L/S release, so it's still the same amount of work.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Ryan McKinley <ry...@gmail.com>.
On Wed, Apr 6, 2011 at 2:39 PM, Grant Ingersoll
<gr...@gmail.com> wrote:
>
> I don't see why we need a compile/test dependency is needed at all:
> We provide a factory based spatial module where one specifies a
> SpatialProvider. We have our own implementation of that which works for
> some set (or all) of the features. An external project (Apache Extras?)
This is the non-starter for me. This would split the dev across
multiple places and mean that the implementations I use (JTS) would
not be a first class citizen in testing.
This is the point of the whole debate... and why i think elsewhere may
be a better option.
ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Bill Bell <bi...@gmail.com>.
I love this idea.!
Bill Bell
Sent from mobile
On Apr 6, 2011, at 2:39 PM, Grant Ingersoll <gr...@gmail.com> wrote:
>
> I don't see why we need a compile/test dependency is needed at all:
> We provide a factory based spatial module where one specifies a SpatialProvider. We have our own implementation of that which works for some set (or all) of the features. An external project (Apache Extras?) could then go and implement that provider using JTS and can easily leverage all of our existing tests as well as it's own (using the handy-dandy test framework). Users who wish to use this would then simply include the external JAR (accepting that it is LGPL on their own free will) and telling L/S to use a different Provider. I thought this is what you already proposed. This allows innovation on our stuff (which may well surpass JTS at some point) as well as satisfies the short term win of JTS w/o violating ASF legal issues (per http://www.apache.org/legal/3party.html#options-optional). It would also make it easy for SIS to add it's own provider if and when it is mature enough.
>
> -Grant
>
>
> On Apr 6, 2011, at 2:12 PM, Ryan McKinley wrote:
>>
>> [] OK with JTS compile dependency. Spatial support should be a module
>> [] OK with JTS, but think this spatial stuff should happen elsewhere
>> [] Please, no LGPL dependencies in lucene build
>
> [x] Please no LGPL in Lucene build, please keep spatial framework here, please implement JTS piece in Apache Extras per a well-defined (and hosted in Lucene) SpatialProvider/Factory mechanism that is completely pluggable. Compile dependency is in JTS needs Lucene spatial module, not Lucene spatial module needs JTS. :-)
>
> -Grant
Re: [POLL] JTS compile/test dependency
Posted by Grant Ingersoll <gr...@gmail.com>.
I don't see why we need a compile/test dependency is needed at all:
We provide a factory based spatial module where one specifies a SpatialProvider. We have our own implementation of that which works for some set (or all) of the features. An external project (Apache Extras?) could then go and implement that provider using JTS and can easily leverage all of our existing tests as well as it's own (using the handy-dandy test framework). Users who wish to use this would then simply include the external JAR (accepting that it is LGPL on their own free will) and telling L/S to use a different Provider. I thought this is what you already proposed. This allows innovation on our stuff (which may well surpass JTS at some point) as well as satisfies the short term win of JTS w/o violating ASF legal issues (per http://www.apache.org/legal/3party.html#options-optional). It would also make it easy for SIS to add it's own provider if and when it is mature enough.
-Grant
On Apr 6, 2011, at 2:12 PM, Ryan McKinley wrote:
>
> [] OK with JTS compile dependency. Spatial support should be a module
> [] OK with JTS, but think this spatial stuff should happen elsewhere
> [] Please, no LGPL dependencies in lucene build
[x] Please no LGPL in Lucene build, please keep spatial framework here, please implement JTS piece in Apache Extras per a well-defined (and hosted in Lucene) SpatialProvider/Factory mechanism that is completely pluggable. Compile dependency is in JTS needs Lucene spatial module, not Lucene spatial module needs JTS. :-)
-Grant
Re: [POLL] JTS compile/test dependency
Posted by Earwin Burrfoot <ea...@gmail.com>.
On Thu, Apr 7, 2011 at 01:11, Robert Muir <rc...@gmail.com> wrote:
> On Wed, Apr 6, 2011 at 5:07 PM, Earwin Burrfoot <ea...@gmail.com> wrote:
>>
>> Handling Unicode code points outside of BMP is highly expert stuff as
>> well. And is totally unneeded by 80% of the users for any other reason
>> except "elegance". I think you two guys can really understand each
>> other here : )
>>
>
> you are wrong: you either support unicode, or your application is
> buggy. Its not an optional feature, its the text standard used by the
> java programming language.
You either handle the the Earth as a proper somewhat-ellipsoid, or
your application is buggy. It's not an optional feature, it's even
stronger than a standard - it is a physical fact experienced by all of
us, earthlings.
Though 80% of the users can throw geoids and unicode planes out of the
window and live happily with some stupid local coordinate system and
two-byte characters (some even manage with one-byte!). Yeah, they
don't really care about being buggy in any geo/unicode-zealot's eyes.
Having said that, it's cool that people like you two exist :) Because
earth is round, maps are ugly, there are lots of different writing
systems and someone has to deal with that.
--
Kirill Zakharenko/Кирилл Захаренко
E-Mail/Jabber: earwin@gmail.com
Phone: +7 (495) 683-567-4
ICQ: 104465785
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Robert Muir <rc...@gmail.com>.
On Wed, Apr 6, 2011 at 5:07 PM, Earwin Burrfoot <ea...@gmail.com> wrote:
>
> Handling Unicode code points outside of BMP is highly expert stuff as
> well. And is totally unneeded by 80% of the users for any other reason
> except "elegance". I think you two guys can really understand each
> other here : )
>
you are wrong: you either support unicode, or your application is
buggy. Its not an optional feature, its the text standard used by the
java programming language.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Earwin Burrfoot <ea...@gmail.com>.
On Wed, Apr 6, 2011 at 22:43, Robert Muir <rc...@gmail.com> wrote:
> On Wed, Apr 6, 2011 at 2:12 PM, Ryan McKinley <ry...@gmail.com> wrote:
>> Some may be following the thread on spatial development... here is a
>> quick summary, and a poll to help decide what may be the best next
>> move.
>>
>> I'm hoping to introduce a high level spatial API that can be used for
>> a variety of indexing strategies and computational needs. For simple
>> point in BBox and point in WGS84 radius, this does not require any
>> external libraries. To support more complex queries -- point in
>> polygon, complex geometry intersections, etc -- we need an LGPL
>> library JTS. The LGPL dependency is only needed to compile/test,
>> there is no runtime requirement for JTS. To enable the more
>> complicated options you would need to add JTS to the classpath and
>> perhaps set a environment variable. This is essentially what we are
>> now doing with the (soon to be removed) bdb contrib.
>>
>> I am trying to figure out the best home for this code and development
>> to live. I think it is essential for the JTS support to be part of
>> the core build/test -- splitting it into a separate module that is
>> tested elsewhere is not an option. This raises the basic question of
>> if people are willing to have the LGPL build dependency as part of the
>> main lucene build. I think it is, but am sympathetic to the idea that
>> it might not be.
>
> I'm sorta confused about this (i'll probably offend someone here, but so be
> it)
> We have a contrib module for spatial that is experimental, people want to
> deprecate, and say has problems.
> Why must the super-expert-polygon stuff sit with the basic capability that
> probably most users want: the ability to do basic searches (probably in
> combination with text too) in their app?
> Its hard for me to tell, i hope the reason isn't "elegance", but why aren't
> we working on making a simple,supported,80-20 case in lucene that
> non-spatial-gurus (and users) understand and can maintain... then it would
> seem ideal for the complex stuff to be outside of this project with any
> dependencies it wants?
> Users are probably really confused about the spatial situation: is it
> because we are floundering around this expert stuff????
Handling Unicode code points outside of BMP is highly expert stuff as
well. And is totally unneeded by 80% of the users for any other reason
except "elegance". I think you two guys can really understand each
other here : )
--
Kirill Zakharenko/Кирилл Захаренко
E-Mail/Jabber: earwin@gmail.com
Phone: +7 (495) 683-567-4
ICQ: 104465785
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Ryan McKinley <ry...@gmail.com>.
On Wed, Apr 6, 2011 at 3:01 PM, Robert Muir <rc...@gmail.com> wrote:
> On Wed, Apr 6, 2011 at 2:54 PM, Ryan McKinley <ry...@gmail.com> wrote:
>>
>> The code can be separated so that the the dependencies are as you
>> suggest -- i have done this, but it makes testing more difficult and
>> less robust. As part of the framework I've introduced a robust way to
>> use the same data and and tests with different strategies and
>> implementations. For me to work on it, i need the stuff i use to be a
>> first class citizen in testing.
>>
>
> Right, but this creates a problem for our testing too: if we open this
> can of worms with optional LGPL stuff I think its going to actually
> complicate build and testing.
> I already stated my concerns about this here: http://s.apache.org/vE
>
> I don't think the bdb should be used as justification already that the
> "can of worms is already open". Personally I didn't realize the
> license it had, and for these same reasons, when i found this out i
> put up a patch on Grant's issue.
>
I totally agree -- this was my preface to the whole discussion, and
why i think it may be more appropriate to move spatial dev to an
environment that can have different compile time choices.
I'd like to figure a way that this is a win for everyone -- this is
why i'm bothering with the prolonged discussion so that at least
motivations are clear and all that.
ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Robert Muir <rc...@gmail.com>.
On Wed, Apr 6, 2011 at 2:54 PM, Ryan McKinley <ry...@gmail.com> wrote:
>
> The code can be separated so that the the dependencies are as you
> suggest -- i have done this, but it makes testing more difficult and
> less robust. As part of the framework I've introduced a robust way to
> use the same data and and tests with different strategies and
> implementations. For me to work on it, i need the stuff i use to be a
> first class citizen in testing.
>
Right, but this creates a problem for our testing too: if we open this
can of worms with optional LGPL stuff I think its going to actually
complicate build and testing.
I already stated my concerns about this here: http://s.apache.org/vE
I don't think the bdb should be used as justification already that the
"can of worms is already open". Personally I didn't realize the
license it had, and for these same reasons, when i found this out i
put up a patch on Grant's issue.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Grant Ingersoll <gs...@apache.org>.
On Apr 6, 2011, at 2:54 PM, Ryan McKinley wrote:
>> I'm sorta confused about this (i'll probably offend someone here, but so be it)
>
> Don't worry
>
>
>> Its hard for me to tell, i hope the reason isn't "elegance", but why aren't
>> we working on making a simple,supported,80-20 case in lucene that
>> non-spatial-gurus (and users) understand and can maintain...
>
> for me it is all about testing and development.
>
> For my needs I can't use the simple stuff, and *need* the features
> that many users won't care about. I have not done any work on the
> existing spatial contrib because it does not meet my needs.
>
> The code can be separated so that the the dependencies are as you
> suggest -- i have done this, but it makes testing more difficult and
> less robust. As part of the framework I've introduced a robust way to
> use the same data and and tests with different strategies and
> implementations. For me to work on it, i need the stuff i use to be a
> first class citizen in testing.
I don't follow why testing is any harder. The core interfaces and baseline implementation (along w/ point search) are tested here. The JTS project does it's own tests. You can certainly, on your machine, run the tests together. As I voted earlier, I think we should just define the interfaces here along w/ a baseline implementation that meets the 80/20 rule and the JTS project (or whatever else) lives somewhere else. I just don't see any valid way to bring in a compile/test dependency on JTS that we can support as a first class citizen, but that doesn't mean we can't support the framework which makes it easy to drop in and test on an individual's machine.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Ryan McKinley <ry...@gmail.com>.
> I'm sorta confused about this (i'll probably offend someone here, but so be it)
Don't worry
> Its hard for me to tell, i hope the reason isn't "elegance", but why aren't
> we working on making a simple,supported,80-20 case in lucene that
> non-spatial-gurus (and users) understand and can maintain...
for me it is all about testing and development.
For my needs I can't use the simple stuff, and *need* the features
that many users won't care about. I have not done any work on the
existing spatial contrib because it does not meet my needs.
The code can be separated so that the the dependencies are as you
suggest -- i have done this, but it makes testing more difficult and
less robust. As part of the framework I've introduced a robust way to
use the same data and and tests with different strategies and
implementations. For me to work on it, i need the stuff i use to be a
first class citizen in testing.
ryan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [POLL] JTS compile/test dependency
Posted by Robert Muir <rc...@gmail.com>.
On Wed, Apr 6, 2011 at 2:12 PM, Ryan McKinley <ry...@gmail.com> wrote:
> Some may be following the thread on spatial development... here is a
> quick summary, and a poll to help decide what may be the best next
> move.
>
> I'm hoping to introduce a high level spatial API that can be used for
> a variety of indexing strategies and computational needs. For simple
> point in BBox and point in WGS84 radius, this does not require any
> external libraries. To support more complex queries -- point in
> polygon, complex geometry intersections, etc -- we need an LGPL
> library JTS. The LGPL dependency is only needed to compile/test,
> there is no runtime requirement for JTS. To enable the more
> complicated options you would need to add JTS to the classpath and
> perhaps set a environment variable. This is essentially what we are
> now doing with the (soon to be removed) bdb contrib.
>
> I am trying to figure out the best home for this code and development
> to live. I think it is essential for the JTS support to be part of
> the core build/test -- splitting it into a separate module that is
> tested elsewhere is not an option. This raises the basic question of
> if people are willing to have the LGPL build dependency as part of the
> main lucene build. I think it is, but am sympathetic to the idea that
> it might not be.
>
I'm sorta confused about this (i'll probably offend someone here, but so be
it)
We have a contrib module for spatial that is experimental, people want to
deprecate, and say has problems.
Why must the super-expert-polygon stuff sit with the basic capability that
probably most users want: the ability to do basic searches (probably in
combination with text too) in their app?
Its hard for me to tell, i hope the reason isn't "elegance", but why aren't
we working on making a simple,supported,80-20 case in lucene that
non-spatial-gurus (and users) understand and can maintain... then it would
seem ideal for the complex stuff to be outside of this project with any
dependencies it wants?
Users are probably really confused about the spatial situation: is it
because we are floundering around this expert stuff????