Re: Code sync between Lucene and Solr, crossing Apache project boundaries, etc.

[Grant Ingersoll <gs...@apache.org>]

On Sep 22, 2009, at 1:54 PM, Mark Bennett wrote:

> To do any serious Solr debugging (or filter development) you also  
> need the
> Solr source code tree.  And you'd like them to be in sync, so that the
> Lucene code you see is exactly the same as what was used for the Solr
> version you're working with.
>
> I did find this link on sync'ing the two source tress, for a  
> specific Solr
> version.  But this seems a bit convoluted, is this really the right  
> answer?
> http://happygiraffe.net/blog/2009/07/16/solrs-lucene-source/
>
> Generally:
>
> 1: Given that many threads on the Solr mailing list seem to end with  
> "yeah,
> you could write some code to do that", it seems like most developers  
> would
> face this problem, and yet I haven't seen it discussed much (at  
> least not
> with the search terms I'm using, I keep finding stuff about SolrJ or  
> just
> getting the Solr code to build)
>
> 2: Do most people pick a version of Solr, then try carefully to sync  
> it with
> the exact version of Lucene code, or do they just not worry about the
> version stuff unless they get an error?

I'm probably atypical, but I check out the exact revision number of  
Lucene and associate it as source in IntelliJ.  Other times, I simply  
rely on the nightly Maven artifacts to get me all the correct versions  
of Lucene and Solr.

>
> 3: Or do most Solr/Lucene developers just live out of the Apache  
> nightly
> snapshots for both source code trees?  And if so, I have a question  
> about
> even doing that.  (see below)
>
> 4: Getting both projects to live together under Eclipse has seemed a  
> bit
> awkward, though again I'd think it's a common task.  Any good links  
> on that?
>
> 5: So do you most of you:
> a: Bother with both code trees?  (Solr and Lucene)

Yes

> b: Live from the command line with ant?  or

Yes

> c: Get them both living under Eclipse?  With dependencies back and  
> forth?  I
> haven't found a good resource for this yet.
> d: Do you also think maven, git and clover are required?

No, they are not, but I sometimes find it easier to get up and running  
in both Solr and Lucene by using the Maven artifacts.

>
> 6: Since advanced developers using Nutch and Solr would typically want
> Lucene's source code as well, wouldn't it be good to have separate
> distribution that includes all of those?  Or perhaps there's some  
> "apache
> thing" that makes this so trivial it's not worth bothering with?

Not sure, there hasn't been too much cross fertilization between the  
projects.

>
> Back to question 3, just using the nightly trees for both Solr and  
> Lucene,
> which presumably are in sync.  If you don't need a specific release,  
> this
> might be a reasonable workaround, but still a few of the details  
> bother me.
>
> Get the code:
> svn co http://svn.apache.org/repos/asf/lucene/java/trunk lucene- 
> nightly
> Checked out revision 817722.
> svn co http://svn.apache.org/repos/asf/lucene/solr/trunk solr-nightly
> Checked out revision 817722.
>
> * Note that both are revision 817722
>


I don't think this is correct.  For the Lucene version, you need to  
get the exact rev that Solr is expecting.  This can be seen in the  
Admin info page or by looking in the Manifest.MF in the Jars or in the  
CHANGES.txt file.

> Now do builds:
> cd lucene-nightly / ant / cd ..
> cd solr-nightly / ant / cd ..
>
> * OK, fine, no clover, I guess I'll live...
>
> Now check the two Lucene core jar files:
> proj $ ls -l lucene-nightly/build/lucene-core-2.9.jar
> -rw-r--r--  1 xyz  staff  1104037 Sep 22 09:44
> lucene-nightly/build/lucene-core-2.9.jar
> proj $ ls -l solr-nightly/lib/lucene-core-2.9.0.jar
> -rw-r--r--  1 xyz  staff  1104049 Sep 22 09:43
> solr-nightly/lib/lucene-core-2.9.0.jar
>
> * Notice the sizes.  They're pretty close, but not identical.  The  
> jar in
> lucene-nightly was created by my tools, whereas the lucene jar in  
> Solr was
> created on another machine.
>
> To prove this use Subversion status command:
> proj $ svn status lucene-nightly/build/lucene-core-2.9.jar
> svn: warning: 'lucene-nightly/build/lucene-core-2.9.jar' is not a  
> working
> copy
> So... not under version control, because it's built when I run ant.
> proj $ svn status solr-nightly/lib/lucene-core-2.9.0.jar
> (no output)
> No complaints from Subversion, so we know that it IS under version  
> control
> and therefore came from the apache, and I haven't messed with it.
>
> So I'd chalk up the difference exclusively to that then?
>
> If folks work with both code trees a lot, maybe having a parent  
> build file
> could copy over the fresh Lucene jar over to Solr.  Also curious if  
> there's
> an automated way to get this working in Eclipse.

I have a little script to copy over the correct things from the Lucene  
build somewhere in JIRA, but it's not really a big deal to do it by  
hand.

--------------------------
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)  
using Solr/Lucene:
http://www.lucidimagination.com/search