You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ryan Josal <ry...@josal.com> on 2015/10/16 19:07:13 UTC
Forking Solr
Hi guys, I'd like to get your tips on how to run a Solr fork at my
company. I know Yonik has a "heliosearch" fork, and I'm sure many others
have a fork. There have been times where I want to add features to an
existing core plugin, and subclassing isn't possible so I end up copying
the source code into my repo, then using some crazy reflection to get it to
work. Sometimes there's a little bug in something and I have to do the
same thing. Sometimes there's something I want to do deeper in core Solr
code that isn't pluggable and I end up doing an interesting workaround.
Sometimes I want to apply a patch from JIRA. I also think forking solr
will make it easier for me to contribute patches back. So here are my
questions:
*) how do I properly fork it outside of github to my own company's git
system?
*) how do I pull new changes? I think I would expect to sync new changes
when there is a new public release. What branches do I need to work
with/on?
*) how do I test my changes? What part of the test suites do I run for
what changes?
*) how do I build a new version when I'm ready to go to prod? This is
slightly more unclear to me now that it isn't just a war.
Thanks,
Ryan
Re: Forking Solr
Posted by Roman Chyla <ro...@gmail.com>.
I've taken the route of extending solr, the repo checks out solr and builds
on top of that. The hard part was to figure out how to use solr test
classes and the default location for integration tests, but once there, it
is relatively easy. Google for montysolr, the repo is on github.
Roman
On Oct 16, 2015 10:52 PM, "Upayavira" <uv...@odoko.co.uk> wrote:
>
>
> On Fri, Oct 16, 2015, at 04:00 PM, Ryan Josal wrote:
> > Thanks for the feedback, forking lucene/solr is my last resort indeed.
> >
> > 1) It's not about creating fresh new plugins. It's about modifying
> > existing ones or core solr code.
> > 2) I want to submit the patch to modify core solr or lucene code, but I
> > also want to run it in prod before its accepted and released publicly.
> > Also I think this helps solidify the patch over time.
> > 3) I have to do this all the time, and I agree it's better than forking,
> > but doing this repeatedly over time has diminishing returns because it
> > increases the cost of upgrading solr. I also requires some ugly
> > reflection
> > in most cases, and in others copying verbatim a pile of other classes.
>
> If you want to patch a component, change its package name and fork that
> component. I have a custom MoreLikeThisHandler in production quite
> happily like this.
>
> I've also done an SVN checkout of Solr, made my code changes there, and
> then created a local git repo that I can track my own changes for stuff
> that will eventually get pushed back to Solr.
>
> I work concurrently on a number of patches to the Admin UI. They tend to
> sit in different JIRAs as patches for a few days before I commit them,
> so this local git repo makes it much easier for me to track my changes,
> but from the Solr community's perspective, I'm just using SVN.
>
> I could easily push this git repo up to github or such if I thought that
> added value.
>
> Then, I regularly run svn update which keeps this checkout up-to-date,
> and confirm it hasn't broken things.
>
> If you wanted to run against a specific version in Solr, you could force
> SVN to a specific revision (e.g. of the 5x branch) - the one that was
> released, and git merge your patches into it, etc, etc, etc.
>
> Upayavira
>
Re: Forking Solr
Posted by Upayavira <uv...@odoko.co.uk>.
On Fri, Oct 16, 2015, at 04:00 PM, Ryan Josal wrote:
> Thanks for the feedback, forking lucene/solr is my last resort indeed.
>
> 1) It's not about creating fresh new plugins. It's about modifying
> existing ones or core solr code.
> 2) I want to submit the patch to modify core solr or lucene code, but I
> also want to run it in prod before its accepted and released publicly.
> Also I think this helps solidify the patch over time.
> 3) I have to do this all the time, and I agree it's better than forking,
> but doing this repeatedly over time has diminishing returns because it
> increases the cost of upgrading solr. I also requires some ugly
> reflection
> in most cases, and in others copying verbatim a pile of other classes.
If you want to patch a component, change its package name and fork that
component. I have a custom MoreLikeThisHandler in production quite
happily like this.
I've also done an SVN checkout of Solr, made my code changes there, and
then created a local git repo that I can track my own changes for stuff
that will eventually get pushed back to Solr.
I work concurrently on a number of patches to the Admin UI. They tend to
sit in different JIRAs as patches for a few days before I commit them,
so this local git repo makes it much easier for me to track my changes,
but from the Solr community's perspective, I'm just using SVN.
I could easily push this git repo up to github or such if I thought that
added value.
Then, I regularly run svn update which keeps this checkout up-to-date,
and confirm it hasn't broken things.
If you wanted to run against a specific version in Solr, you could force
SVN to a specific revision (e.g. of the 5x branch) - the one that was
released, and git merge your patches into it, etc, etc, etc.
Upayavira
Re: Forking Solr
Posted by Ryan Josal <rj...@gmail.com>.
Thanks for the feedback, forking lucene/solr is my last resort indeed.
1) It's not about creating fresh new plugins. It's about modifying
existing ones or core solr code.
2) I want to submit the patch to modify core solr or lucene code, but I
also want to run it in prod before its accepted and released publicly.
Also I think this helps solidify the patch over time.
3) I have to do this all the time, and I agree it's better than forking,
but doing this repeatedly over time has diminishing returns because it
increases the cost of upgrading solr. I also requires some ugly reflection
in most cases, and in others copying verbatim a pile of other classes.
I will send my questions to lucene-dev, thanks!
Ryan
On Friday, October 16, 2015, Doug Turnbull <
dturnbull@opensourceconnections.com> wrote:
> Ryan,
>
> From a "solr-user" perspective :) I would advise against forking Solr. Some
> of our consulting business is "people who forked Solr, need to upgrade, and
> now have gotten themselves into hot water."
>
> I would try, in the following order
> 1. Creating a plugin (sounds like you can't do this)
> 2. Submitting a patch to Solr that makes it easier to create the plugin you
> need
> 3. Copy-pasting code to create a plugin. I once had to do this for a
> highlighter. It's ugly, but its better than forking.
> 4....
> 999. Hiring Yonik :)
> 1000. Forking Solr
>
> 999 a prereq for 1000 :)
>
> Even very heavily customized versions of Solr sold by major vendors that
> staff committers are entirely plugin driven.
>
> Cheers
> -Doug
>
>
> On Fri, Oct 16, 2015 at 3:30 PM, Alexandre Rafalovitch <arafalov@gmail.com
> <javascript:;>>
> wrote:
>
> > I suspect these questions should go the Lucene Dev list instead. This
> > one is more for those who build on top of standard Solr.
> >
> > Regards,
> > Alex.
> >
> > ----
> > Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
> > http://www.solr-start.com/
> >
> >
> > On 16 October 2015 at 12:07, Ryan Josal <ryan@josal.com <javascript:;>>
> wrote:
> > > Hi guys, I'd like to get your tips on how to run a Solr fork at my
> > > company. I know Yonik has a "heliosearch" fork, and I'm sure many
> others
> > > have a fork. There have been times where I want to add features to an
> > > existing core plugin, and subclassing isn't possible so I end up
> copying
> > > the source code into my repo, then using some crazy reflection to get
> it
> > to
> > > work. Sometimes there's a little bug in something and I have to do the
> > > same thing. Sometimes there's something I want to do deeper in core
> Solr
> > > code that isn't pluggable and I end up doing an interesting workaround.
> > > Sometimes I want to apply a patch from JIRA. I also think forking solr
> > > will make it easier for me to contribute patches back. So here are my
> > > questions:
> > >
> > > *) how do I properly fork it outside of github to my own company's git
> > > system?
> > > *) how do I pull new changes? I think I would expect to sync new
> changes
> > > when there is a new public release. What branches do I need to work
> > > with/on?
> > > *) how do I test my changes? What part of the test suites do I run for
> > > what changes?
> > > *) how do I build a new version when I'm ready to go to prod? This is
> > > slightly more unclear to me now that it isn't just a war.
> > >
> > > Thanks,
> > > Ryan
> >
>
>
>
> --
> *Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
> <http://opensourceconnections.com>, LLC | 240.476.9983
> Author: Relevant Search <http://manning.com/turnbull>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>
Re: Forking Solr
Posted by Doug Turnbull <dt...@opensourceconnections.com>.
Ryan,
>From a "solr-user" perspective :) I would advise against forking Solr. Some
of our consulting business is "people who forked Solr, need to upgrade, and
now have gotten themselves into hot water."
I would try, in the following order
1. Creating a plugin (sounds like you can't do this)
2. Submitting a patch to Solr that makes it easier to create the plugin you
need
3. Copy-pasting code to create a plugin. I once had to do this for a
highlighter. It's ugly, but its better than forking.
4....
999. Hiring Yonik :)
1000. Forking Solr
999 a prereq for 1000 :)
Even very heavily customized versions of Solr sold by major vendors that
staff committers are entirely plugin driven.
Cheers
-Doug
On Fri, Oct 16, 2015 at 3:30 PM, Alexandre Rafalovitch <ar...@gmail.com>
wrote:
> I suspect these questions should go the Lucene Dev list instead. This
> one is more for those who build on top of standard Solr.
>
> Regards,
> Alex.
>
> ----
> Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
> http://www.solr-start.com/
>
>
> On 16 October 2015 at 12:07, Ryan Josal <ry...@josal.com> wrote:
> > Hi guys, I'd like to get your tips on how to run a Solr fork at my
> > company. I know Yonik has a "heliosearch" fork, and I'm sure many others
> > have a fork. There have been times where I want to add features to an
> > existing core plugin, and subclassing isn't possible so I end up copying
> > the source code into my repo, then using some crazy reflection to get it
> to
> > work. Sometimes there's a little bug in something and I have to do the
> > same thing. Sometimes there's something I want to do deeper in core Solr
> > code that isn't pluggable and I end up doing an interesting workaround.
> > Sometimes I want to apply a patch from JIRA. I also think forking solr
> > will make it easier for me to contribute patches back. So here are my
> > questions:
> >
> > *) how do I properly fork it outside of github to my own company's git
> > system?
> > *) how do I pull new changes? I think I would expect to sync new changes
> > when there is a new public release. What branches do I need to work
> > with/on?
> > *) how do I test my changes? What part of the test suites do I run for
> > what changes?
> > *) how do I build a new version when I'm ready to go to prod? This is
> > slightly more unclear to me now that it isn't just a war.
> >
> > Thanks,
> > Ryan
>
--
*Doug Turnbull **| *Search Relevance Consultant | OpenSource Connections
<http://opensourceconnections.com>, LLC | 240.476.9983
Author: Relevant Search <http://manning.com/turnbull>
This e-mail and all contents, including attachments, is considered to be
Company Confidential unless explicitly stated otherwise, regardless
of whether attachments are marked as such.
Re: Forking Solr
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
I suspect these questions should go the Lucene Dev list instead. This
one is more for those who build on top of standard Solr.
Regards,
Alex.
----
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 16 October 2015 at 12:07, Ryan Josal <ry...@josal.com> wrote:
> Hi guys, I'd like to get your tips on how to run a Solr fork at my
> company. I know Yonik has a "heliosearch" fork, and I'm sure many others
> have a fork. There have been times where I want to add features to an
> existing core plugin, and subclassing isn't possible so I end up copying
> the source code into my repo, then using some crazy reflection to get it to
> work. Sometimes there's a little bug in something and I have to do the
> same thing. Sometimes there's something I want to do deeper in core Solr
> code that isn't pluggable and I end up doing an interesting workaround.
> Sometimes I want to apply a patch from JIRA. I also think forking solr
> will make it easier for me to contribute patches back. So here are my
> questions:
>
> *) how do I properly fork it outside of github to my own company's git
> system?
> *) how do I pull new changes? I think I would expect to sync new changes
> when there is a new public release. What branches do I need to work
> with/on?
> *) how do I test my changes? What part of the test suites do I run for
> what changes?
> *) how do I build a new version when I'm ready to go to prod? This is
> slightly more unclear to me now that it isn't just a war.
>
> Thanks,
> Ryan