You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Erik Hatcher <er...@gmail.com> on 2016/09/12 23:05:11 UTC

Re: Miserable Experience ..... Again.

Aaron - I for one sympathize. When I pause to think of the stacks upon stacks of technologies that something like Solr are built upon… my head spins and I feel for the folks coming to computer science these days and having the whole Java and Big Data stacks and all that goes along with that (JVM/mem/GC up to network topology and architecture with 3xZK, plus NxM Solr’s, and beyond to data modeling, schema design, and query parameter adjusting).

---

It’s good for us to hear the ugly/painful side of folks experiences. It’s driven us to to where I find myself iterating with Solr in my day job like this….

$ bin/solr create -c my_collection
$ bin/post -c my_collection /data/docs.json

and http://… /select?q=…&wt=csv…

So “it works for me”, but that’s not a nice way to approach the struggles of users. Though we’ve come a long way, we’ve got a ways to go as well.

Erik

p.s. -

> Never mind the fact that the XML-based configuration process is an antiquated nightmare when the rest of the world has long since moved onto databases.

Well, to that point - the world that I work in really boils down to at least plain text (alas, mostly JSON these days, but even that’s an implementation detail) stuffed into git repositories, and played into new Solr environments by uploading configuration files, or more modernly, hitting the Solr configuration API’s to add/configure fields, set up request handlers, and the basics of what needs to be done. No XML needed these days. No (relational, JDBC) databases either, for that matter :)

> Maybe this will help someone else out there.

Thanks for taking the time to detail your struggles to the community. It is helpful to see where the rough edges are in this whole business, and smoothing them out. But it’s no easy business, having these stacks of dependencies and complexities piled on top of one another and trying to get it all fired up properly and usably.

Erik

Re: Miserable Experience ..... Again.

Posted by Joel Bernstein <jo...@gmail.com>.

I'm currently working on upgrading Alfresco from Solr 6.0 to Solr 6.2.
Should be easy. Think again. Lucene analyzer changes between Solr 6.0 and
Solr 6.2 and a new assert in ConjunctionDISI have caused days of work to
perform this simple upgrade.

Joel Bernstein
http://joelsolr.blogspot.com/

On Mon, Sep 12, 2016 at 7:05 PM, Erik Hatcher <er...@gmail.com>
wrote:

> Aaron - I for one sympathize.  When I pause to think of the stacks upon
> stacks of technologies that something like Solr are built upon… my head
> spins and I feel for the folks coming to computer science these days and
> having the whole Java and Big Data stacks and all that goes along with that
> (JVM/mem/GC up to network topology and architecture with 3xZK, plus NxM
> Solr’s, and beyond to data modeling, schema design, and query parameter
> adjusting).
>
> ---
>
> It’s good for us to hear the ugly/painful side of folks experiences.  It’s
> driven us to to where I find myself iterating with Solr in my day job like
> this….
>
>    $ bin/solr create -c my_collection
>    $ bin/post -c my_collection /data/docs.json
>
> and http://… /select?q=…&wt=csv…
>
> So “it works for me”, but that’s not a nice way to approach the struggles
> of users.   Though we’ve come a long way, we’ve got a ways to go as well.
>
>         Erik
>
> p.s. -
>
> > Never mind the fact that the XML-based configuration process is an
> antiquated nightmare when the rest of the world has long since moved onto
> databases.
>
> Well, to that point - the world that I work in really boils down to at
> least plain text (alas, mostly JSON these days, but even that’s an
> implementation detail) stuffed into git repositories, and played into new
> Solr environments by uploading configuration files, or more modernly,
> hitting the Solr configuration API’s to add/configure fields, set up
> request handlers, and the basics of what needs to be done.  No XML needed
> these days.   No (relational, JDBC) databases either, for that matter :)
>
> > Maybe this will help someone else out there.
>
> Thanks for taking the time to detail your struggles to the community.  It
> is helpful to see where the rough edges are in this whole business, and
> smoothing them out.   But it’s no easy business, having these stacks of
> dependencies and complexities piled on top of one another and trying to get
> it all fired up properly and usably.
>
>         Erik
>
>

Re: Miserable Experience ..... Again.

Posted by Bradley Belyeu <br...@life.church>.

I agree with what Erik wrote and some of Aaron’s original post. I’m relatively new to the Solr system (yes, pun intended) having just started diving into it a little over a year ago. I was in the “envious” position of being the only person who wanted to learn it and support it after our previous “expert” left the team. :D

It recently took me a full two week sprint cycle to upgrade our old style master-slaves cluster from 4.3 to 5.1 (we simultaneously went from java 1.7 to 1.8 and updated our NewRelic wrapper). There were many pain points along the way especially since I’m not much of a Java developer (I’m a Python/PHP/JS guy). BUT that being said I learned such an incredible amount over that sprint about the JVM, Solr configs & classes, caching & index tuning, and Lucene itself that I wouldn’t have changed it a bit. Ok, that’s a lie, I would have fixed a problem with NGrams before it caused a couple hour production outage. But basically, the amount of knowledge gained was very well worth the time and effort put into working through the upgrade pain points.

@Aaron another thing I’ve learned over the years is to only change 1 thing at a time, and definitely don’t mix troubleshooting with upgrading.
https://en.wikibooks.org/wiki/Computer_Programming_Principles/Maintaining/Debugging#Change_one_thing_at_a_time

One thing I don’t understand from your email is when you said you were “wondering why everything has to be so awful” about Solr. The initial problems you described were related to a MySQL database change and issues with Zookeeper. I think choosing to do the Java updates and Solr updates are what led to the other issues. The Solr docs packaged with the release do a fairly good job of explaining the breaking changes with each version. I chose to do all my updates one minor version at a time so I could keep up with the changes in the docs (4.3->4.4->…4.10->5.0->5.1) which took longer but made troubleshooting much easier.

As far as the user interface goes, at least it has one unlike some other popular search tools (cough ElasticSearch cough). And as far as XML based configuration is concerned, I personally prefer it to scripts with JSON blobs and Rest calls to set up your collections/entities/docs/etc. But that’s just b/c I’m weird.

I do agree that the error messages are often not helpful, BUT I found it easiest to just look at the source code to find the root cause of the issue. Which is what I think this boils down to, is that rather than complain, we should work to make Solr what we want it to be. If you can identify an issue, then you can either solve it yourself with a pull request or create a Jira issue asking for help to get it fixed.
Here’s a quote I love from Maya Angelou, “What you're supposed to do when you don't like a thing is change it. If you can't change it, change the way you think about it. Don't complain.”

On 9/12/16, 6:05 PM, "Erik Hatcher" <er...@gmail.com> wrote:

---

It’s good for us to hear the ugly/painful side of folks experiences. It’s driven us to to where I find myself iterating with Solr in my day job like this….

$ bin/solr create -c my_collection
$ bin/post -c my_collection /data/docs.json

and http://… /select?q=…&wt=csv…

So “it works for me”, but that’s not a nice way to approach the struggles of users. Though we’ve come a long way, we’ve got a ways to go as well.

Erik

p.s. -

> Never mind the fact that the XML-based configuration process is an antiquated nightmare when the rest of the world has long since moved onto databases.

> Maybe this will help someone else out there.

Erik