You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Aaron Greenspan <aa...@PLAINSITE.ORG> on 2016/09/12 21:48:48 UTC
Miserable Experience Using Solr. Again.
Hi,
I have been on this list for some time because I know that any time I try to do anything related to Solr I’m going to have to spend hours on it, wondering why everything has to be so awful, and I just want somewhere to provide feedback with the dim hope that the product might improve one day. (So far, for my purposes, it hasn’t.) Sure enough, I still absolutely hate using Solr, and I have more feedback.
I started with a confusing error on the web console, which I still can’t figure out how to password protect without going through an insanely process involving "ZooKeeper," which I don’t know anything about, or have, to the best of my knowledge:
Problem accessing /solr/. Reason:
Forbidden
According to logs, this apparently meant that a MySQL query had failed due to a field name change. Since I would have to change my XML configuration files, I decided to use the opportunity to upgrade from Solr 5.1.4 to 6.2.0. It broke everything.
First I was getting errors about "Unsupported major.minor version 52.0", so I needed to install the Linux x64 JRE 1.8.0, which I managed on CentOS 6 with...
yum install openjdk-1.8.0
...going to Oracle’s web site, downloading the latest JRE 1.8 build, and then running...
yum localinstall jre-8u101-linux-x64.rpm
So far so good. But I didn’t have JAVA_HOME set properly apparently, so I needed to do the not-exactly-intuitive…
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.el6_8.x86_64/jre/
As usual, I manually moved over my mysql-connector-java-5.1.38-bin.jar file from the dist/ folder in the old version to the new one. Then after stopping the old process (with kill -9, since there seems to be no graceful way to shut down Solr—README.txt doesn’t mention bin/solr stop) I moved over my two core folders from the old server/solr/ folder. I tried to start it up with bin/solr start, and watched the errors roll in.
There was some kind of problem with StopFilterFactory and the text_general field type. Thanks to Stack Overflow I was able to determine that the apparent problem was that there was a parameter, previously fine, which was no longer fine. So I removed all instances of enablePositionIncrements="true". That helped, but then I ran into a broader error: "Plugin Initializing failure for [schema.xml] fieldType". It didn’t say which field type. Buried in the logs I found a reference in the Java stack trace—which *disappears* (and distorts the viewing window horribly) after a few seconds when you try to view it in the web log UI—to the string "units="degrees"". Sure enough, this string appeared in my schema.xml for a class called "solr.SpatialRecursivePrefixTreeFieldType" that I’m pretty sure I never use. I removed that parameter, and moved on to the next set of errors.
Apparently there is some aspect of the Thai text field type that Solr 6.2.0 doesn’t like. So I disabled it. I don’t use Thai text.
Now Solr was complaining about "Error loading class 'solr.admin.AdminHandlers'". So I found the reference to solr.admin.AdminHandlers in solrconfig.xml for each of my cores and commented it out. Only then did Solr work again.
This was not a smooth process. It took about two hours. The user interface is still as buggy as an early alpha of most products, the errors are difficult to understand when they don’t actually specify what’s wrong (and they almost never do), and there should have been an automatic process to highlight and fix problems in old (pre-6) configuration files. Never mind the fact that the XML-based configuration process is an antiquated nightmare when the rest of the world has long since moved onto databases.
Maybe this will help someone else out there.
Aaron
PlainSite | http://www.plainsite.org
Re: Miserable Experience ..... Again.
Posted by Joel Bernstein <jo...@gmail.com>.
I'm currently working on upgrading Alfresco from Solr 6.0 to Solr 6.2.
Should be easy. Think again. Lucene analyzer changes between Solr 6.0 and
Solr 6.2 and a new assert in ConjunctionDISI have caused days of work to
perform this simple upgrade.
Joel Bernstein
http://joelsolr.blogspot.com/
On Mon, Sep 12, 2016 at 7:05 PM, Erik Hatcher <er...@gmail.com>
wrote:
> Aaron - I for one sympathize. When I pause to think of the stacks upon
> stacks of technologies that something like Solr are built upon… my head
> spins and I feel for the folks coming to computer science these days and
> having the whole Java and Big Data stacks and all that goes along with that
> (JVM/mem/GC up to network topology and architecture with 3xZK, plus NxM
> Solr’s, and beyond to data modeling, schema design, and query parameter
> adjusting).
>
> ---
>
> It’s good for us to hear the ugly/painful side of folks experiences. It’s
> driven us to to where I find myself iterating with Solr in my day job like
> this….
>
> $ bin/solr create -c my_collection
> $ bin/post -c my_collection /data/docs.json
>
> and http://… /select?q=…&wt=csv…
>
> So “it works for me”, but that’s not a nice way to approach the struggles
> of users. Though we’ve come a long way, we’ve got a ways to go as well.
>
> Erik
>
> p.s. -
>
> > Never mind the fact that the XML-based configuration process is an
> antiquated nightmare when the rest of the world has long since moved onto
> databases.
>
> Well, to that point - the world that I work in really boils down to at
> least plain text (alas, mostly JSON these days, but even that’s an
> implementation detail) stuffed into git repositories, and played into new
> Solr environments by uploading configuration files, or more modernly,
> hitting the Solr configuration API’s to add/configure fields, set up
> request handlers, and the basics of what needs to be done. No XML needed
> these days. No (relational, JDBC) databases either, for that matter :)
>
> > Maybe this will help someone else out there.
>
> Thanks for taking the time to detail your struggles to the community. It
> is helpful to see where the rough edges are in this whole business, and
> smoothing them out. But it’s no easy business, having these stacks of
> dependencies and complexities piled on top of one another and trying to get
> it all fired up properly and usably.
>
> Erik
>
>
Re: Miserable Experience ..... Again.
Posted by Bradley Belyeu <br...@life.church>.
I agree with what Erik wrote and some of Aaron’s original post. I’m relatively new to the Solr system (yes, pun intended) having just started diving into it a little over a year ago. I was in the “envious” position of being the only person who wanted to learn it and support it after our previous “expert” left the team. :D
It recently took me a full two week sprint cycle to upgrade our old style master-slaves cluster from 4.3 to 5.1 (we simultaneously went from java 1.7 to 1.8 and updated our NewRelic wrapper). There were many pain points along the way especially since I’m not much of a Java developer (I’m a Python/PHP/JS guy). BUT that being said I learned such an incredible amount over that sprint about the JVM, Solr configs & classes, caching & index tuning, and Lucene itself that I wouldn’t have changed it a bit. Ok, that’s a lie, I would have fixed a problem with NGrams before it caused a couple hour production outage. But basically, the amount of knowledge gained was very well worth the time and effort put into working through the upgrade pain points.
@Aaron another thing I’ve learned over the years is to only change 1 thing at a time, and definitely don’t mix troubleshooting with upgrading.
https://en.wikibooks.org/wiki/Computer_Programming_Principles/Maintaining/Debugging#Change_one_thing_at_a_time
One thing I don’t understand from your email is when you said you were “wondering why everything has to be so awful” about Solr. The initial problems you described were related to a MySQL database change and issues with Zookeeper. I think choosing to do the Java updates and Solr updates are what led to the other issues. The Solr docs packaged with the release do a fairly good job of explaining the breaking changes with each version. I chose to do all my updates one minor version at a time so I could keep up with the changes in the docs (4.3->4.4->…4.10->5.0->5.1) which took longer but made troubleshooting much easier.
As far as the user interface goes, at least it has one unlike some other popular search tools (cough ElasticSearch cough). And as far as XML based configuration is concerned, I personally prefer it to scripts with JSON blobs and Rest calls to set up your collections/entities/docs/etc. But that’s just b/c I’m weird.
I do agree that the error messages are often not helpful, BUT I found it easiest to just look at the source code to find the root cause of the issue. Which is what I think this boils down to, is that rather than complain, we should work to make Solr what we want it to be. If you can identify an issue, then you can either solve it yourself with a pull request or create a Jira issue asking for help to get it fixed.
Here’s a quote I love from Maya Angelou, “What you're supposed to do when you don't like a thing is change it. If you can't change it, change the way you think about it. Don't complain.”
On 9/12/16, 6:05 PM, "Erik Hatcher" <er...@gmail.com> wrote:
Aaron - I for one sympathize. When I pause to think of the stacks upon stacks of technologies that something like Solr are built upon… my head spins and I feel for the folks coming to computer science these days and having the whole Java and Big Data stacks and all that goes along with that (JVM/mem/GC up to network topology and architecture with 3xZK, plus NxM Solr’s, and beyond to data modeling, schema design, and query parameter adjusting).
---
It’s good for us to hear the ugly/painful side of folks experiences. It’s driven us to to where I find myself iterating with Solr in my day job like this….
$ bin/solr create -c my_collection
$ bin/post -c my_collection /data/docs.json
and http://… /select?q=…&wt=csv…
So “it works for me”, but that’s not a nice way to approach the struggles of users. Though we’ve come a long way, we’ve got a ways to go as well.
Erik
p.s. -
> Never mind the fact that the XML-based configuration process is an antiquated nightmare when the rest of the world has long since moved onto databases.
Well, to that point - the world that I work in really boils down to at least plain text (alas, mostly JSON these days, but even that’s an implementation detail) stuffed into git repositories, and played into new Solr environments by uploading configuration files, or more modernly, hitting the Solr configuration API’s to add/configure fields, set up request handlers, and the basics of what needs to be done. No XML needed these days. No (relational, JDBC) databases either, for that matter :)
> Maybe this will help someone else out there.
Thanks for taking the time to detail your struggles to the community. It is helpful to see where the rough edges are in this whole business, and smoothing them out. But it’s no easy business, having these stacks of dependencies and complexities piled on top of one another and trying to get it all fired up properly and usably.
Erik
Re: Miserable Experience ..... Again.
Posted by Erik Hatcher <er...@gmail.com>.
Aaron - I for one sympathize. When I pause to think of the stacks upon stacks of technologies that something like Solr are built upon… my head spins and I feel for the folks coming to computer science these days and having the whole Java and Big Data stacks and all that goes along with that (JVM/mem/GC up to network topology and architecture with 3xZK, plus NxM Solr’s, and beyond to data modeling, schema design, and query parameter adjusting).
---
It’s good for us to hear the ugly/painful side of folks experiences. It’s driven us to to where I find myself iterating with Solr in my day job like this….
$ bin/solr create -c my_collection
$ bin/post -c my_collection /data/docs.json
and http://… /select?q=…&wt=csv…
So “it works for me”, but that’s not a nice way to approach the struggles of users. Though we’ve come a long way, we’ve got a ways to go as well.
Erik
p.s. -
> Never mind the fact that the XML-based configuration process is an antiquated nightmare when the rest of the world has long since moved onto databases.
Well, to that point - the world that I work in really boils down to at least plain text (alas, mostly JSON these days, but even that’s an implementation detail) stuffed into git repositories, and played into new Solr environments by uploading configuration files, or more modernly, hitting the Solr configuration API’s to add/configure fields, set up request handlers, and the basics of what needs to be done. No XML needed these days. No (relational, JDBC) databases either, for that matter :)
> Maybe this will help someone else out there.
Thanks for taking the time to detail your struggles to the community. It is helpful to see where the rough edges are in this whole business, and smoothing them out. But it’s no easy business, having these stacks of dependencies and complexities piled on top of one another and trying to get it all fired up properly and usably.
Erik
Re: Miserable Experience Using Solr. Again.
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
On 13 September 2016 at 16:46, Alessandro Benedetti
<ab...@apache.org> wrote:
>> It didn’t say which field type. Buried in the logs I found a reference in
>> the Java stack trace—which *disappears* (and distorts the viewing window
>> horribly) after a few seconds when you try to view it in the web log UI—to
>> the string "units="degrees"".
>>
>
> This si a bug, and it is really annoying, not sure anyone already raised
> it, if not I suggest you to do that :)
I don't remember seeing it opened. So this could be a great practice
for somebody who is good at finding bugs and issues. :-)
I love a good rant. I used to produce those myself. I love even more a
good rant that includes specific granular improvements others can get
behind. The bug report as suggested above would be a great one example
of such a granular thing.
I would also point out that most of the contributors to Lucene/Solr
open source are able to contribute because somebody _pays_ them to
develop something on top of/with those projects and they hit
limitations they cannot solve in other easier ways. Those _usually_
are the cutting edge features such as CDCR, new performance
improvements, etc. We could always do with _more_ people who will
focus on the more user-oriented features or on making those new
cutting edge features more easily accessible.
Regards,
Alex.
P.s. And if anybody wants to rant and will be at Lucene/Solr
revolution, I will be more than happy to sit and listen to you during
any of the food breaks. And I'll help figuring out what those granular
improvement suggestions could be. Feel free to reach out directly if
you want to have a rant scheduled too, instead of catching me
organically :-)
----
Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/
Re: Miserable Experience Using Solr. Again.
Posted by Yago Riveiro <ya...@gmail.com>.
I stuck in 5.3.1 because if upgrade to 5.5 or 6.x my cluster dies.
Doing a rolling upgrade, when I upgrade the second node to 5.5 both die in the
per-sync phase, I don't know what changes in 5.5 but it's demanding a huge
quantity of memory to check if the replica it's in sync.
This kind of stuff and the full re-index (12T) between major releases are
indeed a pain.
Cryptical errors and a deficient system to get metrics from what it's going on
inside the cluster is another issue, I'm unable to get the throughput in a
collection as a whole, the number of http connection in each node, the
utilization of the jetty thread pool and stuff like that.
Solr is a great tool, but it's hard, too hard to get in.
\--
/Yago Riveiro
On Sep 13 2016, at 10:46 am, Alessandro Benedetti <ab...@apache.org>
wrote:
> First of all I second Bram, I am sorry you had a bad experience with Solr,
but I think that:
\- without a minimum study and documentation
\- without trying to follow the best practices
I think you are going to have a "miserable" experience with any software,
don't you ?
>
> In addition to Bram :
>
> On Mon, Sep 12, 2016 at 10:48 PM, Aaron Greenspan <
aaron.greenspan@plainsite.org> wrote:
>
> It didn’t say which field type. Buried in the logs I found a reference in
> the Java stack trace—which *disappears* (and distorts the viewing window
> horribly) after a few seconds when you try to view it in the web log UI—to
> the string "units="degrees"".
>
>
> This si a bug, and it is really annoying, not sure anyone already raised
it, if not I suggest you to do that :)
But you can use the logs themselves without any problem.
>
> >
> Apparently there is some aspect of the Thai text field type that Solr
> 6.2.0 doesn’t like. So I disabled it. I don’t use Thai text.
>
>
> If you were not using the Thai text, why had you the Thai Text field type
defined ?
Keep It Simple Stupid is the way :)
I find tons of Solr instances in production mith monster solrconfig.xml and
schema.xml. basically the old default ones, without any particular reason.
Don't do that !
>
> >
> Now Solr was complaining about "Error loading class
> 'solr.admin.AdminHandlers'". So I found the reference to
> solr.admin.AdminHandlers in solrconfig.xml for each of my cores and
> commented it out. Only then did Solr work again.
>
>
> Seems to be you didn't take care of reading the update release notes, did
you ?
>
>
Cheers
\--
\--------------------------
>
> Benedetti Alessandro
Visiting card : [http://about.me/alessandro_benedetti](http://about.me/alessan
dro_benedetti&r=c29sci11c2VyQGx1Y2VuZS5hcGFjaGUub3Jn)
>
> "Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
Re: Miserable Experience Using Solr. Again.
Posted by Alessandro Benedetti <ab...@apache.org>.
First of all I second Bram, I am sorry you had a bad experience with Solr,
but I think that:
- without a minimum study and documentation
- without trying to follow the best practices
I think you are going to have a "miserable" experience with any software,
don't you ?
In addition to Bram :
On Mon, Sep 12, 2016 at 10:48 PM, Aaron Greenspan <
aaron.greenspan@plainsite.org> wrote:
>
> It didn’t say which field type. Buried in the logs I found a reference in
> the Java stack trace—which *disappears* (and distorts the viewing window
> horribly) after a few seconds when you try to view it in the web log UI—to
> the string "units="degrees"".
>
This si a bug, and it is really annoying, not sure anyone already raised
it, if not I suggest you to do that :)
But you can use the logs themselves without any problem.
>
> Apparently there is some aspect of the Thai text field type that Solr
> 6.2.0 doesn’t like. So I disabled it. I don’t use Thai text.
>
If you were not using the Thai text, why had you the Thai Text field type
defined ?
Keep It Simple Stupid is the way :)
I find tons of Solr instances in production mith monster solrconfig.xml and
schema.xml. basically the old default ones, without any particular reason.
Don't do that !
>
> Now Solr was complaining about "Error loading class
> 'solr.admin.AdminHandlers'". So I found the reference to
> solr.admin.AdminHandlers in solrconfig.xml for each of my cores and
> commented it out. Only then did Solr work again.
>
Seems to be you didn't take care of reading the update release notes, did
you ?
Cheers
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Miserable Experience Using Solr. Again.
Posted by Shawn Heisey <ap...@elyograg.org>.
On 9/12/2016 3:48 PM, Aaron Greenspan wrote:
> I have been on this list for some time because I know that any time I
> try to do anything related to Solr I\u2019m going to have to spend hours on
> it, wondering why everything has to be so awful, and I just want
> somewhere to provide feedback with the dim hope that the product might
> improve one day. (So far, for my purposes, it hasn\u2019t.) Sure enough, I
> still absolutely hate using Solr, and I have more feedback.
First, let me thank you for mentioning your experiences. It's harsh
feedback, but I still welcome it. I'm going to say some things that may
sound to you like excuses ... and you aren't really wrong to think that,
but we *do* take your comments seriously, and in many cases, we already
know that there are problems we need to solve.
As others have said, and as I'm sure you probably know, open source is
created by people trying to solve a problem for themselves, and is done
on a volunteer basis. If a project is lucky, it attracts interested
volunteers and truly magical things happen for the project users. I
think Solr is a good project, with an awesome community.
Beginner documentation is one of the hardest things to write. It's
difficult for people who live and breathe the software to view the
system from the perspective of someone who has never touched it at all,
and to write something that explains to that novice exactly how to make
it work.
> I started with a confusing error on the web console, which I still
> can\u2019t figure out how to password protect without going through an
> insanely process involving "ZooKeeper," which I don\u2019t know anything
> about, or have, to the best of my knowledge: Problem accessing /solr/.
> Reason: Forbidden
This particular part of your message involves an old fight in the Solr
project: Security. Those of us who have been in the industry forever
have learned that many of the security features that people expect for
Internet-facing services are not at all helpful for the security of
internal systems like Solr. The best thing you can do to prevent
problems is to place Solr in a location where it cannot be reached by
people who cannot be trusted. If you can trust those who have access,
then there's no need for intrinsic security features.
Any security that you layer on top of Solr (encryption, authentication,
etc) is useless if somebody compromises the system that talks to Solr,
which already has all the keys/passwords/etc required to get right in.
As evidenced by the fact that authentication has come to Solr, we *are*
listening to our users that demand security features. The
authentication feature that you are trying to use, which involves basic
username/password authentication of the API calls that the admin UI
makes (*not* the admin UI itself), was originally developed for
SolrCloud -- which utilizes Zookeeper as a central configuration
database. Work is underway right now to bring basic authentication to
standalone Solr, but it is not going to be available until at least 6.3,
and may take even longer to finish. It will also require separate
configuration on every host, which is not required for SolrCloud.
> According to logs, this apparently meant that a MySQL query had failed
> due to a field name change. Since I would have to change my XML
> configuration files, I decided to use the opportunity to upgrade from
> Solr 5.1.4 to 6.2.0. It broke everything.
For almost ANY software, but especially for an open source package,
upgrading to a new major version is a good way to cause problems, not
usually a good way to solve them. Solr does maintain compatibility with
configs that are completely current for the later versions in the
previous major release ... but there are a LOT of configs out in the
world (even in the latest versions of their software!) that were
originally designed for Solr 3.x, 4.x, or *early* 5.x. Solr makes zero
guarantees about configs designed for software that old.
For an example of a similar situation in a different software package:
If you try to copy configs for the Apache webserver (httpd) from a 2.2
install to a 2.4 install, chances are excellent that you're going to
have to change those configs before Apache will even start, much less
operate as expected. Upgrading Apache from 2.2 to 2.4 is technically a
"minor" version upgrade, but in terms of capability and configuration,
is similar to a major Solr version upgrade.
> First I was getting errors about "Unsupported major.minor version
> 52.0", so I needed to install the Linux x64 JRE 1.8.0, which I managed
> on CentOS 6 with... yum install openjdk-1.8.0 ...going to Oracle\u2019s web
> site, downloading the latest JRE 1.8 build, and then running... yum
> localinstall jre-8u101-linux-x64.rpm So far so good. But I didn\u2019t have
> JAVA_HOME set properly apparently, so I needed to do the
> not-exactly-intuitive\u2026 export
> JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.el6_8.x86_64/jre/
Others have already covered this topic. Managing Java installations is
required before Solr will work, and there are so many ways to handle it
that Solr cannot cover the topic in its documentation.
> As usual, I manually moved over my mysql-connector-java-5.1.38-bin.jar
> file from the dist/ folder in the old version to the new one. Then
> after stopping the old process (with kill -9, since there seems to be
> no graceful way to shut down Solr\u2014README.txt doesn\u2019t mention bin/solr
> stop) I moved over my two core folders from the old server/solr/
> folder. I tried to start it up with bin/solr start, and watched the
> errors roll in.
A standard "kill" would have stopped Solr gracefully, with no need for
the -9 signal. It can take a while to happen, though. Using "kill -9"
can cause large problems on restart that are difficult to track down.
If you change the the extracted download or the installed program root
(normally /opt/solr on Linux), and type "bin/solr" without options, it
gives you enough information to track down the stop command. If you do
use the installer script for Linux with default options, you can
typically type "service solr stop" because the installer handles an init
script.
> There was some kind of problem with StopFilterFactory and the
> text_general field type. Thanks to Stack Overflow I was able to
> determine that the apparent problem was that there was a parameter,
> previously fine, which was no longer fine. So I removed all instances
> of enablePositionIncrements="true". That helped, but then I ran into a
> broader error: "Plugin Initializing failure for [schema.xml]
> fieldType". It didn\u2019t say which field type. Buried in the logs I found
> a reference in the Java stack trace\u2014which *disappears* (and distorts
> the viewing window horribly) after a few seconds when you try to view
> it in the web log UI\u2014to the string "units="degrees"". Sure enough,
> this string appeared in my schema.xml for a class called
> "solr.SpatialRecursivePrefixTreeFieldType" that I\u2019m pretty sure I
> never use. I removed that parameter, and moved on to the next set of
> errors.
The actual logfile (solr.log) is generally MUCH more useful for tracking
down problems than the Logging tab in the admin UI. This is another
example of something that is not immediately obvious to a novice.
> Apparently there is some aspect of the Thai text field type that Solr
> 6.2.0 doesn\u2019t like. So I disabled it. I don\u2019t use Thai text. Now Solr
> was complaining about "Error loading class
> 'solr.admin.AdminHandlers'". So I found the reference to
> solr.admin.AdminHandlers in solrconfig.xml for each of my cores and
> commented it out. Only then did Solr work again. This was not a smooth
> process. It took about two hours. The user interface is still as buggy
> as an early alpha of most products, the errors are difficult to
> understand when they don\u2019t actually specify what\u2019s wrong (and they
> almost never do), and there should have been an automatic process to
> highlight and fix problems in old (pre-6) configuration files. Never
> mind the fact that the XML-based configuration process is an
> antiquated nightmare when the rest of the world has long since moved
> onto databases.
I've already mentioned how configs designed for very old versions won't
work. It sounds like your configs were designed for 4.x, which means
they will usually work in 5.x, but won't work if you go higher.
Zookeeper is a configuration database. If it is properly set up,
SolrCloud uses zookeeper to build a true cluster of nodes with no single
point of failure and very little config stored on individual servers.
It's true that the config files that you add to the zookeeper database
are still in XML format. XML is a well-defined standard that is
particularly good at holding structured configuration information.
Switching to another format like JSON is possible, but that has a strong
potential to cause *additional* upgrade headaches, and I think you've
already encountered enough of those!
Solr's documentation is not where we want it to be. Detailed
step-by-step instructions for novices (and tools to match) is the
biggest omission. It would also be useful if we had more automated
tools for upgrading. Error messages are another area for improvement.
Writing all these things is not easy. Good documentation is HARD.
Writing tools that work right takes time. It's a struggle to provide
error messages that are helpful for *developers* ... error messages that
are helpful for *users* are even harder.
The change in Solr 5.0 where we took control of the servlet container
away from the user was one of the first steps in making the end-user
experience more consistent and easier to document. It was only an
initial step, though. There is lots of work to do. If you're able to
help, we would appreciate it.
https://wiki.apache.org/solr/WhyNoWar
Thanks,
Shawn
Re: Miserable Experience Using Solr. Again.
Posted by Bram Van Dam <br...@intix.eu>.
I'm sorry you're having a "miserable" experience "again". That's
certainly not my experience with Solr. That being said:
> First I was getting errors about "Unsupported major.minor version 52.0", so I needed to install the Linux x64 JRE 1.8.0, which I managed on CentOS 6 with...
> yum install openjdk-1.8.0
This is not a Solr problem. Solr requires Java 8. Java 7 has been
officially end-of-lifed since april 2015. This means no more patches, no
more performance improvements and no more security updates (unless
you're paying Oracle). This is clearly stated in the (very decent) Solr
documentation. To use your own words: Java 7 is an antiquated nightmare
and the rest of the world has moved on to Java 8.
> So far so good. But I didn\u2019t have JAVA_HOME set properly apparently, so I needed to do the not-exactly-intuitive\u2026
> export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.el6_8.x86_64/jre/
You don't need to set JAVA_HOME to run Solr. But if you do have a
JAVA_HOME environment variable, and it points to a wrong Java version,
you're going to have a bad time.
> Then after stopping the old process (with kill -9, since there seems to be no graceful way to shut down Solr)
There's a stop command, which is documented. It's a non-surprising
location and has a non-surprising name. And even if there wasn't, "kill"
would have sufficed.
> There was some kind of problem with StopFilterFactory and the text_general field type. Thanks to Stack Overflow I was able to determine that the apparent problem was that there was a parameter, previously fine, which was no longer fine. So I removed all instances of enablePositionIncrements="true". That helped, but then I ran into a broader error: "Plugin Initializing failure for [schema.xml] fieldType". It didn\u2019t say which field type. Buried in the logs I found a reference in the Java stack trace\u2014which *disappears* (and distorts the viewing window horribly) after a few seconds when you try to view it in the web log UI\u2014to the string "units="degrees"". Sure enough, this string appeared in my schema.xml for a class called "solr.SpatialRecursivePrefixTreeFieldType" that I\u2019m pretty sure I never use. I removed that parameter, and moved on to the next set of errors.
Releases come with release notes and -- when required -- upgrade
instructions and admonitions. It's certainly possible that there's been
an oversight here or there and you're more than welcome to point those out.
> The user interface is still as buggy as an early alpha of most
products, the errors are difficult to understand when they don\u2019t
actually specify what\u2019s wrong (and they almost never do), and there
should have been an automatic process to highlight and fix problems in
old (pre-6) configuration files.
What user interface? Are you talking about the Admin UI? That's a
convenience feature which helps you manage Solr. It makes life a lot
easier, even if it's not perfect. The logs are generally quite good at
explaining what's wrong.
> Never mind the fact that the XML-based configuration process is an
antiquated nightmare when the rest of the world has long since moved
onto databases.
An antiquated nightmare? The rest of the world? How would this work?
What benefit would it possibly have?
You're more than welcome to report any bugs you find
(https://issues.apache.org/jira/browse/SOLR). But I feel like general
ranting on the mailing list isn't very productive. Well, I suppose
venting feels good, so there's that.
Things that would be more productive:
1. Reading the documentation.
2. Taking a basic system administration class or two.
3. Pointing out -- or contributing to -- parts of the documentation that
aren't up to par. Either on the mailing list, or on Jira. Preferably in
a constructive way instead of a "miserable experience"-way.
I feel like you're missing the part where most open source development,
documentation, release management etc is done by volunteers. Volunteers
who tend to scratch their own itch first, and are then kind enough to
donate the fruit of their labour to the rest of the world. You can
certainly make requests, and you can certainly hope for things to improve.
If you're having a "miserable" time "again", then you can always hire a
Solr consultant to do the work for you. You can't demand free stuff to
scratch your every itch. You can either invest your time and figure out
how to do things yourself, or your money and have things done for you.
But there's no such thing as a free lunch.
- Bram
Re: Miserable Experience Using Solr. Again.
Posted by John Bickerstaff <jo...@johnbickerstaff.com>.
I would also add that dealing with Java versions has always been a pain
until you get used to the whole "JAVA HOME" thing, but that this isn't
anything to do with SOLR per se - it's just part and parcel of dealing with
open source software that uses Java...
Big changes between major versions of any software are common - there is
some documentation here that may help...
https://cwiki.apache.org/confluence/display/solr/Major+Changes+from+Solr+5+to+Solr+6
It was reading this that made me decide to try "upgrade by replace" to
avoid the whole "update" issue entirely - although I also had to upgrade
Java on my VMs...
On Mon, Sep 12, 2016 at 4:05 PM, John Bickerstaff <jo...@johnbickerstaff.com>
wrote:
> For what it's worth - I found enough frustration upgrading that I decided
> to "upgrade by replacement"
>
> Now, I suppose if you've got a huge dataset to re-index that could be a
> problem, but just in case an option like that helps you, I'll suggest this.
>
> 1. Install 6.x on a new machine using the "install for production"
> instructions
> 2. Use the configs from one of the sample projects to create an
> appropriately-named collection
> 3. Use the ability to "include" your configs into the other configs (they
> live in separate files)
> I can provide more help here if you're interested
> 4. Re-index all your data into the new version of SOLR...
>
> I have rough, but useable docs on this if you are interested in attempting
> this approach.
>
> On Mon, Sep 12, 2016 at 3:48 PM, Aaron Greenspan <
> aaron.greenspan@plainsite.org> wrote:
>
>> Hi,
>>
>> I have been on this list for some time because I know that any time I try
>> to do anything related to Solr I’m going to have to spend hours on it,
>> wondering why everything has to be so awful, and I just want somewhere to
>> provide feedback with the dim hope that the product might improve one day.
>> (So far, for my purposes, it hasn’t.) Sure enough, I still absolutely hate
>> using Solr, and I have more feedback.
>>
>> I started with a confusing error on the web console, which I still can’t
>> figure out how to password protect without going through an insanely
>> process involving "ZooKeeper," which I don’t know anything about, or have,
>> to the best of my knowledge:
>>
>> Problem accessing /solr/. Reason:
>>
>> Forbidden
>>
>> According to logs, this apparently meant that a MySQL query had failed
>> due to a field name change. Since I would have to change my XML
>> configuration files, I decided to use the opportunity to upgrade from Solr
>> 5.1.4 to 6.2.0. It broke everything.
>>
>> First I was getting errors about "Unsupported major.minor version 52.0",
>> so I needed to install the Linux x64 JRE 1.8.0, which I managed on CentOS 6
>> with...
>>
>> yum install openjdk-1.8.0
>>
>> ...going to Oracle’s web site, downloading the latest JRE 1.8 build, and
>> then running...
>>
>> yum localinstall jre-8u101-linux-x64.rpm
>>
>> So far so good. But I didn’t have JAVA_HOME set properly apparently, so I
>> needed to do the not-exactly-intuitive…
>>
>> export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.el
>> 6_8.x86_64/jre/
>>
>> As usual, I manually moved over my mysql-connector-java-5.1.38-bin.jar
>> file from the dist/ folder in the old version to the new one. Then after
>> stopping the old process (with kill -9, since there seems to be no graceful
>> way to shut down Solr—README.txt doesn’t mention bin/solr stop) I moved
>> over my two core folders from the old server/solr/ folder. I tried to start
>> it up with bin/solr start, and watched the errors roll in.
>>
>> There was some kind of problem with StopFilterFactory and the
>> text_general field type. Thanks to Stack Overflow I was able to determine
>> that the apparent problem was that there was a parameter, previously fine,
>> which was no longer fine. So I removed all instances of
>> enablePositionIncrements="true". That helped, but then I ran into a
>> broader error: "Plugin Initializing failure for [schema.xml] fieldType". It
>> didn’t say which field type. Buried in the logs I found a reference in the
>> Java stack trace—which *disappears* (and distorts the viewing window
>> horribly) after a few seconds when you try to view it in the web log UI—to
>> the string "units="degrees"". Sure enough, this string appeared in my
>> schema.xml for a class called "solr.SpatialRecursivePrefixTreeFieldType"
>> that I’m pretty sure I never use. I removed that parameter, and moved on to
>> the next set of errors.
>>
>> Apparently there is some aspect of the Thai text field type that Solr
>> 6.2.0 doesn’t like. So I disabled it. I don’t use Thai text.
>>
>> Now Solr was complaining about "Error loading class
>> 'solr.admin.AdminHandlers'". So I found the reference to
>> solr.admin.AdminHandlers in solrconfig.xml for each of my cores and
>> commented it out. Only then did Solr work again.
>>
>> This was not a smooth process. It took about two hours. The user
>> interface is still as buggy as an early alpha of most products, the errors
>> are difficult to understand when they don’t actually specify what’s wrong
>> (and they almost never do), and there should have been an automatic process
>> to highlight and fix problems in old (pre-6) configuration files. Never
>> mind the fact that the XML-based configuration process is an antiquated
>> nightmare when the rest of the world has long since moved onto databases.
>>
>> Maybe this will help someone else out there.
>>
>> Aaron
>>
>> PlainSite | http://www.plainsite.org
>
>
>
Re: Miserable Experience Using Solr. Again.
Posted by John Bickerstaff <jo...@johnbickerstaff.com>.
Sure - ping me off the list and I'll send my text file docs.
They're rough and (of course) focused on what I'm doing, but they just
might relieve some of the pain.
Caveat - all on Linux and command line - no Admin UI api's -- I like the
feel of the command line so I use it.
On Mon, Sep 12, 2016 at 8:41 PM, <bi...@gmail.com> wrote:
> Interested for sure
>
> Bill Bell
> Sent from mobile
>
>
> > On Sep 12, 2016, at 4:05 PM, John Bickerstaff <jo...@johnbickerstaff.com>
> wrote:
> >
> > For what it's worth - I found enough frustration upgrading that I decided
> > to "upgrade by replacement"
> >
> > Now, I suppose if you've got a huge dataset to re-index that could be a
> > problem, but just in case an option like that helps you, I'll suggest
> this.
> >
> > 1. Install 6.x on a new machine using the "install for production"
> > instructions
> > 2. Use the configs from one of the sample projects to create an
> > appropriately-named collection
> > 3. Use the ability to "include" your configs into the other configs (they
> > live in separate files)
> > I can provide more help here if you're interested
> > 4. Re-index all your data into the new version of SOLR...
> >
> > I have rough, but useable docs on this if you are interested in
> attempting
> > this approach.
> >
> > On Mon, Sep 12, 2016 at 3:48 PM, Aaron Greenspan <
> > aaron.greenspan@plainsite.org> wrote:
> >
> >> Hi,
> >>
> >> I have been on this list for some time because I know that any time I
> try
> >> to do anything related to Solr I’m going to have to spend hours on it,
> >> wondering why everything has to be so awful, and I just want somewhere
> to
> >> provide feedback with the dim hope that the product might improve one
> day.
> >> (So far, for my purposes, it hasn’t.) Sure enough, I still absolutely
> hate
> >> using Solr, and I have more feedback.
> >>
> >> I started with a confusing error on the web console, which I still can’t
> >> figure out how to password protect without going through an insanely
> >> process involving "ZooKeeper," which I don’t know anything about, or
> have,
> >> to the best of my knowledge:
> >>
> >> Problem accessing /solr/. Reason:
> >>
> >> Forbidden
> >>
> >> According to logs, this apparently meant that a MySQL query had failed
> due
> >> to a field name change. Since I would have to change my XML
> configuration
> >> files, I decided to use the opportunity to upgrade from Solr 5.1.4 to
> >> 6.2.0. It broke everything.
> >>
> >> First I was getting errors about "Unsupported major.minor version 52.0",
> >> so I needed to install the Linux x64 JRE 1.8.0, which I managed on
> CentOS 6
> >> with...
> >>
> >> yum install openjdk-1.8.0
> >>
> >> ...going to Oracle’s web site, downloading the latest JRE 1.8 build, and
> >> then running...
> >>
> >> yum localinstall jre-8u101-linux-x64.rpm
> >>
> >> So far so good. But I didn’t have JAVA_HOME set properly apparently, so
> I
> >> needed to do the not-exactly-intuitive…
> >>
> >> export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.
> >> el6_8.x86_64/jre/
> >>
> >> As usual, I manually moved over my mysql-connector-java-5.1.38-bin.jar
> >> file from the dist/ folder in the old version to the new one. Then after
> >> stopping the old process (with kill -9, since there seems to be no
> graceful
> >> way to shut down Solr—README.txt doesn’t mention bin/solr stop) I moved
> >> over my two core folders from the old server/solr/ folder. I tried to
> start
> >> it up with bin/solr start, and watched the errors roll in.
> >>
> >> There was some kind of problem with StopFilterFactory and the
> text_general
> >> field type. Thanks to Stack Overflow I was able to determine that the
> >> apparent problem was that there was a parameter, previously fine, which
> was
> >> no longer fine. So I removed all instances of enablePositionIncrements="
> true".
> >> That helped, but then I ran into a broader error: "Plugin Initializing
> >> failure for [schema.xml] fieldType". It didn’t say which field type.
> Buried
> >> in the logs I found a reference in the Java stack trace—which
> *disappears*
> >> (and distorts the viewing window horribly) after a few seconds when you
> try
> >> to view it in the web log UI—to the string "units="degrees"". Sure
> enough,
> >> this string appeared in my schema.xml for a class called "solr.
> >> SpatialRecursivePrefixTreeFieldType" that I’m pretty sure I never use.
> I
> >> removed that parameter, and moved on to the next set of errors.
> >>
> >> Apparently there is some aspect of the Thai text field type that Solr
> >> 6.2.0 doesn’t like. So I disabled it. I don’t use Thai text.
> >>
> >> Now Solr was complaining about "Error loading class
> >> 'solr.admin.AdminHandlers'". So I found the reference to
> >> solr.admin.AdminHandlers in solrconfig.xml for each of my cores and
> >> commented it out. Only then did Solr work again.
> >>
> >> This was not a smooth process. It took about two hours. The user
> interface
> >> is still as buggy as an early alpha of most products, the errors are
> >> difficult to understand when they don’t actually specify what’s wrong
> (and
> >> they almost never do), and there should have been an automatic process
> to
> >> highlight and fix problems in old (pre-6) configuration files. Never
> mind
> >> the fact that the XML-based configuration process is an antiquated
> >> nightmare when the rest of the world has long since moved onto
> databases.
> >>
> >> Maybe this will help someone else out there.
> >>
> >> Aaron
> >>
> >> PlainSite | http://www.plainsite.org
>
Re: Miserable Experience Using Solr. Again.
Posted by bi...@gmail.com.
Interested for sure
Bill Bell
Sent from mobile
> On Sep 12, 2016, at 4:05 PM, John Bickerstaff <jo...@johnbickerstaff.com> wrote:
>
> For what it's worth - I found enough frustration upgrading that I decided
> to "upgrade by replacement"
>
> Now, I suppose if you've got a huge dataset to re-index that could be a
> problem, but just in case an option like that helps you, I'll suggest this.
>
> 1. Install 6.x on a new machine using the "install for production"
> instructions
> 2. Use the configs from one of the sample projects to create an
> appropriately-named collection
> 3. Use the ability to "include" your configs into the other configs (they
> live in separate files)
> I can provide more help here if you're interested
> 4. Re-index all your data into the new version of SOLR...
>
> I have rough, but useable docs on this if you are interested in attempting
> this approach.
>
> On Mon, Sep 12, 2016 at 3:48 PM, Aaron Greenspan <
> aaron.greenspan@plainsite.org> wrote:
>
>> Hi,
>>
>> I have been on this list for some time because I know that any time I try
>> to do anything related to Solr I’m going to have to spend hours on it,
>> wondering why everything has to be so awful, and I just want somewhere to
>> provide feedback with the dim hope that the product might improve one day.
>> (So far, for my purposes, it hasn’t.) Sure enough, I still absolutely hate
>> using Solr, and I have more feedback.
>>
>> I started with a confusing error on the web console, which I still can’t
>> figure out how to password protect without going through an insanely
>> process involving "ZooKeeper," which I don’t know anything about, or have,
>> to the best of my knowledge:
>>
>> Problem accessing /solr/. Reason:
>>
>> Forbidden
>>
>> According to logs, this apparently meant that a MySQL query had failed due
>> to a field name change. Since I would have to change my XML configuration
>> files, I decided to use the opportunity to upgrade from Solr 5.1.4 to
>> 6.2.0. It broke everything.
>>
>> First I was getting errors about "Unsupported major.minor version 52.0",
>> so I needed to install the Linux x64 JRE 1.8.0, which I managed on CentOS 6
>> with...
>>
>> yum install openjdk-1.8.0
>>
>> ...going to Oracle’s web site, downloading the latest JRE 1.8 build, and
>> then running...
>>
>> yum localinstall jre-8u101-linux-x64.rpm
>>
>> So far so good. But I didn’t have JAVA_HOME set properly apparently, so I
>> needed to do the not-exactly-intuitive…
>>
>> export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.
>> el6_8.x86_64/jre/
>>
>> As usual, I manually moved over my mysql-connector-java-5.1.38-bin.jar
>> file from the dist/ folder in the old version to the new one. Then after
>> stopping the old process (with kill -9, since there seems to be no graceful
>> way to shut down Solr—README.txt doesn’t mention bin/solr stop) I moved
>> over my two core folders from the old server/solr/ folder. I tried to start
>> it up with bin/solr start, and watched the errors roll in.
>>
>> There was some kind of problem with StopFilterFactory and the text_general
>> field type. Thanks to Stack Overflow I was able to determine that the
>> apparent problem was that there was a parameter, previously fine, which was
>> no longer fine. So I removed all instances of enablePositionIncrements="true".
>> That helped, but then I ran into a broader error: "Plugin Initializing
>> failure for [schema.xml] fieldType". It didn’t say which field type. Buried
>> in the logs I found a reference in the Java stack trace—which *disappears*
>> (and distorts the viewing window horribly) after a few seconds when you try
>> to view it in the web log UI—to the string "units="degrees"". Sure enough,
>> this string appeared in my schema.xml for a class called "solr.
>> SpatialRecursivePrefixTreeFieldType" that I’m pretty sure I never use. I
>> removed that parameter, and moved on to the next set of errors.
>>
>> Apparently there is some aspect of the Thai text field type that Solr
>> 6.2.0 doesn’t like. So I disabled it. I don’t use Thai text.
>>
>> Now Solr was complaining about "Error loading class
>> 'solr.admin.AdminHandlers'". So I found the reference to
>> solr.admin.AdminHandlers in solrconfig.xml for each of my cores and
>> commented it out. Only then did Solr work again.
>>
>> This was not a smooth process. It took about two hours. The user interface
>> is still as buggy as an early alpha of most products, the errors are
>> difficult to understand when they don’t actually specify what’s wrong (and
>> they almost never do), and there should have been an automatic process to
>> highlight and fix problems in old (pre-6) configuration files. Never mind
>> the fact that the XML-based configuration process is an antiquated
>> nightmare when the rest of the world has long since moved onto databases.
>>
>> Maybe this will help someone else out there.
>>
>> Aaron
>>
>> PlainSite | http://www.plainsite.org
Re: Miserable Experience Using Solr. Again.
Posted by John Bickerstaff <jo...@johnbickerstaff.com>.
For what it's worth - I found enough frustration upgrading that I decided
to "upgrade by replacement"
Now, I suppose if you've got a huge dataset to re-index that could be a
problem, but just in case an option like that helps you, I'll suggest this.
1. Install 6.x on a new machine using the "install for production"
instructions
2. Use the configs from one of the sample projects to create an
appropriately-named collection
3. Use the ability to "include" your configs into the other configs (they
live in separate files)
I can provide more help here if you're interested
4. Re-index all your data into the new version of SOLR...
I have rough, but useable docs on this if you are interested in attempting
this approach.
On Mon, Sep 12, 2016 at 3:48 PM, Aaron Greenspan <
aaron.greenspan@plainsite.org> wrote:
> Hi,
>
> I have been on this list for some time because I know that any time I try
> to do anything related to Solr I’m going to have to spend hours on it,
> wondering why everything has to be so awful, and I just want somewhere to
> provide feedback with the dim hope that the product might improve one day.
> (So far, for my purposes, it hasn’t.) Sure enough, I still absolutely hate
> using Solr, and I have more feedback.
>
> I started with a confusing error on the web console, which I still can’t
> figure out how to password protect without going through an insanely
> process involving "ZooKeeper," which I don’t know anything about, or have,
> to the best of my knowledge:
>
> Problem accessing /solr/. Reason:
>
> Forbidden
>
> According to logs, this apparently meant that a MySQL query had failed due
> to a field name change. Since I would have to change my XML configuration
> files, I decided to use the opportunity to upgrade from Solr 5.1.4 to
> 6.2.0. It broke everything.
>
> First I was getting errors about "Unsupported major.minor version 52.0",
> so I needed to install the Linux x64 JRE 1.8.0, which I managed on CentOS 6
> with...
>
> yum install openjdk-1.8.0
>
> ...going to Oracle’s web site, downloading the latest JRE 1.8 build, and
> then running...
>
> yum localinstall jre-8u101-linux-x64.rpm
>
> So far so good. But I didn’t have JAVA_HOME set properly apparently, so I
> needed to do the not-exactly-intuitive…
>
> export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.
> el6_8.x86_64/jre/
>
> As usual, I manually moved over my mysql-connector-java-5.1.38-bin.jar
> file from the dist/ folder in the old version to the new one. Then after
> stopping the old process (with kill -9, since there seems to be no graceful
> way to shut down Solr—README.txt doesn’t mention bin/solr stop) I moved
> over my two core folders from the old server/solr/ folder. I tried to start
> it up with bin/solr start, and watched the errors roll in.
>
> There was some kind of problem with StopFilterFactory and the text_general
> field type. Thanks to Stack Overflow I was able to determine that the
> apparent problem was that there was a parameter, previously fine, which was
> no longer fine. So I removed all instances of enablePositionIncrements="true".
> That helped, but then I ran into a broader error: "Plugin Initializing
> failure for [schema.xml] fieldType". It didn’t say which field type. Buried
> in the logs I found a reference in the Java stack trace—which *disappears*
> (and distorts the viewing window horribly) after a few seconds when you try
> to view it in the web log UI—to the string "units="degrees"". Sure enough,
> this string appeared in my schema.xml for a class called "solr.
> SpatialRecursivePrefixTreeFieldType" that I’m pretty sure I never use. I
> removed that parameter, and moved on to the next set of errors.
>
> Apparently there is some aspect of the Thai text field type that Solr
> 6.2.0 doesn’t like. So I disabled it. I don’t use Thai text.
>
> Now Solr was complaining about "Error loading class
> 'solr.admin.AdminHandlers'". So I found the reference to
> solr.admin.AdminHandlers in solrconfig.xml for each of my cores and
> commented it out. Only then did Solr work again.
>
> This was not a smooth process. It took about two hours. The user interface
> is still as buggy as an early alpha of most products, the errors are
> difficult to understand when they don’t actually specify what’s wrong (and
> they almost never do), and there should have been an automatic process to
> highlight and fix problems in old (pre-6) configuration files. Never mind
> the fact that the XML-based configuration process is an antiquated
> nightmare when the rest of the world has long since moved onto databases.
>
> Maybe this will help someone else out there.
>
> Aaron
>
> PlainSite | http://www.plainsite.org