You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Saar Carmi <sa...@gmail.com> on 2013/10/25 11:31:28 UTC
Solr - what's the next big thing?
If I am not mistaken the most impressive improvement of Solr 4.0 compared
to previous versions was the Solr Cloud architecture.
What would be the next big thing in Solr 5.0 ?
Saar
Re: Solr - what's the next big thing?
Posted by Shawn Heisey <so...@elyograg.org>.
On 10/25/2013 3:31 AM, Saar Carmi wrote:
> If I am not mistaken the most impressive improvement of Solr 4.0 compared
> to previous versions was the Solr Cloud architecture.
>
> What would be the next big thing in Solr 5.0 ?
There are a huge number of behind-the-scenes changes in 4.x compared to
3.x. SolrCloud is impressive, but equally impressive are things that
most people never see - the changes in Lucene that reduce memory
requirements, increase performance, and get rid of limitations.
Most changes that are currently being implemented are going into 4.x as
well as trunk, which will become 5.0 eventually. If you're looking for
something extremely visible as "the next big thing" then I would say
this will be the big one:
https://issues.apache.org/jira/browse/SOLR-4792
Solr is currently a webapp, a servlet. It is packaged as a war file,
and requires a servlet container. The 'start.jar' that comes in the
example is not Solr - it's the executable entry point for a stripped
install of Jetty.
The fact that Solr is a servlet has become restrictive in many ways.
SolrCloud is not as easy to use as it could be, because certain things
have to be configured that ideally should happen automatically.
When 5.0 begins its release process, it is likely that 'start.jar' will
not be a third-party application. One idea that has been proposed
(which I really like) is that start.jar will be an extremely tiny
management application that controls a separate process for Solr itself
- providing the ability to reconfigure and restart Solr, as well as
control things like java heap size and system properties in a config file.
There are no current plans to begin the release process on 5.0. The 4.x
branch has a lot of life left in it.
Thanks,
Shawn
Re: Solr - what's the next big thing?
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
On Sun, Oct 27, 2013 at 2:57 PM, Saar Carmi <sa...@gmail.com> wrote:
> If I get it right, Solr can store its data files on HDFS but it will not
Correct.
And can be used to build indices in parallel, using MapReduce, from
data living on HDFS.
> use map reduce to process the data (e.g. evaluating queries).
Right. MapReduce jobs are typically not a sub-second process, while
search queries typically need to be very quick.
That said, one could run a query and then apply MapReduce-based
processing on the search results. There is no support for that in
Solr today.
> I was wondering whether Solr could utilize the Hadoop job distribution
> mechanism to utlize resources better.
> On the otherhand, maybe this is not needed with the availability of Solr
> Cloud.
Maybe you are thinking Solr on YARN?
Mark Miller can probably say a word or two or three on this topic.
> Bill Bell, could you elaborate about complex object indexing?
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
> On Sat, Oct 26, 2013 at 10:04 PM, Otis Gospodnetic <
> otis.gospodnetic@gmail.com> wrote:
>
>> Hi,
>>
>> On Sat, Oct 26, 2013 at 5:58 AM, Saar Carmi <sa...@gmail.com> wrote:
>> > LOL, Jack. I can imagine Otis saying that.
>>
>> Funny indeed, but not really.
>>
>> > Otis, with these marriage, are we going to see map reduce based
>> queries?
>>
>> Can you please describe what you mean by that? Maybe with an example.
>>
>> Thanks,
>> Otis
>> --
>> Performance Monitoring * Log Analytics * Search Analytics
>> Solr & Elasticsearch Support * http://sematext.com/
>>
>>
>>
>> > On Oct 25, 2013 10:03 PM, "Jack Krupansky" <ja...@basetechnology.com>
>> wrote:
>> >
>> >> But a lot of that big yellow elephant stuff is in 4.x anyway.
>> >>
>> >> (Otis: I was afraid that you were going to say that the next big thing
>> in
>> >> Solr is... Elasticsearch!)
>> >>
>> >> -- Jack Krupansky
>> >>
>> >> -----Original Message----- From: Otis Gospodnetic
>> >> Sent: Friday, October 25, 2013 2:43 PM
>> >> To: solr-user@lucene.apache.org
>> >> Subject: Re: Solr - what's the next big thing?
>> >>
>> >> Saar,
>> >>
>> >> The marriage with the big yellow elephant is a big deal. It changes the
>> >> scale.
>> >>
>> >> Otis
>> >> Solr & ElasticSearch Support
>> >> http://sematext.com/
>> >> On Oct 25, 2013 5:32 AM, "Saar Carmi" <sa...@gmail.com> wrote:
>> >>
>> >> If I am not mistaken the most impressive improvement of Solr 4.0
>> compared
>> >>> to previous versions was the Solr Cloud architecture.
>> >>>
>> >>> What would be the next big thing in Solr 5.0 ?
>> >>>
>> >>> Saar
>> >>>
>> >>>
>> >>
>>
>
>
>
> --
> Saar Carmi
>
> Mobile: 054-7782417
> Email: saarcarmi@gmail.com
Re: Solr - what's the next big thing?
Posted by Saar Carmi <sa...@gmail.com>.
If I get it right, Solr can store its data files on HDFS but it will not
use map reduce to process the data (e.g. evaluating queries).
I was wondering whether Solr could utilize the Hadoop job distribution
mechanism to utlize resources better.
On the otherhand, maybe this is not needed with the availability of Solr
Cloud.
Bill Bell, could you elaborate about complex object indexing?
On Sat, Oct 26, 2013 at 10:04 PM, Otis Gospodnetic <
otis.gospodnetic@gmail.com> wrote:
> Hi,
>
> On Sat, Oct 26, 2013 at 5:58 AM, Saar Carmi <sa...@gmail.com> wrote:
> > LOL, Jack. I can imagine Otis saying that.
>
> Funny indeed, but not really.
>
> > Otis, with these marriage, are we going to see map reduce based
> queries?
>
> Can you please describe what you mean by that? Maybe with an example.
>
> Thanks,
> Otis
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>
>
>
> > On Oct 25, 2013 10:03 PM, "Jack Krupansky" <ja...@basetechnology.com>
> wrote:
> >
> >> But a lot of that big yellow elephant stuff is in 4.x anyway.
> >>
> >> (Otis: I was afraid that you were going to say that the next big thing
> in
> >> Solr is... Elasticsearch!)
> >>
> >> -- Jack Krupansky
> >>
> >> -----Original Message----- From: Otis Gospodnetic
> >> Sent: Friday, October 25, 2013 2:43 PM
> >> To: solr-user@lucene.apache.org
> >> Subject: Re: Solr - what's the next big thing?
> >>
> >> Saar,
> >>
> >> The marriage with the big yellow elephant is a big deal. It changes the
> >> scale.
> >>
> >> Otis
> >> Solr & ElasticSearch Support
> >> http://sematext.com/
> >> On Oct 25, 2013 5:32 AM, "Saar Carmi" <sa...@gmail.com> wrote:
> >>
> >> If I am not mistaken the most impressive improvement of Solr 4.0
> compared
> >>> to previous versions was the Solr Cloud architecture.
> >>>
> >>> What would be the next big thing in Solr 5.0 ?
> >>>
> >>> Saar
> >>>
> >>>
> >>
>
--
Saar Carmi
Mobile: 054-7782417
Email: saarcarmi@gmail.com
Re: Solr - what's the next big thing?
Posted by Michael Sokolov <ms...@safaribooksonline.com>.
On 10/26/2013 8:31 PM, Bill Bell wrote:
> Full JSON support deep complex object indexing and search.... Game changer
>
> Bill Bell
> Sent from mobile
>
>
Not JSON (yet?) but take a look at http://luxdb.org which does XML
indexing and search. We index all the text of all the nodes in your
tree: no need to define full text indexes for those. I think this might
be something like what you are talking about? I'll be giving a talk on
that at Lucene Revolution in Dublin next week. I'm also curious to see
this talk: http://www.lucenerevolution.org/sessions#Renaud_Delbru, and
you might be interested in what the folks at 28.io are doing with JSONiq
(not Solr, though they talk about being able to plug into different back
ends)
-Mike
Re: Solr - what's the next big thing?
Posted by Bill Bell <bi...@gmail.com>.
Full JSON support deep complex object indexing and search.... Game changer
Bill Bell
Sent from mobile
> On Oct 26, 2013, at 1:04 PM, Otis Gospodnetic <ot...@gmail.com> wrote:
>
> Hi,
>
>> On Sat, Oct 26, 2013 at 5:58 AM, Saar Carmi <sa...@gmail.com> wrote:
>> LOL, Jack. I can imagine Otis saying that.
>
> Funny indeed, but not really.
>
>> Otis, with these marriage, are we going to see map reduce based queries?
>
> Can you please describe what you mean by that? Maybe with an example.
>
> Thanks,
> Otis
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>
>
>
>>> On Oct 25, 2013 10:03 PM, "Jack Krupansky" <ja...@basetechnology.com> wrote:
>>>
>>> But a lot of that big yellow elephant stuff is in 4.x anyway.
>>>
>>> (Otis: I was afraid that you were going to say that the next big thing in
>>> Solr is... Elasticsearch!)
>>>
>>> -- Jack Krupansky
>>>
>>> -----Original Message----- From: Otis Gospodnetic
>>> Sent: Friday, October 25, 2013 2:43 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Solr - what's the next big thing?
>>>
>>> Saar,
>>>
>>> The marriage with the big yellow elephant is a big deal. It changes the
>>> scale.
>>>
>>> Otis
>>> Solr & ElasticSearch Support
>>> http://sematext.com/
>>> On Oct 25, 2013 5:32 AM, "Saar Carmi" <sa...@gmail.com> wrote:
>>>
>>> If I am not mistaken the most impressive improvement of Solr 4.0 compared
>>>> to previous versions was the Solr Cloud architecture.
>>>>
>>>> What would be the next big thing in Solr 5.0 ?
>>>>
>>>> Saar
>>>
Re: Solr - what's the next big thing?
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
On Sat, Oct 26, 2013 at 5:58 AM, Saar Carmi <sa...@gmail.com> wrote:
> LOL, Jack. I can imagine Otis saying that.
Funny indeed, but not really.
> Otis, with these marriage, are we going to see map reduce based queries?
Can you please describe what you mean by that? Maybe with an example.
Thanks,
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
> On Oct 25, 2013 10:03 PM, "Jack Krupansky" <ja...@basetechnology.com> wrote:
>
>> But a lot of that big yellow elephant stuff is in 4.x anyway.
>>
>> (Otis: I was afraid that you were going to say that the next big thing in
>> Solr is... Elasticsearch!)
>>
>> -- Jack Krupansky
>>
>> -----Original Message----- From: Otis Gospodnetic
>> Sent: Friday, October 25, 2013 2:43 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Solr - what's the next big thing?
>>
>> Saar,
>>
>> The marriage with the big yellow elephant is a big deal. It changes the
>> scale.
>>
>> Otis
>> Solr & ElasticSearch Support
>> http://sematext.com/
>> On Oct 25, 2013 5:32 AM, "Saar Carmi" <sa...@gmail.com> wrote:
>>
>> If I am not mistaken the most impressive improvement of Solr 4.0 compared
>>> to previous versions was the Solr Cloud architecture.
>>>
>>> What would be the next big thing in Solr 5.0 ?
>>>
>>> Saar
>>>
>>>
>>
Re: Solr - what's the next big thing?
Posted by Saar Carmi <sa...@gmail.com>.
LOL, Jack. I can imagine Otis saying that.
Otis, with these marriage, are we going to see map reduce based queries?
On Oct 25, 2013 10:03 PM, "Jack Krupansky" <ja...@basetechnology.com> wrote:
> But a lot of that big yellow elephant stuff is in 4.x anyway.
>
> (Otis: I was afraid that you were going to say that the next big thing in
> Solr is... Elasticsearch!)
>
> -- Jack Krupansky
>
> -----Original Message----- From: Otis Gospodnetic
> Sent: Friday, October 25, 2013 2:43 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr - what's the next big thing?
>
> Saar,
>
> The marriage with the big yellow elephant is a big deal. It changes the
> scale.
>
> Otis
> Solr & ElasticSearch Support
> http://sematext.com/
> On Oct 25, 2013 5:32 AM, "Saar Carmi" <sa...@gmail.com> wrote:
>
> If I am not mistaken the most impressive improvement of Solr 4.0 compared
>> to previous versions was the Solr Cloud architecture.
>>
>> What would be the next big thing in Solr 5.0 ?
>>
>> Saar
>>
>>
>
Re: Solr - what's the next big thing?
Posted by Jack Krupansky <ja...@basetechnology.com>.
But a lot of that big yellow elephant stuff is in 4.x anyway.
(Otis: I was afraid that you were going to say that the next big thing in
Solr is... Elasticsearch!)
-- Jack Krupansky
-----Original Message-----
From: Otis Gospodnetic
Sent: Friday, October 25, 2013 2:43 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr - what's the next big thing?
Saar,
The marriage with the big yellow elephant is a big deal. It changes the
scale.
Otis
Solr & ElasticSearch Support
http://sematext.com/
On Oct 25, 2013 5:32 AM, "Saar Carmi" <sa...@gmail.com> wrote:
> If I am not mistaken the most impressive improvement of Solr 4.0 compared
> to previous versions was the Solr Cloud architecture.
>
> What would be the next big thing in Solr 5.0 ?
>
> Saar
>
Re: Solr - what's the next big thing?
Posted by Otis Gospodnetic <ot...@gmail.com>.
Saar,
The marriage with the big yellow elephant is a big deal. It changes the
scale.
Otis
Solr & ElasticSearch Support
http://sematext.com/
On Oct 25, 2013 5:32 AM, "Saar Carmi" <sa...@gmail.com> wrote:
> If I am not mistaken the most impressive improvement of Solr 4.0 compared
> to previous versions was the Solr Cloud architecture.
>
> What would be the next big thing in Solr 5.0 ?
>
> Saar
>
Re: Solr - what's the next big thing?
Posted by Jack Krupansky <ja...@basetechnology.com>.
Nothing of great significance that I have heard of, yet. Solr 4.x has lots
of ongoing significant improvements and will likely to have a lot more in
the months ahead. Ditto with Lucene improvements - lots of ongoing work to
enhance 4.x.
-- Jack Krupansky
-----Original Message-----
From: Saar Carmi
Sent: Friday, October 25, 2013 5:31 AM
To: solr-user@lucene.apache.org
Subject: Solr - what's the next big thing?
If I am not mistaken the most impressive improvement of Solr 4.0 compared
to previous versions was the Solr Cloud architecture.
What would be the next big thing in Solr 5.0 ?
Saar