You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Arkadi Colson <ar...@smartbit.be> on 2013/04/23 16:45:32 UTC
Update on shards
Hi
Is it correct that when inserting or updating document into solr you
have to talk to a solr host where at least one shard of that collection
is stored?
For select you can talk to any host within the collection.configName?
BR,
Arkadi
Re: Update on shards
Posted by Furkan KAMACI <fu...@gmail.com>.
Mark, by the way is there an approximate date about when will Solr 4.3 be
ready?
2013/4/23 Mark Miller <ma...@gmail.com>
> We have a 3rd release candidate for 4.3 being voted on now.
>
> I have never tested this feature with Tomcat - only Jetty. Users have
> reported it does not work with Tomcat. That leads one to think it may have
> a problem in other containers as well.
>
> A previous contributor donated a patch that explicitly flushes a stream in
> our proxy code - he says this allows the feature to work with Tomcat. I
> committed this feature - the flush can't hurt, and given the previous
> contributions of this individual, I'm fairly confident the fix makes things
> work in Tomcat. I have no first hand knowledge that it does work though.
>
> You might take the RC for a spin and test it our yourself:
> http://people.apache.org/~simonw/staging_area/lucene-solr-4.3.0-RC3-rev1470846/
>
> - Mark
>
> On Apr 23, 2013, at 3:20 PM, Furkan KAMACI <fu...@gmail.com> wrote:
>
> > Hi Mark;
> >
> > All in all you say that when 4.3 is tagged at repository (I mean when it
> is
> > ready) this feature will work for Tomcat too at a stable version?
> >
> >
> > 2013/4/23 Mark Miller <ma...@gmail.com>
> >
> >>
> >> On Apr 23, 2013, at 2:49 PM, Shawn Heisey <so...@elyograg.org> wrote:
> >>
> >>> What exactly is the 'request proxying' thing that doesn't work on
> >> tomcat? Is this something different from basic SolrCloud operation
> where
> >> you send any kind of request to any server and they get directed where
> they
> >> need to go? I haven't heard of that not working on tomcat before.
> >>
> >> Before 4.2, if you made a read request to a node that didn't contain
> part
> >> of the collection you where searching, it would return 404. Write
> requests
> >> would be forwarded to where they belong no matter what node you sent
> them
> >> to, but read requests required that node have a part of the collection
> you
> >> were accessing.
> >>
> >> In 4.2 we added request proxying for this read side case. If a piece of
> >> the collection you are querying is not found on the node you hit, a
> simple
> >> proxy of the request is done to a node that does contain a piece of the
> >> collection.
> >>
> >> - Mark
>
>
Re: Update on shards
Posted by Arkadi Colson <ar...@smartbit.be>.
Anyone else having this problem that an update needs to go to a host
where a shard exists.
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
Solr 4.2.1
apache-tomcat-7.0.33
Thx!
Met vriendelijke groeten
Arkadi Colson
Smartbit bvba • Hoogstraat 13 • 3670 Meeuwen
T +32 11 64 08 80 • F +32 11 64 08 81
On 04/25/2013 09:18 AM, Arkadi Colson wrote:
> Hi
>
> It seems not to work in my case. We are using the solr php module for
> talking to Solr. Currently we have 2 collections 'intradesk' and 'lvs'
> for 10 solr hosts (shards: 5 - repl: 2). Because there is no more disc
> space I created 6 new hosts for collection 'messages' (shards: 3 -
> repl: 2).
>
> 'intradesk + lvs':
> solr01-dcg
> solr01-gs
> solr02-dcg
> solr02-gs
> solr03-dcg
> solr03-gs
> solr04-dcg
> solr04-gs
> solr05-dcg
> solr05-gs
>
> 'messages':
> solr06-dcg
> solr06-gs
> solr07-dcg
> solr07-gs
> solr08-dcg
> solr08-gs
>
> So when doing a select, I can talk to any host. When updating I must
> talk to a host with at least 1 shard on it.
>
> I created the new messages shard with the following command to get
> them on the new hosts (06 -> 08):
> http://solr01-dcg.intnet.smartbit.be:8983/solr/admin/collections?action=CREATE&name=messages&numShards=3&replicationFactor=2&collection.configName=smsc&createNodeSet=solr06-gs.intnet.smartbit.be:8983_solr,solr06-dcg.intnet.smartbit.be:8983_solr,solr07-gs.intnet.smartbit.be:8983_solr,solr07-dcg.intnet.smartbit.be:8983_solr,solr08-gs.intnet.smartbit.be:8983_solr,solr08-dcg.intnet.smartbit.be:8983_solr
>
>
> They are all in the same config set 'smsc'.
>
> Below is the code:
>
> $client = new SolrClient(
> array(
> 'hostname' => "solr01-dcg.intnet.smartbit.be
> <http://solr01-dcg.intnet.smartbit.be:8983/solr/admin/collections?action=CREATE&name=messages&numShards=3&replicationFactor=2&collection.configName=smsc&createNodeSet=solr06-gs.intnet.smartbit.be:8983_solr,solr06-dcg.intnet.smartbit.be:8983_solr,solr07-gs.intnet.smartbit.be:8983_solr,solr07-dcg.intnet.smartbit.be:8983_solr,solr08-gs.intnet.smartbit.be:8983_solr,solr08-dcg.intnet.smartbit.be:8983_solr>",
> 'port' => "8983",
> 'login' => "***",
> 'password' => "***",
> 'path' => "solr/messages",
> 'wt' => "json"
> )
> );
>
> $doc = new SolrInputDocument();
> $doc->addField('id', $uniqueID);
> $doc->addField('smsc_ssid', $ssID);
> $doc->addField('smsc_module', $i['module']);
> $doc->addField('smsc_modulekey', $i['moduleKey']);
> $doc->addField('smsc_courseid', $courseID);
> $doc->addField('smsc_description', $i['description']);
> $doc->addField('smsc_content', $i['content']);
> $doc->addField('smsc_lastdate', $lastdate);
> $doc->addField('smsc_userid', $userID);
>
> $client->addDocument($doc);
>
> The exception I get look like this:
> exception 'SolrClientException' with message 'Unsuccessful update
> request. Response Code 200. (null)'
>
> Nothing special to find in the solr log.
>
> Any idea?
>
>
> Arkadi
>
> On 04/24/2013 08:43 PM, Mark Miller wrote:
>> Sorry - need to correct myself - updates worked the same as read
>> requests - they also needed to hit a SolrCore in order to get
>> forwarded to the right node. I was not thinking clearly when I said
>> this applied to just reads and not writes. Both needed a SolrCore to
>> do their work - with the request proxying, this is no longer the
>> case, so you can hit Solr instances with no SolrCores or with
>> SolrCores that are not part of the collection you are working with,
>> and both read and write side requests are now proxied to a suitable
>> node that has a SolrCore that can do the search or forward the update
>> (or accept the update).
>>
>> - Mark
>>
>> On Apr 23, 2013, at 3:38 PM, Mark Miller <ma...@gmail.com> wrote:
>>
>>> We have a 3rd release candidate for 4.3 being voted on now.
>>>
>>> I have never tested this feature with Tomcat - only Jetty. Users
>>> have reported it does not work with Tomcat. That leads one to think
>>> it may have a problem in other containers as well.
>>>
>>> A previous contributor donated a patch that explicitly flushes a
>>> stream in our proxy code - he says this allows the feature to work
>>> with Tomcat. I committed this feature - the flush can't hurt, and
>>> given the previous contributions of this individual, I'm fairly
>>> confident the fix makes things work in Tomcat. I have no first hand
>>> knowledge that it does work though.
>>>
>>> You might take the RC for a spin and test it our yourself:
>>> http://people.apache.org/~simonw/staging_area/lucene-solr-4.3.0-RC3-rev1470846/
>>>
>>> - Mark
>>>
>>> On Apr 23, 2013, at 3:20 PM, Furkan KAMACI <fu...@gmail.com>
>>> wrote:
>>>
>>>> Hi Mark;
>>>>
>>>> All in all you say that when 4.3 is tagged at repository (I mean
>>>> when it is
>>>> ready) this feature will work for Tomcat too at a stable version?
>>>>
>>>>
>>>> 2013/4/23 Mark Miller <ma...@gmail.com>
>>>>
>>>>> On Apr 23, 2013, at 2:49 PM, Shawn Heisey <so...@elyograg.org> wrote:
>>>>>
>>>>>> What exactly is the 'request proxying' thing that doesn't work on
>>>>> tomcat? Is this something different from basic SolrCloud operation
>>>>> where
>>>>> you send any kind of request to any server and they get directed
>>>>> where they
>>>>> need to go? I haven't heard of that not working on tomcat before.
>>>>>
>>>>> Before 4.2, if you made a read request to a node that didn't
>>>>> contain part
>>>>> of the collection you where searching, it would return 404. Write
>>>>> requests
>>>>> would be forwarded to where they belong no matter what node you
>>>>> sent them
>>>>> to, but read requests required that node have a part of the
>>>>> collection you
>>>>> were accessing.
>>>>>
>>>>> In 4.2 we added request proxying for this read side case. If a
>>>>> piece of
>>>>> the collection you are querying is not found on the node you hit,
>>>>> a simple
>>>>> proxy of the request is done to a node that does contain a piece
>>>>> of the
>>>>> collection.
>>>>>
>>>>> - Mark
>>
>>
>
>
>
>
Re: Update on shards
Posted by Arkadi Colson <ar...@smartbit.be>.
Anyone an idea how to debug this?
Thx!
On 04/25/2013 09:18 AM, Arkadi Colson wrote:
> Hi
>
> It seems not to work in my case. We are using the solr php module for
> talking to Solr. Currently we have 2 collections 'intradesk' and 'lvs'
> for 10 solr hosts (shards: 5 - repl: 2). Because there is no more disc
> space I created 6 new hosts for collection 'messages' (shards: 3 -
> repl: 2).
>
> 'intradesk + lvs':
> solr01-dcg
> solr01-gs
> solr02-dcg
> solr02-gs
> solr03-dcg
> solr03-gs
> solr04-dcg
> solr04-gs
> solr05-dcg
> solr05-gs
>
> 'messages':
> solr06-dcg
> solr06-gs
> solr07-dcg
> solr07-gs
> solr08-dcg
> solr08-gs
>
> So when doing a select, I can talk to any host. When updating I must
> talk to a host with at least 1 shard on it.
>
> I created the new messages shard with the following command to get
> them on the new hosts (06 -> 08):
> http://solr01-dcg.intnet.smartbit.be:8983/solr/admin/collections?action=CREATE&name=messages&numShards=3&replicationFactor=2&collection.configName=smsc&createNodeSet=solr06-gs.intnet.smartbit.be:8983_solr,solr06-dcg.intnet.smartbit.be:8983_solr,solr07-gs.intnet.smartbit.be:8983_solr,solr07-dcg.intnet.smartbit.be:8983_solr,solr08-gs.intnet.smartbit.be:8983_solr,solr08-dcg.intnet.smartbit.be:8983_solr
>
>
> They are all in the same config set 'smsc'.
>
> Below is the code:
>
> $client = new SolrClient(
> array(
> 'hostname' => "solr01-dcg.intnet.smartbit.be
> <http://solr01-dcg.intnet.smartbit.be:8983/solr/admin/collections?action=CREATE&name=messages&numShards=3&replicationFactor=2&collection.configName=smsc&createNodeSet=solr06-gs.intnet.smartbit.be:8983_solr,solr06-dcg.intnet.smartbit.be:8983_solr,solr07-gs.intnet.smartbit.be:8983_solr,solr07-dcg.intnet.smartbit.be:8983_solr,solr08-gs.intnet.smartbit.be:8983_solr,solr08-dcg.intnet.smartbit.be:8983_solr>",
> 'port' => "8983",
> 'login' => "***",
> 'password' => "***",
> 'path' => "solr/messages",
> 'wt' => "json"
> )
> );
>
> $doc = new
> SolrInputDocument();
> $doc->addField('id', $uniqueID);
> $doc->addField('smsc_ssid', $ssID);
> $doc->addField('smsc_module', $i['module']);
> $doc->addField('smsc_modulekey', $i['moduleKey']);
> $doc->addField('smsc_courseid', $courseID);
> $doc->addField('smsc_description', $i['description']);
> $doc->addField('smsc_content', $i['content']);
> $doc->addField('smsc_lastdate', $lastdate);
> $doc->addField('smsc_userid', $userID);
>
> $client->addDocument($doc);
>
> The exception I get look like this:
> exception 'SolrClientException' with message 'Unsuccessful update
> request. Response Code 200. (null)'
>
> Nothing special to find in the solr log.
>
> Any idea?
>
>
> Arkadi
>
> On 04/24/2013 08:43 PM, Mark Miller wrote:
>> Sorry - need to correct myself - updates worked the same as read
>> requests - they also needed to hit a SolrCore in order to get
>> forwarded to the right node. I was not thinking clearly when I said
>> this applied to just reads and not writes. Both needed a SolrCore to
>> do their work - with the request proxying, this is no longer the
>> case, so you can hit Solr instances with no SolrCores or with
>> SolrCores that are not part of the collection you are working with,
>> and both read and write side requests are now proxied to a suitable
>> node that has a SolrCore that can do the search or forward the update
>> (or accept the update).
>>
>> - Mark
>>
>> On Apr 23, 2013, at 3:38 PM, Mark Miller <ma...@gmail.com> wrote:
>>
>>> We have a 3rd release candidate for 4.3 being voted on now.
>>>
>>> I have never tested this feature with Tomcat - only Jetty. Users
>>> have reported it does not work with Tomcat. That leads one to think
>>> it may have a problem in other containers as well.
>>>
>>> A previous contributor donated a patch that explicitly flushes a
>>> stream in our proxy code - he says this allows the feature to work
>>> with Tomcat. I committed this feature - the flush can't hurt, and
>>> given the previous contributions of this individual, I'm fairly
>>> confident the fix makes things work in Tomcat. I have no first hand
>>> knowledge that it does work though.
>>>
>>> You might take the RC for a spin and test it our yourself:
>>> http://people.apache.org/~simonw/staging_area/lucene-solr-4.3.0-RC3-rev1470846/
>>>
>>> - Mark
>>>
>>> On Apr 23, 2013, at 3:20 PM, Furkan KAMACI <fu...@gmail.com>
>>> wrote:
>>>
>>>> Hi Mark;
>>>>
>>>> All in all you say that when 4.3 is tagged at repository (I mean
>>>> when it is
>>>> ready) this feature will work for Tomcat too at a stable version?
>>>>
>>>>
>>>> 2013/4/23 Mark Miller <ma...@gmail.com>
>>>>
>>>>> On Apr 23, 2013, at 2:49 PM, Shawn Heisey <so...@elyograg.org> wrote:
>>>>>
>>>>>> What exactly is the 'request proxying' thing that doesn't work on
>>>>> tomcat? Is this something different from basic SolrCloud
>>>>> operation where
>>>>> you send any kind of request to any server and they get directed
>>>>> where they
>>>>> need to go? I haven't heard of that not working on tomcat before.
>>>>>
>>>>> Before 4.2, if you made a read request to a node that didn't
>>>>> contain part
>>>>> of the collection you where searching, it would return 404. Write
>>>>> requests
>>>>> would be forwarded to where they belong no matter what node you
>>>>> sent them
>>>>> to, but read requests required that node have a part of the
>>>>> collection you
>>>>> were accessing.
>>>>>
>>>>> In 4.2 we added request proxying for this read side case. If a
>>>>> piece of
>>>>> the collection you are querying is not found on the node you hit,
>>>>> a simple
>>>>> proxy of the request is done to a node that does contain a piece
>>>>> of the
>>>>> collection.
>>>>>
>>>>> - Mark
>>
>>
>
>
>
>
Re: Update on shards
Posted by Arkadi Colson <ar...@smartbit.be>.
Hi
It seems not to work in my case. We are using the solr php module for
talking to Solr. Currently we have 2 collections 'intradesk' and 'lvs'
for 10 solr hosts (shards: 5 - repl: 2). Because there is no more disc
space I created 6 new hosts for collection 'messages' (shards: 3 - repl: 2).
'intradesk + lvs':
solr01-dcg
solr01-gs
solr02-dcg
solr02-gs
solr03-dcg
solr03-gs
solr04-dcg
solr04-gs
solr05-dcg
solr05-gs
'messages':
solr06-dcg
solr06-gs
solr07-dcg
solr07-gs
solr08-dcg
solr08-gs
So when doing a select, I can talk to any host. When updating I must
talk to a host with at least 1 shard on it.
I created the new messages shard with the following command to get them
on the new hosts (06 -> 08):
http://solr01-dcg.intnet.smartbit.be:8983/solr/admin/collections?action=CREATE&name=messages&numShards=3&replicationFactor=2&collection.configName=smsc&createNodeSet=solr06-gs.intnet.smartbit.be:8983_solr,solr06-dcg.intnet.smartbit.be:8983_solr,solr07-gs.intnet.smartbit.be:8983_solr,solr07-dcg.intnet.smartbit.be:8983_solr,solr08-gs.intnet.smartbit.be:8983_solr,solr08-dcg.intnet.smartbit.be:8983_solr
They are all in the same config set 'smsc'.
Below is the code:
$client = new SolrClient(
array(
'hostname' => "solr01-dcg.intnet.smartbit.be
<http://solr01-dcg.intnet.smartbit.be:8983/solr/admin/collections?action=CREATE&name=messages&numShards=3&replicationFactor=2&collection.configName=smsc&createNodeSet=solr06-gs.intnet.smartbit.be:8983_solr,solr06-dcg.intnet.smartbit.be:8983_solr,solr07-gs.intnet.smartbit.be:8983_solr,solr07-dcg.intnet.smartbit.be:8983_solr,solr08-gs.intnet.smartbit.be:8983_solr,solr08-dcg.intnet.smartbit.be:8983_solr>",
'port' => "8983",
'login' => "***",
'password' => "***",
'path' => "solr/messages",
'wt' => "json"
)
);
$doc = new
SolrInputDocument();
$doc->addField('id', $uniqueID);
$doc->addField('smsc_ssid', $ssID);
$doc->addField('smsc_module', $i['module']);
$doc->addField('smsc_modulekey', $i['moduleKey']);
$doc->addField('smsc_courseid', $courseID);
$doc->addField('smsc_description', $i['description']);
$doc->addField('smsc_content', $i['content']);
$doc->addField('smsc_lastdate', $lastdate);
$doc->addField('smsc_userid', $userID);
$client->addDocument($doc);
The exception I get look like this:
exception 'SolrClientException' with message 'Unsuccessful update
request. Response Code 200. (null)'
Nothing special to find in the solr log.
Any idea?
Arkadi
On 04/24/2013 08:43 PM, Mark Miller wrote:
> Sorry - need to correct myself - updates worked the same as read requests - they also needed to hit a SolrCore in order to get forwarded to the right node. I was not thinking clearly when I said this applied to just reads and not writes. Both needed a SolrCore to do their work - with the request proxying, this is no longer the case, so you can hit Solr instances with no SolrCores or with SolrCores that are not part of the collection you are working with, and both read and write side requests are now proxied to a suitable node that has a SolrCore that can do the search or forward the update (or accept the update).
>
> - Mark
>
> On Apr 23, 2013, at 3:38 PM, Mark Miller <ma...@gmail.com> wrote:
>
>> We have a 3rd release candidate for 4.3 being voted on now.
>>
>> I have never tested this feature with Tomcat - only Jetty. Users have reported it does not work with Tomcat. That leads one to think it may have a problem in other containers as well.
>>
>> A previous contributor donated a patch that explicitly flushes a stream in our proxy code - he says this allows the feature to work with Tomcat. I committed this feature - the flush can't hurt, and given the previous contributions of this individual, I'm fairly confident the fix makes things work in Tomcat. I have no first hand knowledge that it does work though.
>>
>> You might take the RC for a spin and test it our yourself: http://people.apache.org/~simonw/staging_area/lucene-solr-4.3.0-RC3-rev1470846/
>>
>> - Mark
>>
>> On Apr 23, 2013, at 3:20 PM, Furkan KAMACI <fu...@gmail.com> wrote:
>>
>>> Hi Mark;
>>>
>>> All in all you say that when 4.3 is tagged at repository (I mean when it is
>>> ready) this feature will work for Tomcat too at a stable version?
>>>
>>>
>>> 2013/4/23 Mark Miller <ma...@gmail.com>
>>>
>>>> On Apr 23, 2013, at 2:49 PM, Shawn Heisey <so...@elyograg.org> wrote:
>>>>
>>>>> What exactly is the 'request proxying' thing that doesn't work on
>>>> tomcat? Is this something different from basic SolrCloud operation where
>>>> you send any kind of request to any server and they get directed where they
>>>> need to go? I haven't heard of that not working on tomcat before.
>>>>
>>>> Before 4.2, if you made a read request to a node that didn't contain part
>>>> of the collection you where searching, it would return 404. Write requests
>>>> would be forwarded to where they belong no matter what node you sent them
>>>> to, but read requests required that node have a part of the collection you
>>>> were accessing.
>>>>
>>>> In 4.2 we added request proxying for this read side case. If a piece of
>>>> the collection you are querying is not found on the node you hit, a simple
>>>> proxy of the request is done to a node that does contain a piece of the
>>>> collection.
>>>>
>>>> - Mark
>
>
Re: Update on shards
Posted by Mark Miller <ma...@gmail.com>.
Sorry - need to correct myself - updates worked the same as read requests - they also needed to hit a SolrCore in order to get forwarded to the right node. I was not thinking clearly when I said this applied to just reads and not writes. Both needed a SolrCore to do their work - with the request proxying, this is no longer the case, so you can hit Solr instances with no SolrCores or with SolrCores that are not part of the collection you are working with, and both read and write side requests are now proxied to a suitable node that has a SolrCore that can do the search or forward the update (or accept the update).
- Mark
On Apr 23, 2013, at 3:38 PM, Mark Miller <ma...@gmail.com> wrote:
> We have a 3rd release candidate for 4.3 being voted on now.
>
> I have never tested this feature with Tomcat - only Jetty. Users have reported it does not work with Tomcat. That leads one to think it may have a problem in other containers as well.
>
> A previous contributor donated a patch that explicitly flushes a stream in our proxy code - he says this allows the feature to work with Tomcat. I committed this feature - the flush can't hurt, and given the previous contributions of this individual, I'm fairly confident the fix makes things work in Tomcat. I have no first hand knowledge that it does work though.
>
> You might take the RC for a spin and test it our yourself: http://people.apache.org/~simonw/staging_area/lucene-solr-4.3.0-RC3-rev1470846/
>
> - Mark
>
> On Apr 23, 2013, at 3:20 PM, Furkan KAMACI <fu...@gmail.com> wrote:
>
>> Hi Mark;
>>
>> All in all you say that when 4.3 is tagged at repository (I mean when it is
>> ready) this feature will work for Tomcat too at a stable version?
>>
>>
>> 2013/4/23 Mark Miller <ma...@gmail.com>
>>
>>>
>>> On Apr 23, 2013, at 2:49 PM, Shawn Heisey <so...@elyograg.org> wrote:
>>>
>>>> What exactly is the 'request proxying' thing that doesn't work on
>>> tomcat? Is this something different from basic SolrCloud operation where
>>> you send any kind of request to any server and they get directed where they
>>> need to go? I haven't heard of that not working on tomcat before.
>>>
>>> Before 4.2, if you made a read request to a node that didn't contain part
>>> of the collection you where searching, it would return 404. Write requests
>>> would be forwarded to where they belong no matter what node you sent them
>>> to, but read requests required that node have a part of the collection you
>>> were accessing.
>>>
>>> In 4.2 we added request proxying for this read side case. If a piece of
>>> the collection you are querying is not found on the node you hit, a simple
>>> proxy of the request is done to a node that does contain a piece of the
>>> collection.
>>>
>>> - Mark
>
Re: Update on shards
Posted by Mark Miller <ma...@gmail.com>.
We have a 3rd release candidate for 4.3 being voted on now.
I have never tested this feature with Tomcat - only Jetty. Users have reported it does not work with Tomcat. That leads one to think it may have a problem in other containers as well.
A previous contributor donated a patch that explicitly flushes a stream in our proxy code - he says this allows the feature to work with Tomcat. I committed this feature - the flush can't hurt, and given the previous contributions of this individual, I'm fairly confident the fix makes things work in Tomcat. I have no first hand knowledge that it does work though.
You might take the RC for a spin and test it our yourself: http://people.apache.org/~simonw/staging_area/lucene-solr-4.3.0-RC3-rev1470846/
- Mark
On Apr 23, 2013, at 3:20 PM, Furkan KAMACI <fu...@gmail.com> wrote:
> Hi Mark;
>
> All in all you say that when 4.3 is tagged at repository (I mean when it is
> ready) this feature will work for Tomcat too at a stable version?
>
>
> 2013/4/23 Mark Miller <ma...@gmail.com>
>
>>
>> On Apr 23, 2013, at 2:49 PM, Shawn Heisey <so...@elyograg.org> wrote:
>>
>>> What exactly is the 'request proxying' thing that doesn't work on
>> tomcat? Is this something different from basic SolrCloud operation where
>> you send any kind of request to any server and they get directed where they
>> need to go? I haven't heard of that not working on tomcat before.
>>
>> Before 4.2, if you made a read request to a node that didn't contain part
>> of the collection you where searching, it would return 404. Write requests
>> would be forwarded to where they belong no matter what node you sent them
>> to, but read requests required that node have a part of the collection you
>> were accessing.
>>
>> In 4.2 we added request proxying for this read side case. If a piece of
>> the collection you are querying is not found on the node you hit, a simple
>> proxy of the request is done to a node that does contain a piece of the
>> collection.
>>
>> - Mark
Re: Update on shards
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi Mark;
All in all you say that when 4.3 is tagged at repository (I mean when it is
ready) this feature will work for Tomcat too at a stable version?
2013/4/23 Mark Miller <ma...@gmail.com>
>
> On Apr 23, 2013, at 2:49 PM, Shawn Heisey <so...@elyograg.org> wrote:
>
> > What exactly is the 'request proxying' thing that doesn't work on
> tomcat? Is this something different from basic SolrCloud operation where
> you send any kind of request to any server and they get directed where they
> need to go? I haven't heard of that not working on tomcat before.
>
> Before 4.2, if you made a read request to a node that didn't contain part
> of the collection you where searching, it would return 404. Write requests
> would be forwarded to where they belong no matter what node you sent them
> to, but read requests required that node have a part of the collection you
> were accessing.
>
> In 4.2 we added request proxying for this read side case. If a piece of
> the collection you are querying is not found on the node you hit, a simple
> proxy of the request is done to a node that does contain a piece of the
> collection.
>
> - Mark
Re: Update on shards
Posted by Mark Miller <ma...@gmail.com>.
On Apr 23, 2013, at 2:49 PM, Shawn Heisey <so...@elyograg.org> wrote:
> What exactly is the 'request proxying' thing that doesn't work on tomcat? Is this something different from basic SolrCloud operation where you send any kind of request to any server and they get directed where they need to go? I haven't heard of that not working on tomcat before.
Before 4.2, if you made a read request to a node that didn't contain part of the collection you where searching, it would return 404. Write requests would be forwarded to where they belong no matter what node you sent them to, but read requests required that node have a part of the collection you were accessing.
In 4.2 we added request proxying for this read side case. If a piece of the collection you are querying is not found on the node you hit, a simple proxy of the request is done to a node that does contain a piece of the collection.
- Mark
Re: Update on shards
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/23/2013 10:14 AM, Mark Miller wrote:
> If you use jetty - which you should :) It's what we test with. Tomcat only gets user testing.
>
> If you use tomcat, this won't work in 4.2 or 4.2.1, but probably will in 4.3 (we are voting on 4.3 now).
>
> No clue on other containers.
>
> - Mark
>
> On Apr 23, 2013, at 10:59 AM, Michael Della Bitta <mi...@appinions.com> wrote:
>
>> I believe as of 4.2 you can talk to any host in the cloud.
What exactly is the 'request proxying' thing that doesn't work on
tomcat? Is this something different from basic SolrCloud operation
where you send any kind of request to any server and they get directed
where they need to go? I haven't heard of that not working on tomcat
before.
If there's a Jira issue that explains this in detail, you can just send
me there.
Thanks,
Shawn
Re: Update on shards
Posted by Arkadi Colson <ar...@smartbit.be>.
Thx!
On 04/24/2013 04:46 PM, Shawn Heisey wrote:
> On 4/24/2013 12:49 AM, Arkadi Colson wrote:
>> We are using tomcat so we'll just wait. Hopefully it's fixed in 4.3 but
>> we have a work around for now so...
>>
>> What exactly is the difference between jetty and tomcat. We are using
>> tomcat because we've read somewhere that it should be more robust in
>> heavily loaded production environments.
> Both are servlet containers - a Java executable server program that can
> run other programs written using the Java Servlet API. The servlet API
> was invented by Sun, who also invented Java itself.
>
> For comparison purposes, first think about Apache's HTTPD, which is a
> web server designed to serve files. Through its rich modular
> capability, it does have the ability to run web applications, but the
> core HTTPD is designed to grab a file off the hard drive and send it to
> a user.
>
> A servlet container is different. You can think of a servlet container
> as a smart web server designed from the ground up to run web applications.
>
> http://en.wikipedia.org/wiki/Java_Servlet
>
> Solr is a servlet. By itself, Solr can't run. It requires a servlet
> container.
>
> Here is what wikipedia has to say about the histories of the two projects:
>
> http://en.wikipedia.org/wiki/Apache_Tomcat#History
> http://en.wikipedia.org/wiki/Jetty_%28Web_server%29#History
>
> If you google "difference between jetty and tomcat" you'll find a lot of
> links. The one written by Jetty folks is particularly detailed, but has
> an obvious bias.
>
> With emphasis on tuning, you can probably get good performance out of
> either container. Jetty is smaller with a default configuration, but as
> others have pointed out, most of the resource utilization will be done
> by Solr, not the container.
>
> There is one major reason that I chose to use Jetty. It was already
> there in the Solr download. The reasons that I have stuck with it even
> after having time to research: It works well, and it is extensively
> tested every time anyone runs the tests that come with the Solr build
> system.
>
> Thanks,
> Shawn
>
>
>
Re: Update on shards
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/24/2013 12:49 AM, Arkadi Colson wrote:
> We are using tomcat so we'll just wait. Hopefully it's fixed in 4.3 but
> we have a work around for now so...
>
> What exactly is the difference between jetty and tomcat. We are using
> tomcat because we've read somewhere that it should be more robust in
> heavily loaded production environments.
Both are servlet containers - a Java executable server program that can
run other programs written using the Java Servlet API. The servlet API
was invented by Sun, who also invented Java itself.
For comparison purposes, first think about Apache's HTTPD, which is a
web server designed to serve files. Through its rich modular
capability, it does have the ability to run web applications, but the
core HTTPD is designed to grab a file off the hard drive and send it to
a user.
A servlet container is different. You can think of a servlet container
as a smart web server designed from the ground up to run web applications.
http://en.wikipedia.org/wiki/Java_Servlet
Solr is a servlet. By itself, Solr can't run. It requires a servlet
container.
Here is what wikipedia has to say about the histories of the two projects:
http://en.wikipedia.org/wiki/Apache_Tomcat#History
http://en.wikipedia.org/wiki/Jetty_%28Web_server%29#History
If you google "difference between jetty and tomcat" you'll find a lot of
links. The one written by Jetty folks is particularly detailed, but has
an obvious bias.
With emphasis on tuning, you can probably get good performance out of
either container. Jetty is smaller with a default configuration, but as
others have pointed out, most of the resource utilization will be done
by Solr, not the container.
There is one major reason that I chose to use Jetty. It was already
there in the Solr download. The reasons that I have stuck with it even
after having time to research: It works well, and it is extensively
tested every time anyone runs the tests that come with the Solr build
system.
Thanks,
Shawn
Re: Update on shards
Posted by Arkadi Colson <ar...@smartbit.be>.
We are using tomcat so we'll just wait. Hopefully it's fixed in 4.3 but
we have a work around for now so...
What exactly is the difference between jetty and tomcat. We are using
tomcat because we've read somewhere that it should be more robust in
heavily loaded production environments.
Arkadi
On 04/23/2013 06:14 PM, Mark Miller wrote:
> If you use jetty - which you should :) It's what we test with. Tomcat only gets user testing.
>
> If you use tomcat, this won't work in 4.2 or 4.2.1, but probably will in 4.3 (we are voting on 4.3 now).
>
> No clue on other containers.
>
> - Mark
>
> On Apr 23, 2013, at 10:59 AM, Michael Della Bitta <mi...@appinions.com> wrote:
>
>> I believe as of 4.2 you can talk to any host in the cloud.
>>
>> Michael Della Bitta
>>
>> ------------------------------------------------
>> Appinions
>> 18 East 41st Street, 2nd Floor
>> New York, NY 10017-6271
>>
>> www.appinions.com
>>
>> Where Influence Isn’t a Game
>>
>>
>> On Tue, Apr 23, 2013 at 10:45 AM, Arkadi Colson <ar...@smartbit.be> wrote:
>>> Hi
>>>
>>> Is it correct that when inserting or updating document into solr you have to
>>> talk to a solr host where at least one shard of that collection is stored?
>>> For select you can talk to any host within the collection.configName?
>>>
>>> BR,
>>> Arkadi
>
>
Re: Update on shards
Posted by Mark Miller <ma...@gmail.com>.
This request proxying only applies to the read side. The write side forwards updates around, it doesn't proxy requests.
- Mark
On Apr 23, 2013, at 2:33 PM, Furkan KAMACI <fu...@gmail.com> wrote:
> Sorry but I want to make clears the things in my mind. Is there any
> documentation that explains Solr proxying? Is it same thing with that: when
> I use SolrCloud and if I send document any of the nodes at my cluster the
> document will be routed into the leader of appropriate shard. So you mean I
> can not do that if I use Tomcat?
>
> 2013/4/23 Mark Miller <ma...@gmail.com>
>
>> The request proxying does not work with tomcat without calling an explicit
>> flush in the code - jetty (which the unit tests are written against) worked
>> without this flush. The flush is added to 4.3.
>>
>>
>> - Mark
>>
>> On Apr 23, 2013, at 2:02 PM, Furkan KAMACI <fu...@gmail.com> wrote:
>>
>>> Oopps, Mark you said: "If you use tomcat, this won't work in 4.2 or
>> 4.2.1"
>>>
>>> Can you explain more what won't be at Tomcat and what will change at 4.3?
>>>
>>> 2013/4/23 Mark Miller <ma...@gmail.com>
>>>
>>>> If you use jetty - which you should :) It's what we test with. Tomcat
>> only
>>>> gets user testing.
>>>>
>>>> If you use tomcat, this won't work in 4.2 or 4.2.1, but probably will in
>>>> 4.3 (we are voting on 4.3 now).
>>>>
>>>> No clue on other containers.
>>>>
>>>> - Mark
>>>>
>>>> On Apr 23, 2013, at 10:59 AM, Michael Della Bitta <
>>>> michael.della.bitta@appinions.com> wrote:
>>>>
>>>>> I believe as of 4.2 you can talk to any host in the cloud.
>>>>>
>>>>> Michael Della Bitta
>>>>>
>>>>> ------------------------------------------------
>>>>> Appinions
>>>>> 18 East 41st Street, 2nd Floor
>>>>> New York, NY 10017-6271
>>>>>
>>>>> www.appinions.com
>>>>>
>>>>> Where Influence Isn’t a Game
>>>>>
>>>>>
>>>>> On Tue, Apr 23, 2013 at 10:45 AM, Arkadi Colson <ar...@smartbit.be>
>>>> wrote:
>>>>>> Hi
>>>>>>
>>>>>> Is it correct that when inserting or updating document into solr you
>>>> have to
>>>>>> talk to a solr host where at least one shard of that collection is
>>>> stored?
>>>>>> For select you can talk to any host within the collection.configName?
>>>>>>
>>>>>> BR,
>>>>>> Arkadi
>>>>
>>>>
>>
>>
Re: Update on shards
Posted by Jack Krupansky <ja...@basetechnology.com>.
Yeah, I'm confused now too. Do all Solr nodes in a distributed cloud really
have to run in the same container type?? Why isn't it just raw HTTP for one
cloud no to talk to another? I mean each node could/should be on another
machine, right?
-- Jack Krupansky
-----Original Message-----
From: Furkan KAMACI
Sent: Tuesday, April 23, 2013 2:33 PM
To: solr-user@lucene.apache.org
Subject: Re: Update on shards
Sorry but I want to make clears the things in my mind. Is there any
documentation that explains Solr proxying? Is it same thing with that: when
I use SolrCloud and if I send document any of the nodes at my cluster the
document will be routed into the leader of appropriate shard. So you mean I
can not do that if I use Tomcat?
2013/4/23 Mark Miller <ma...@gmail.com>
> The request proxying does not work with tomcat without calling an explicit
> flush in the code - jetty (which the unit tests are written against)
> worked
> without this flush. The flush is added to 4.3.
>
>
> - Mark
>
> On Apr 23, 2013, at 2:02 PM, Furkan KAMACI <fu...@gmail.com> wrote:
>
> > Oopps, Mark you said: "If you use tomcat, this won't work in 4.2 or
> 4.2.1"
> >
> > Can you explain more what won't be at Tomcat and what will change at
> > 4.3?
> >
> > 2013/4/23 Mark Miller <ma...@gmail.com>
> >
> >> If you use jetty - which you should :) It's what we test with. Tomcat
> only
> >> gets user testing.
> >>
> >> If you use tomcat, this won't work in 4.2 or 4.2.1, but probably will
> >> in
> >> 4.3 (we are voting on 4.3 now).
> >>
> >> No clue on other containers.
> >>
> >> - Mark
> >>
> >> On Apr 23, 2013, at 10:59 AM, Michael Della Bitta <
> >> michael.della.bitta@appinions.com> wrote:
> >>
> >>> I believe as of 4.2 you can talk to any host in the cloud.
> >>>
> >>> Michael Della Bitta
> >>>
> >>> ------------------------------------------------
> >>> Appinions
> >>> 18 East 41st Street, 2nd Floor
> >>> New York, NY 10017-6271
> >>>
> >>> www.appinions.com
> >>>
> >>> Where Influence Isn’t a Game
> >>>
> >>>
> >>> On Tue, Apr 23, 2013 at 10:45 AM, Arkadi Colson <ar...@smartbit.be>
> >> wrote:
> >>>> Hi
> >>>>
> >>>> Is it correct that when inserting or updating document into solr you
> >> have to
> >>>> talk to a solr host where at least one shard of that collection is
> >> stored?
> >>>> For select you can talk to any host within the collection.configName?
> >>>>
> >>>> BR,
> >>>> Arkadi
> >>
> >>
>
>
Re: Update on shards
Posted by Furkan KAMACI <fu...@gmail.com>.
Sorry but I want to make clears the things in my mind. Is there any
documentation that explains Solr proxying? Is it same thing with that: when
I use SolrCloud and if I send document any of the nodes at my cluster the
document will be routed into the leader of appropriate shard. So you mean I
can not do that if I use Tomcat?
2013/4/23 Mark Miller <ma...@gmail.com>
> The request proxying does not work with tomcat without calling an explicit
> flush in the code - jetty (which the unit tests are written against) worked
> without this flush. The flush is added to 4.3.
>
>
> - Mark
>
> On Apr 23, 2013, at 2:02 PM, Furkan KAMACI <fu...@gmail.com> wrote:
>
> > Oopps, Mark you said: "If you use tomcat, this won't work in 4.2 or
> 4.2.1"
> >
> > Can you explain more what won't be at Tomcat and what will change at 4.3?
> >
> > 2013/4/23 Mark Miller <ma...@gmail.com>
> >
> >> If you use jetty - which you should :) It's what we test with. Tomcat
> only
> >> gets user testing.
> >>
> >> If you use tomcat, this won't work in 4.2 or 4.2.1, but probably will in
> >> 4.3 (we are voting on 4.3 now).
> >>
> >> No clue on other containers.
> >>
> >> - Mark
> >>
> >> On Apr 23, 2013, at 10:59 AM, Michael Della Bitta <
> >> michael.della.bitta@appinions.com> wrote:
> >>
> >>> I believe as of 4.2 you can talk to any host in the cloud.
> >>>
> >>> Michael Della Bitta
> >>>
> >>> ------------------------------------------------
> >>> Appinions
> >>> 18 East 41st Street, 2nd Floor
> >>> New York, NY 10017-6271
> >>>
> >>> www.appinions.com
> >>>
> >>> Where Influence Isn’t a Game
> >>>
> >>>
> >>> On Tue, Apr 23, 2013 at 10:45 AM, Arkadi Colson <ar...@smartbit.be>
> >> wrote:
> >>>> Hi
> >>>>
> >>>> Is it correct that when inserting or updating document into solr you
> >> have to
> >>>> talk to a solr host where at least one shard of that collection is
> >> stored?
> >>>> For select you can talk to any host within the collection.configName?
> >>>>
> >>>> BR,
> >>>> Arkadi
> >>
> >>
>
>
Re: Update on shards
Posted by Mark Miller <ma...@gmail.com>.
The request proxying does not work with tomcat without calling an explicit flush in the code - jetty (which the unit tests are written against) worked without this flush. The flush is added to 4.3.
- Mark
On Apr 23, 2013, at 2:02 PM, Furkan KAMACI <fu...@gmail.com> wrote:
> Oopps, Mark you said: "If you use tomcat, this won't work in 4.2 or 4.2.1"
>
> Can you explain more what won't be at Tomcat and what will change at 4.3?
>
> 2013/4/23 Mark Miller <ma...@gmail.com>
>
>> If you use jetty - which you should :) It's what we test with. Tomcat only
>> gets user testing.
>>
>> If you use tomcat, this won't work in 4.2 or 4.2.1, but probably will in
>> 4.3 (we are voting on 4.3 now).
>>
>> No clue on other containers.
>>
>> - Mark
>>
>> On Apr 23, 2013, at 10:59 AM, Michael Della Bitta <
>> michael.della.bitta@appinions.com> wrote:
>>
>>> I believe as of 4.2 you can talk to any host in the cloud.
>>>
>>> Michael Della Bitta
>>>
>>> ------------------------------------------------
>>> Appinions
>>> 18 East 41st Street, 2nd Floor
>>> New York, NY 10017-6271
>>>
>>> www.appinions.com
>>>
>>> Where Influence Isn’t a Game
>>>
>>>
>>> On Tue, Apr 23, 2013 at 10:45 AM, Arkadi Colson <ar...@smartbit.be>
>> wrote:
>>>> Hi
>>>>
>>>> Is it correct that when inserting or updating document into solr you
>> have to
>>>> talk to a solr host where at least one shard of that collection is
>> stored?
>>>> For select you can talk to any host within the collection.configName?
>>>>
>>>> BR,
>>>> Arkadi
>>
>>
Re: Update on shards
Posted by Furkan KAMACI <fu...@gmail.com>.
Oopps, Mark you said: "If you use tomcat, this won't work in 4.2 or 4.2.1"
Can you explain more what won't be at Tomcat and what will change at 4.3?
2013/4/23 Mark Miller <ma...@gmail.com>
> If you use jetty - which you should :) It's what we test with. Tomcat only
> gets user testing.
>
> If you use tomcat, this won't work in 4.2 or 4.2.1, but probably will in
> 4.3 (we are voting on 4.3 now).
>
> No clue on other containers.
>
> - Mark
>
> On Apr 23, 2013, at 10:59 AM, Michael Della Bitta <
> michael.della.bitta@appinions.com> wrote:
>
> > I believe as of 4.2 you can talk to any host in the cloud.
> >
> > Michael Della Bitta
> >
> > ------------------------------------------------
> > Appinions
> > 18 East 41st Street, 2nd Floor
> > New York, NY 10017-6271
> >
> > www.appinions.com
> >
> > Where Influence Isn’t a Game
> >
> >
> > On Tue, Apr 23, 2013 at 10:45 AM, Arkadi Colson <ar...@smartbit.be>
> wrote:
> >> Hi
> >>
> >> Is it correct that when inserting or updating document into solr you
> have to
> >> talk to a solr host where at least one shard of that collection is
> stored?
> >> For select you can talk to any host within the collection.configName?
> >>
> >> BR,
> >> Arkadi
>
>
Re: Update on shards
Posted by Mark Miller <ma...@gmail.com>.
If you use jetty - which you should :) It's what we test with. Tomcat only gets user testing.
If you use tomcat, this won't work in 4.2 or 4.2.1, but probably will in 4.3 (we are voting on 4.3 now).
No clue on other containers.
- Mark
On Apr 23, 2013, at 10:59 AM, Michael Della Bitta <mi...@appinions.com> wrote:
> I believe as of 4.2 you can talk to any host in the cloud.
>
> Michael Della Bitta
>
> ------------------------------------------------
> Appinions
> 18 East 41st Street, 2nd Floor
> New York, NY 10017-6271
>
> www.appinions.com
>
> Where Influence Isn’t a Game
>
>
> On Tue, Apr 23, 2013 at 10:45 AM, Arkadi Colson <ar...@smartbit.be> wrote:
>> Hi
>>
>> Is it correct that when inserting or updating document into solr you have to
>> talk to a solr host where at least one shard of that collection is stored?
>> For select you can talk to any host within the collection.configName?
>>
>> BR,
>> Arkadi
Re: Update on shards
Posted by Michael Della Bitta <mi...@appinions.com>.
I believe as of 4.2 you can talk to any host in the cloud.
Michael Della Bitta
------------------------------------------------
Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271
www.appinions.com
Where Influence Isn’t a Game
On Tue, Apr 23, 2013 at 10:45 AM, Arkadi Colson <ar...@smartbit.be> wrote:
> Hi
>
> Is it correct that when inserting or updating document into solr you have to
> talk to a solr host where at least one shard of that collection is stored?
> For select you can talk to any host within the collection.configName?
>
> BR,
> Arkadi