You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Salmaan Rashid Syed <sa...@mroads.com> on 2019/05/03 05:54:04 UTC
Accessing Solr collections at different ports - Need help
Hi Solr Users,
I am using Solr 7.6 in cloud mode with external zookeeper installed at
ports 2181, 2182, 2183. Currently we have only one server allocated for
Solr. We are planning to move to multiple servers for better sharing,
replication etc in near future.
Now the issue is that, our organisation has data indexed for different
clients as separate collections. We want to uniquely access, update and
index each collection separately so that each individual client has access
to their respective collections at their respective ports. Eg:— Collection1
at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
I have two options I guess, one is to run Solr in cloud mode with 4 nodes
(max as limited by Solr) at 4 different ports. I don’t know how to go
beyond 4 nodes/ports in this case.
The other option is to run Solr as service and create multiple copies of
Solr folder within the Server folder and access each Solr at different port
with its own collection as shown by
https://www.youtube.com/watch?v=wmQFwK2sujE
I am really confused as to which is the better path to choose. Please help
me out.
Thanks.
Regards,
Salmaan
*Thanks and Regards,*
Salmaan Rashid Syed
+91 8978353445 | www.panna.ai |
5550 Granite Pkwy, Suite #225, Plano TX-75024.
Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
Re: Accessing Solr collections at different ports - Need help
Posted by Salmaan Rashid Syed <sa...@mroads.com>.
Thanks Walter,
Since I am new to Solr and by looking at your suggestion, it looks like I
am trying to do something very complicated and out-of-box capabilities of
Solr. I really don't want to do that.
I am not from Computer Science background and my specialisation is in
Analytics and AI.
Let me put my case scenario briefly.
We have developed a customised Solr-search engine that can search for data
(prepared, cleaned and preprocessed by us) in each individual Solr
collection.
Every client of ours is from a different vertical (like health,
engineering, public services, finance, casual works etc). They search for
data in their respective Solr collection. They also add, update and
re-index their respective data periodically.
As suggested by you, if I port-out all the collections from a single port,
will not the latency increase, wil not the burden on a single server
increase, will not the computational speed slows down as all the clients
are trying to speak to the same port simultaneously.
Or do you think that Solr-as-service is better option, where I can create
multiple Solr instances at different ports with collections of individual
clients in each solr instance.
To be honest, I really don't know what Solr-as-service is really trying to
accomplish.
Apologies for lengthy question and Thanks in advance.
*Thanks and Regards,*
Salmaan Rashid Syed
+91 8978353445 | www.panna.ai |
5550 Granite Pkwy, Suite #225, Plano TX-75024.
Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
On Fri, May 3, 2019 at 11:59 AM Walter Underwood <wu...@wunderwood.org>
wrote:
> The best option is to run all the collections at the same port.
> Intra-cluster communication cannot be split over multiple ports, so this
> would require big internal changes to Solr. And what about communication
> that does not belong to a collection, like electing an overseer node?
>
> Why do you want the very non-standard configuration?
>
> If you must have it, run a webserver like nginx on each node, configure it
> to do this crazy multiple port thing for external traffic and to forward
> all traffic to Solr’s single port.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
> > On May 3, 2019, at 7:54 AM, Salmaan Rashid Syed <
> salmaan.rashid@mroads.com> wrote:
> >
> > Hi Solr Users,
> >
> > I am using Solr 7.6 in cloud mode with external zookeeper installed at
> > ports 2181, 2182, 2183. Currently we have only one server allocated for
> > Solr. We are planning to move to multiple servers for better sharing,
> > replication etc in near future.
> >
> > Now the issue is that, our organisation has data indexed for different
> > clients as separate collections. We want to uniquely access, update and
> > index each collection separately so that each individual client has
> access
> > to their respective collections at their respective ports. Eg:—
> Collection1
> > at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
> >
> > I have two options I guess, one is to run Solr in cloud mode with 4 nodes
> > (max as limited by Solr) at 4 different ports. I don’t know how to go
> > beyond 4 nodes/ports in this case.
> >
> > The other option is to run Solr as service and create multiple copies of
> > Solr folder within the Server folder and access each Solr at different
> port
> > with its own collection as shown by
> > https://www.youtube.com/watch?v=wmQFwK2sujE
> >
> > I am really confused as to which is the better path to choose. Please
> help
> > me out.
> >
> > Thanks.
> >
> > Regards,
> > Salmaan
> >
> >
> > *Thanks and Regards,*
> > Salmaan Rashid Syed
> > +91 8978353445 | www.panna.ai |
> > 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> > Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
>
>
Re: Accessing Solr collections at different ports - Need help
Posted by Walter Underwood <wu...@wunderwood.org>.
The best option is to run all the collections at the same port. Intra-cluster communication cannot be split over multiple ports, so this would require big internal changes to Solr. And what about communication that does not belong to a collection, like electing an overseer node?
Why do you want the very non-standard configuration?
If you must have it, run a webserver like nginx on each node, configure it to do this crazy multiple port thing for external traffic and to forward all traffic to Solr’s single port.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On May 3, 2019, at 7:54 AM, Salmaan Rashid Syed <sa...@mroads.com> wrote:
>
> Hi Solr Users,
>
> I am using Solr 7.6 in cloud mode with external zookeeper installed at
> ports 2181, 2182, 2183. Currently we have only one server allocated for
> Solr. We are planning to move to multiple servers for better sharing,
> replication etc in near future.
>
> Now the issue is that, our organisation has data indexed for different
> clients as separate collections. We want to uniquely access, update and
> index each collection separately so that each individual client has access
> to their respective collections at their respective ports. Eg:— Collection1
> at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
>
> I have two options I guess, one is to run Solr in cloud mode with 4 nodes
> (max as limited by Solr) at 4 different ports. I don’t know how to go
> beyond 4 nodes/ports in this case.
>
> The other option is to run Solr as service and create multiple copies of
> Solr folder within the Server folder and access each Solr at different port
> with its own collection as shown by
> https://www.youtube.com/watch?v=wmQFwK2sujE
>
> I am really confused as to which is the better path to choose. Please help
> me out.
>
> Thanks.
>
> Regards,
> Salmaan
>
>
> *Thanks and Regards,*
> Salmaan Rashid Syed
> +91 8978353445 | www.panna.ai |
> 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
Re: Accessing Solr collections at different ports - Need help
Posted by Jörn Franke <jo...@gmail.com>.
You can have dedicarse clusters per Client and/or you can protect it via Kerberos or Basic Auth or write your own authorization plugin based on OAuth.
I am not sure why you want to offer this on different ports to different clients.
> Am 03.05.2019 um 07:54 schrieb Salmaan Rashid Syed <sa...@mroads.com>:
>
> Hi Solr Users,
>
> I am using Solr 7.6 in cloud mode with external zookeeper installed at
> ports 2181, 2182, 2183. Currently we have only one server allocated for
> Solr. We are planning to move to multiple servers for better sharing,
> replication etc in near future.
>
> Now the issue is that, our organisation has data indexed for different
> clients as separate collections. We want to uniquely access, update and
> index each collection separately so that each individual client has access
> to their respective collections at their respective ports. Eg:— Collection1
> at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
>
> I have two options I guess, one is to run Solr in cloud mode with 4 nodes
> (max as limited by Solr) at 4 different ports. I don’t know how to go
> beyond 4 nodes/ports in this case.
>
> The other option is to run Solr as service and create multiple copies of
> Solr folder within the Server folder and access each Solr at different port
> with its own collection as shown by
> https://www.youtube.com/watch?v=wmQFwK2sujE
>
> I am really confused as to which is the better path to choose. Please help
> me out.
>
> Thanks.
>
> Regards,
> Salmaan
>
>
> *Thanks and Regards,*
> Salmaan Rashid Syed
> +91 8978353445 | www.panna.ai |
> 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
Re: Accessing Solr collections at different ports - Need help
Posted by Jörn Franke <jo...@gmail.com>.
This is just the setup for an experimental cluster (generally it does also not make sense to have many instances on the same server). Once you have got more experience take a look at https://lucene.apache.org/solr/guide/7_7/taking-solr-to-production.html
To see how to set up clusters.
> Am 03.05.2019 um 08:52 schrieb Salmaan Rashid Syed <sa...@mroads.com>:
>
> Thanks Jorn for your reply.
>
> I say that the nodes are limited to 4 because when I launch Solr in cloud
> mode, the first prompt that I get is to choose number of nodes [1-4]. When
> I tried to enter 7, it says that they are more than 4 and choose a smaller
> number.
>
>
> *Thanks and Regards,*
> Salmaan Rashid Syed
> +91 8978353445 | www.panna.ai |
> 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
>
>
>
>> On Fri, May 3, 2019 at 12:05 PM Jörn Franke <jo...@gmail.com> wrote:
>>
>> BTW why do you think that SolrCloud is limited to 4 nodes? More are for
>> sure possible.
>>
>>> Am 03.05.2019 um 07:54 schrieb Salmaan Rashid Syed <
>> salmaan.rashid@mroads.com>:
>>>
>>> Hi Solr Users,
>>>
>>> I am using Solr 7.6 in cloud mode with external zookeeper installed at
>>> ports 2181, 2182, 2183. Currently we have only one server allocated for
>>> Solr. We are planning to move to multiple servers for better sharing,
>>> replication etc in near future.
>>>
>>> Now the issue is that, our organisation has data indexed for different
>>> clients as separate collections. We want to uniquely access, update and
>>> index each collection separately so that each individual client has
>> access
>>> to their respective collections at their respective ports. Eg:—
>> Collection1
>>> at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
>>>
>>> I have two options I guess, one is to run Solr in cloud mode with 4 nodes
>>> (max as limited by Solr) at 4 different ports. I don’t know how to go
>>> beyond 4 nodes/ports in this case.
>>>
>>> The other option is to run Solr as service and create multiple copies of
>>> Solr folder within the Server folder and access each Solr at different
>> port
>>> with its own collection as shown by
>>> https://www.youtube.com/watch?v=wmQFwK2sujE
>>>
>>> I am really confused as to which is the better path to choose. Please
>> help
>>> me out.
>>>
>>> Thanks.
>>>
>>> Regards,
>>> Salmaan
>>>
>>>
>>> *Thanks and Regards,*
>>> Salmaan Rashid Syed
>>> +91 8978353445 | www.panna.ai |
>>> 5550 Granite Pkwy, Suite #225, Plano TX-75024.
>>> Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
>>
Re: Accessing Solr collections at different ports - Need help
Posted by Shawn Heisey <ap...@elyograg.org>.
On 5/3/2019 12:52 AM, Salmaan Rashid Syed wrote:
> I say that the nodes are limited to 4 because when I launch Solr in cloud
> mode, the first prompt that I get is to choose number of nodes [1-4]. When
> I tried to enter 7, it says that they are more than 4 and choose a smaller
> number.
That's the cloud *EXAMPLE*. It sets everything up on one server that
would normally be on separate servers, and runs an embedded zookeeper in
the first node.
Example setups are not meant for production.
Thanks,
Shawn
Re: Accessing Solr collections at different ports - Need help
Posted by Salmaan Rashid Syed <sa...@mroads.com>.
Thanks Jorn for your reply.
I say that the nodes are limited to 4 because when I launch Solr in cloud
mode, the first prompt that I get is to choose number of nodes [1-4]. When
I tried to enter 7, it says that they are more than 4 and choose a smaller
number.
*Thanks and Regards,*
Salmaan Rashid Syed
+91 8978353445 | www.panna.ai |
5550 Granite Pkwy, Suite #225, Plano TX-75024.
Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
On Fri, May 3, 2019 at 12:05 PM Jörn Franke <jo...@gmail.com> wrote:
> BTW why do you think that SolrCloud is limited to 4 nodes? More are for
> sure possible.
>
> > Am 03.05.2019 um 07:54 schrieb Salmaan Rashid Syed <
> salmaan.rashid@mroads.com>:
> >
> > Hi Solr Users,
> >
> > I am using Solr 7.6 in cloud mode with external zookeeper installed at
> > ports 2181, 2182, 2183. Currently we have only one server allocated for
> > Solr. We are planning to move to multiple servers for better sharing,
> > replication etc in near future.
> >
> > Now the issue is that, our organisation has data indexed for different
> > clients as separate collections. We want to uniquely access, update and
> > index each collection separately so that each individual client has
> access
> > to their respective collections at their respective ports. Eg:—
> Collection1
> > at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
> >
> > I have two options I guess, one is to run Solr in cloud mode with 4 nodes
> > (max as limited by Solr) at 4 different ports. I don’t know how to go
> > beyond 4 nodes/ports in this case.
> >
> > The other option is to run Solr as service and create multiple copies of
> > Solr folder within the Server folder and access each Solr at different
> port
> > with its own collection as shown by
> > https://www.youtube.com/watch?v=wmQFwK2sujE
> >
> > I am really confused as to which is the better path to choose. Please
> help
> > me out.
> >
> > Thanks.
> >
> > Regards,
> > Salmaan
> >
> >
> > *Thanks and Regards,*
> > Salmaan Rashid Syed
> > +91 8978353445 | www.panna.ai |
> > 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> > Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
>
Re: Accessing Solr collections at different ports - Need help
Posted by Jörn Franke <jo...@gmail.com>.
BTW why do you think that SolrCloud is limited to 4 nodes? More are for sure possible.
> Am 03.05.2019 um 07:54 schrieb Salmaan Rashid Syed <sa...@mroads.com>:
>
> Hi Solr Users,
>
> I am using Solr 7.6 in cloud mode with external zookeeper installed at
> ports 2181, 2182, 2183. Currently we have only one server allocated for
> Solr. We are planning to move to multiple servers for better sharing,
> replication etc in near future.
>
> Now the issue is that, our organisation has data indexed for different
> clients as separate collections. We want to uniquely access, update and
> index each collection separately so that each individual client has access
> to their respective collections at their respective ports. Eg:— Collection1
> at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
>
> I have two options I guess, one is to run Solr in cloud mode with 4 nodes
> (max as limited by Solr) at 4 different ports. I don’t know how to go
> beyond 4 nodes/ports in this case.
>
> The other option is to run Solr as service and create multiple copies of
> Solr folder within the Server folder and access each Solr at different port
> with its own collection as shown by
> https://www.youtube.com/watch?v=wmQFwK2sujE
>
> I am really confused as to which is the better path to choose. Please help
> me out.
>
> Thanks.
>
> Regards,
> Salmaan
>
>
> *Thanks and Regards,*
> Salmaan Rashid Syed
> +91 8978353445 | www.panna.ai |
> 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.