You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Manoj Jain <ma...@hcl.com.INVALID> on 2023/05/30 18:29:07 UTC
Issue in connecting zookeeper host from node js service
Classification: Confidential
Hello,
We are migrating from Solr-standalone to Solr-cloud of version 8.11 using external zookeeper. We have the node and java microservices which are connected to Solr for indexing and query and everything is deployed within same GKE cluster.
For the migration of solr cloud we have the task to modify the connection string so that it can point to Solr Cloud via zookeeper. For the java services we use 'CloudSolrClient' class of SolrJ library to connect to the zookeeper host and it worked well. And for the node service we are using 'Solr-client' library, but we are not able to connect to the zookeeper host using the same. For testing purpose, we give solr-cluster I.P. and port instead after which the service can connect but the same failed when we try to connect to zookeeper host (neither from DNS name or I.P. and port). We do not see any other specific solr library for node js to connect to Solr Cloud. Can anyone please suggest how we can connect to zookeeper host from node js service?
Regards
Manoj Jain
::DISCLAIMER::
________________________________
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
________________________________
RE: Issue in connecting zookeeper host from node js service
Posted by Manoj Jain <ma...@hcl.com.INVALID>.
Classification: Public
Hi Ishan,
Sorry for the confusion. I was using my organization outlook client that may added the default footnote and classification. I confirm that it is not confidential and seeking for the feedback and inputs for the problem I am facing. Thanks for your input.
Regards
Manoj Jain
-----Original Message-----
From: Ishan Chattopadhyaya <ic...@gmail.com>
Sent: Wednesday, May 31, 2023 1:19 PM
To: manoj.jain@hcl.com.invalid
Cc: users@solr.apache.org
Subject: Re: Issue in connecting zookeeper host from node js service
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don’t click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
Hi Manoj,
Your message body contains a footnote
"Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. "
It is also classified as confidential. Can you confirm that this message is not confidential?
Thanks,
Ishan Chattopadhyaya
On Wed, 31 May, 2023, 12:00 am Manoj Jain, <ma...@hcl.com.invalid>
wrote:
> Classification: Confidential
> Hello,
>
> We are migrating from Solr-standalone to Solr-cloud of version 8.11
> using external zookeeper. We have the node and java microservices
> which are connected to Solr for indexing and query and everything is
> deployed within same GKE cluster.
>
> For the migration of solr cloud we have the task to modify the
> connection string so that it can point to Solr Cloud via zookeeper.
> For the java services we use 'CloudSolrClient' class of SolrJ library
> to connect to the zookeeper host and it worked well. And for the node
> service we are using 'Solr-client' library, but we are not able to
> connect to the zookeeper host using the same. For testing purpose, we
> give solr-cluster I.P. and port instead after which the service can
> connect but the same failed when we try to connect to zookeeper host
> (neither from DNS name or I.P. and port). We do not see any other
> specific solr library for node js to connect to Solr Cloud. Can anyone
> please suggest how we can connect to zookeeper host from node js service?
>
> Regards
> Manoj Jain
>
::DISCLAIMER::
________________________________
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
________________________________
Re: Issue in connecting zookeeper host from node js service
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
Hi Manoj,
Your message body contains a footnote
"Any form of reproduction, dissemination, copying, disclosure,
modification, distribution and / or publication of this message without the
prior written consent of authorized representative of HCL is strictly
prohibited. "
It is also classified as confidential. Can you confirm that this message is
not confidential?
Thanks,
Ishan Chattopadhyaya
On Wed, 31 May, 2023, 12:00 am Manoj Jain, <ma...@hcl.com.invalid>
wrote:
> Classification: Confidential
> Hello,
>
> We are migrating from Solr-standalone to Solr-cloud of version 8.11 using
> external zookeeper. We have the node and java microservices which are
> connected to Solr for indexing and query and everything is deployed within
> same GKE cluster.
>
> For the migration of solr cloud we have the task to modify the connection
> string so that it can point to Solr Cloud via zookeeper. For the java
> services we use 'CloudSolrClient' class of SolrJ library to connect to the
> zookeeper host and it worked well. And for the node service we are using
> 'Solr-client' library, but we are not able to connect to the zookeeper host
> using the same. For testing purpose, we give solr-cluster I.P. and port
> instead after which the service can connect but the same failed when we try
> to connect to zookeeper host (neither from DNS name or I.P. and port). We
> do not see any other specific solr library for node js to connect to Solr
> Cloud. Can anyone please suggest how we can connect to zookeeper host from
> node js service?
>
> Regards
> Manoj Jain
>
> ::DISCLAIMER::
> ________________________________
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only. E-mail transmission is not
> guaranteed to be secure or error-free as information could be intercepted,
> corrupted, lost, destroyed, arrive late or incomplete, or may contain
> viruses in transmission. The e mail and its contents (with or without
> referred errors) shall therefore not attach any liability on the originator
> or HCL or its affiliates. Views or opinions, if any, presented in this
> email are solely those of the author and may not necessarily reflect the
> views or opinions of HCL or its affiliates. Any form of reproduction,
> dissemination, copying, disclosure, modification, distribution and / or
> publication of this message without the prior written consent of authorized
> representative of HCL is strictly prohibited. If you have received this
> email in error please delete it and notify the sender immediately. Before
> opening any email and/or attachments, please check them for viruses and
> other defects.
> ________________________________
>
Re: Issue in connecting zookeeper host from node js service
Posted by Jan Høydahl <ja...@cominvent.com>.
Hi,
It is safe to talk to any of the Solr nodes directly. Inside of GKE cluster, you will have a Service name for Solr that clients can connect to, and it will only forward traffic to healthy nodes. Besides, if a replica on a certain POD is e.g. in recovery mode, requests sent to that node will be re-routed by Solr to another live replica.
If using Solr-Operator project, there will already be a Service setup for you that forwards traffic to the StatefulSet with all Solr PODs. No need for client ot talk to Zookeeper.
Jan
> 31. mai 2023 kl. 12:52 skrev Manoj Jain <ma...@hcl.com.INVALID>:
>
> Classification: Public
>
> Thanks for the quick response. As far as ZK connection string is concerned it is valid and accessible and with the same I can successfully connect to the java service (via Solrj library - CloudSolrCleint)
>
> We are using external zookeeper in our solr cloud setup in Kubernetes. Our headless zookeeper internally has three zookeeper pods (to make a quorum) which stores the configuration data for solr in a hierarchical data structure (Znode tree). Further solr cluster have three solr pods. The reason we want to connect to ZK instead of Solr cluster directly because(as per my understanding) its Zk only which is aware of the true state of Solr cluster. If we directly connect to Solr cluster instead of going through the ZK then I believe in production there are chances to encounter some weird scenarios, for instance the request may go to a Solr node which may not have the latest data.
>
> Please share your feedback/inputs on the same.
>
> Regards
> Manoj Jain
>
> -----Original Message-----
> From: Ishan Chattopadhyaya <ic...@gmail.com>
> Sent: Wednesday, May 31, 2023 1:56 PM
> To: manoj.jain@hcl.com.invalid
> Cc: users@solr.apache.org
> Subject: Re: Issue in connecting zookeeper host from node js service
>
> [CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don’t click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
>
> If you're using Solr in Kubernetes, you can continue connecting to Solr pods without connecting to ZK.
>
> We don't have any officially supported Solr client for NodeJS, but the HTTP endpoints of Solr are supported and can be used directly.
>
> If you're having trouble connecting to ZK via a third party (unofficial) Solr client in NodeJS, maybe you can reach out to the support for that client library. Also, you can make sure the ZK connection string you're using is valid and accessible from outside the Kubernetes pods.
>
> On Wed, 31 May, 2023, 12:00 am Manoj Jain, <ma...@hcl.com.invalid>
> wrote:
>
>> Classification: Confidential
>> Hello,
>>
>> We are migrating from Solr-standalone to Solr-cloud of version 8.11
>> using external zookeeper. We have the node and java microservices
>> which are connected to Solr for indexing and query and everything is
>> deployed within same GKE cluster.
>>
>> For the migration of solr cloud we have the task to modify the
>> connection string so that it can point to Solr Cloud via zookeeper.
>> For the java services we use 'CloudSolrClient' class of SolrJ library
>> to connect to the zookeeper host and it worked well. And for the node
>> service we are using 'Solr-client' library, but we are not able to
>> connect to the zookeeper host using the same. For testing purpose, we
>> give solr-cluster I.P. and port instead after which the service can
>> connect but the same failed when we try to connect to zookeeper host
>> (neither from DNS name or I.P. and port). We do not see any other
>> specific solr library for node js to connect to Solr Cloud. Can anyone
>> please suggest how we can connect to zookeeper host from node js service?
>>
>> Regards
>> Manoj Jain
>>
>> ::DISCLAIMER::
>> ________________________________
>> The contents of this e-mail and any attachment(s) are confidential and
>> intended for the named recipient(s) only. E-mail transmission is not
>> guaranteed to be secure or error-free as information could be
>> intercepted, corrupted, lost, destroyed, arrive late or incomplete, or
>> may contain viruses in transmission. The e mail and its contents (with
>> or without referred errors) shall therefore not attach any liability
>> on the originator or HCL or its affiliates. Views or opinions, if any,
>> presented in this email are solely those of the author and may not
>> necessarily reflect the views or opinions of HCL or its affiliates.
>> Any form of reproduction, dissemination, copying, disclosure,
>> modification, distribution and / or publication of this message
>> without the prior written consent of authorized representative of HCL
>> is strictly prohibited. If you have received this email in error
>> please delete it and notify the sender immediately. Before opening any
>> email and/or attachments, please check them for viruses and other defects.
>> ________________________________
>>
RE: Issue in connecting zookeeper host from node js service
Posted by Manoj Jain <ma...@hcl.com.INVALID>.
Classification: Public
It would be great If anyone can help on this. Thanks for the assistance.
Regards
Manoj Jain
-----Original Message-----
From: Manoj Jain
Sent: Wednesday, May 31, 2023 4:22 PM
To: users@solr.apache.org; Ishan Chattopadhyaya <ic...@gmail.com>
Subject: RE: Issue in connecting zookeeper host from node js service
Classification: Public
Thanks for the quick response. As far as ZK connection string is concerned it is valid and accessible and with the same I can successfully connect to the java service (via Solrj library - CloudSolrCleint)
We are using external zookeeper in our solr cloud setup in Kubernetes. Our headless zookeeper internally has three zookeeper pods (to make a quorum) which stores the configuration data for solr in a hierarchical data structure (Znode tree). Further solr cluster have three solr pods. The reason we want to connect to ZK instead of Solr cluster directly because(as per my understanding) its Zk only which is aware of the true state of Solr cluster. If we directly connect to Solr cluster instead of going through the ZK then I believe in production there are chances to encounter some weird scenarios, for instance the request may go to a Solr node which may not have the latest data.
Please share your feedback/inputs on the same.
Regards
Manoj Jain
-----Original Message-----
From: Ishan Chattopadhyaya <ic...@gmail.com>
Sent: Wednesday, May 31, 2023 1:56 PM
To: manoj.jain@hcl.com.invalid
Cc: users@solr.apache.org
Subject: Re: Issue in connecting zookeeper host from node js service
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don’t click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
If you're using Solr in Kubernetes, you can continue connecting to Solr pods without connecting to ZK.
We don't have any officially supported Solr client for NodeJS, but the HTTP endpoints of Solr are supported and can be used directly.
If you're having trouble connecting to ZK via a third party (unofficial) Solr client in NodeJS, maybe you can reach out to the support for that client library. Also, you can make sure the ZK connection string you're using is valid and accessible from outside the Kubernetes pods.
On Wed, 31 May, 2023, 12:00 am Manoj Jain, <ma...@hcl.com.invalid>
wrote:
> Classification: Confidential
> Hello,
>
> We are migrating from Solr-standalone to Solr-cloud of version 8.11
> using external zookeeper. We have the node and java microservices
> which are connected to Solr for indexing and query and everything is
> deployed within same GKE cluster.
>
> For the migration of solr cloud we have the task to modify the
> connection string so that it can point to Solr Cloud via zookeeper.
> For the java services we use 'CloudSolrClient' class of SolrJ library
> to connect to the zookeeper host and it worked well. And for the node
> service we are using 'Solr-client' library, but we are not able to
> connect to the zookeeper host using the same. For testing purpose, we
> give solr-cluster I.P. and port instead after which the service can
> connect but the same failed when we try to connect to zookeeper host
> (neither from DNS name or I.P. and port). We do not see any other
> specific solr library for node js to connect to Solr Cloud. Can anyone
> please suggest how we can connect to zookeeper host from node js service?
>
> Regards
> Manoj Jain
>
> ::DISCLAIMER::
> ________________________________
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only. E-mail transmission is not
> guaranteed to be secure or error-free as information could be
> intercepted, corrupted, lost, destroyed, arrive late or incomplete, or
> may contain viruses in transmission. The e mail and its contents (with
> or without referred errors) shall therefore not attach any liability
> on the originator or HCL or its affiliates. Views or opinions, if any,
> presented in this email are solely those of the author and may not
> necessarily reflect the views or opinions of HCL or its affiliates.
> Any form of reproduction, dissemination, copying, disclosure,
> modification, distribution and / or publication of this message
> without the prior written consent of authorized representative of HCL
> is strictly prohibited. If you have received this email in error
> please delete it and notify the sender immediately. Before opening any
> email and/or attachments, please check them for viruses and other defects.
> ________________________________
>
RE: Issue in connecting zookeeper host from node js service
Posted by Manoj Jain <ma...@hcl.com.INVALID>.
Classification: Public
Thanks for the quick response. As far as ZK connection string is concerned it is valid and accessible and with the same I can successfully connect to the java service (via Solrj library - CloudSolrCleint)
We are using external zookeeper in our solr cloud setup in Kubernetes. Our headless zookeeper internally has three zookeeper pods (to make a quorum) which stores the configuration data for solr in a hierarchical data structure (Znode tree). Further solr cluster have three solr pods. The reason we want to connect to ZK instead of Solr cluster directly because(as per my understanding) its Zk only which is aware of the true state of Solr cluster. If we directly connect to Solr cluster instead of going through the ZK then I believe in production there are chances to encounter some weird scenarios, for instance the request may go to a Solr node which may not have the latest data.
Please share your feedback/inputs on the same.
Regards
Manoj Jain
-----Original Message-----
From: Ishan Chattopadhyaya <ic...@gmail.com>
Sent: Wednesday, May 31, 2023 1:56 PM
To: manoj.jain@hcl.com.invalid
Cc: users@solr.apache.org
Subject: Re: Issue in connecting zookeeper host from node js service
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don’t click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
If you're using Solr in Kubernetes, you can continue connecting to Solr pods without connecting to ZK.
We don't have any officially supported Solr client for NodeJS, but the HTTP endpoints of Solr are supported and can be used directly.
If you're having trouble connecting to ZK via a third party (unofficial) Solr client in NodeJS, maybe you can reach out to the support for that client library. Also, you can make sure the ZK connection string you're using is valid and accessible from outside the Kubernetes pods.
On Wed, 31 May, 2023, 12:00 am Manoj Jain, <ma...@hcl.com.invalid>
wrote:
> Classification: Confidential
> Hello,
>
> We are migrating from Solr-standalone to Solr-cloud of version 8.11
> using external zookeeper. We have the node and java microservices
> which are connected to Solr for indexing and query and everything is
> deployed within same GKE cluster.
>
> For the migration of solr cloud we have the task to modify the
> connection string so that it can point to Solr Cloud via zookeeper.
> For the java services we use 'CloudSolrClient' class of SolrJ library
> to connect to the zookeeper host and it worked well. And for the node
> service we are using 'Solr-client' library, but we are not able to
> connect to the zookeeper host using the same. For testing purpose, we
> give solr-cluster I.P. and port instead after which the service can
> connect but the same failed when we try to connect to zookeeper host
> (neither from DNS name or I.P. and port). We do not see any other
> specific solr library for node js to connect to Solr Cloud. Can anyone
> please suggest how we can connect to zookeeper host from node js service?
>
> Regards
> Manoj Jain
>
> ::DISCLAIMER::
> ________________________________
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only. E-mail transmission is not
> guaranteed to be secure or error-free as information could be
> intercepted, corrupted, lost, destroyed, arrive late or incomplete, or
> may contain viruses in transmission. The e mail and its contents (with
> or without referred errors) shall therefore not attach any liability
> on the originator or HCL or its affiliates. Views or opinions, if any,
> presented in this email are solely those of the author and may not
> necessarily reflect the views or opinions of HCL or its affiliates.
> Any form of reproduction, dissemination, copying, disclosure,
> modification, distribution and / or publication of this message
> without the prior written consent of authorized representative of HCL
> is strictly prohibited. If you have received this email in error
> please delete it and notify the sender immediately. Before opening any
> email and/or attachments, please check them for viruses and other defects.
> ________________________________
>
Re: Issue in connecting zookeeper host from node js service
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
If you're using Solr in Kubernetes, you can continue connecting to Solr
pods without connecting to ZK.
We don't have any officially supported Solr client for NodeJS, but the HTTP
endpoints of Solr are supported and can be used directly.
If you're having trouble connecting to ZK via a third party (unofficial)
Solr client in NodeJS, maybe you can reach out to the support for that
client library. Also, you can make sure the ZK connection string you're
using is valid and accessible from outside the Kubernetes pods.
On Wed, 31 May, 2023, 12:00 am Manoj Jain, <ma...@hcl.com.invalid>
wrote:
> Classification: Confidential
> Hello,
>
> We are migrating from Solr-standalone to Solr-cloud of version 8.11 using
> external zookeeper. We have the node and java microservices which are
> connected to Solr for indexing and query and everything is deployed within
> same GKE cluster.
>
> For the migration of solr cloud we have the task to modify the connection
> string so that it can point to Solr Cloud via zookeeper. For the java
> services we use 'CloudSolrClient' class of SolrJ library to connect to the
> zookeeper host and it worked well. And for the node service we are using
> 'Solr-client' library, but we are not able to connect to the zookeeper host
> using the same. For testing purpose, we give solr-cluster I.P. and port
> instead after which the service can connect but the same failed when we try
> to connect to zookeeper host (neither from DNS name or I.P. and port). We
> do not see any other specific solr library for node js to connect to Solr
> Cloud. Can anyone please suggest how we can connect to zookeeper host from
> node js service?
>
> Regards
> Manoj Jain
>
> ::DISCLAIMER::
> ________________________________
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only. E-mail transmission is not
> guaranteed to be secure or error-free as information could be intercepted,
> corrupted, lost, destroyed, arrive late or incomplete, or may contain
> viruses in transmission. The e mail and its contents (with or without
> referred errors) shall therefore not attach any liability on the originator
> or HCL or its affiliates. Views or opinions, if any, presented in this
> email are solely those of the author and may not necessarily reflect the
> views or opinions of HCL or its affiliates. Any form of reproduction,
> dissemination, copying, disclosure, modification, distribution and / or
> publication of this message without the prior written consent of authorized
> representative of HCL is strictly prohibited. If you have received this
> email in error please delete it and notify the sender immediately. Before
> opening any email and/or attachments, please check them for viruses and
> other defects.
> ________________________________
>