You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@manifoldcf.apache.org by Jörn Franke <jo...@gmail.com> on 2019/12/16 22:31:29 UTC
Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Hallo,
does the Solr Output Connector support SolrCloud with Kerberos
authentication and Zookeeper with Kerberos authentication?
If so, how can this be configured?
If it is not supported, is there an "easy" way to integrate this? From a
development perspective the Kerberos Authentication with both is not
difficult to achieve, but of course it stil needs to be integrated in the
whole solution.
Thank you.
Best regards
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Jörn Franke <jo...@gmail.com>.
Hi Karl,
I found a solution for Kerberos authentication and tested it on a
kerberized Solr 8.3 with kerberized Zookeeper 3.5 (you do NOT need to
update the ZK/Solr versions in manifold, those one found there work
perfectly fine in this scenario).
Find here my detailed report:
https://issues.apache.org/jira/browse/CONNECTORS-1629
Please let me know how I can make a contribution to one of the
next Manifold versions, so that it can be integrated.
Thank you.
best regards
On Wed, Dec 18, 2019 at 2:23 AM Karl Wright <da...@gmail.com> wrote:
> Found the problem: needed to update a pom dependency.
> Everything passes now.
>
> Karl
>
>
> On Tue, Dec 17, 2019 at 8:07 PM Karl Wright <da...@gmail.com> wrote:
>
>> I just created a plugin directory at
>> https://svn.apache.org/repos/asf/manifoldcf/integration/solr-8.x/trunk
>> . Code committed there builds but it doesn't test properly because of the
>> following exception:
>>
>> >>>>>>
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test)
>> on project apache-manifoldcf-solr-8.x-plugin: Execution default-test of
>> goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test failed:
>> There was an error in the forked process
>> [ERROR] org.apache.maven.surefire.util.SurefireReflectionException:
>> java.lang.ClassNotFoundException:
>> org.apache.maven.surefire.junit4.JUnit4Provider
>> [ERROR] at
>> org.apache.maven.surefire.util.ReflectionUtils.loadClass(ReflectionUtils.java:252)
>> [ERROR] at
>> org.apache.maven.surefire.util.ReflectionUtils.instantiateOneArg(ReflectionUtils.java:128)
>> [ERROR] at
>> org.apache.maven.surefire.booter.ForkedBooter.createProviderInCurrentClassloader(ForkedBooter.java:230)
>> [ERROR] at
>> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:199)
>> [ERROR] at
>> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
>> [ERROR] at
>> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
>> [ERROR] Caused by: java.lang.ClassNotFoundException:
>> org.apache.maven.surefire.junit4.JUnit4Provider
>> [ERROR] at
>> java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> [ERROR] at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> [ERROR] at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
>> [ERROR] at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> [ERROR] at
>> org.apache.maven.surefire.util.ReflectionUtils.loadClass(ReflectionUtils.java:244)
>> [ERROR] ... 5 more
>> [ERROR]
>> <<<<<<
>>
>> This is odd because the plugin depends on solr and solr has a transitive
>> maven dependency on junit. I'll see if a direct dependency works...
>>
>> Karl
>>
>>
>> On Tue, Dec 17, 2019 at 3:30 PM Jörn Franke <jo...@gmail.com> wrote:
>>
>>> Here you find it: https://issues.apache.org/jira/browse/CONNECTORS-1629
>>> I will try it out this year I hope.
>>> I will try it though with Solr 8.3.1 and will take into account
>>> https://issues.apache.org/jira/browse/CONNECTORS-1586
>>>
>>> On Tue, Dec 17, 2019 at 1:09 PM Karl Wright <da...@gmail.com> wrote:
>>>
>>>> Please do!
>>>> Karl
>>>>
>>>>
>>>> On Tue, Dec 17, 2019 at 7:06 AM Jörn Franke <jo...@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks a lot Karl for your feedback. Do you mind if I create a Jira
>>>>> where I report on the progress?
>>>>>
>>>>> Am 17.12.2019 um 12:22 schrieb Karl Wright <da...@gmail.com>:
>>>>>
>>>>>
>>>>> Well, you can certainly attempt this simply enough then if you build
>>>>> from source. I'd prefer that you validate the approach before we make
>>>>> permanent commits.
>>>>>
>>>>> Please let me know what works and what doesn't.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>> On Tue, Dec 17, 2019 at 1:22 AM Jörn Franke <jo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I agree.
>>>>>> The delegation part is not relevant for me. I also do not believe it
>>>>>> makes sense at the ETL level.
>>>>>> I think still we need add the one line of code that allows to use
>>>>>> Kerberos (second line in the example).
>>>>>>
>>>>>> Am 17.12.2019 um 01:35 schrieb Karl Wright <da...@gmail.com>:
>>>>>>
>>>>>>
>>>>>> Hi Jorn,
>>>>>>
>>>>>> The code referenced cannot be set up differently from connection to
>>>>>> connection so there is no point in having this be anything other than
>>>>>> global. In that case you can point at the config file with
>>>>>> -D<parameter>=value and it will do the same thing as setting a system
>>>>>> property.
>>>>>>
>>>>>> The token delegation with HttpClient I'll have to study to confirm
>>>>>> that we're doing this right in the connector.
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Dec 16, 2019 at 6:15 PM Jörn Franke <jo...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks a lot for the quick reply. Actually it is here:
>>>>>>> https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
>>>>>>> It is also available in the previous versions of Solr.
>>>>>>> I wonder how easy it would be to add a configuration to the Manifold
>>>>>>> UI to point to a jaas-client.conf. However, it is also not strictly
>>>>>>> necessary that this one is configurable in the UI. It could be also a
>>>>>>> checkbox yes/no Kerberos authentication and the jaas-client.conf could be
>>>>>>> put in a certain folder.
>>>>>>>
>>>>>>> Interesting would be also if Solr 8.x can be made work in this
>>>>>>> setting.
>>>>>>>
>>>>>>> On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> The Solr Output Connector uses a patched HttpComponents/HttpClient
>>>>>>>> for communication with the various Solr Cloud replicas, along with custom
>>>>>>>> versions of some of the SolrJ classes which allow multipart posts to work.
>>>>>>>> Other than that it's standard SolrJ. Whatever SolrJ needs to work with
>>>>>>>> Kerberos, therefore, should work with the ManifoldCF Solr Output
>>>>>>>> Connector. So if you can point me at the SolrJ documentation for this
>>>>>>>> configuration I can perhaps review it and give you my opinion as to the
>>>>>>>> difficulty involved.
>>>>>>>>
>>>>>>>> Karl
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hallo,
>>>>>>>>>
>>>>>>>>> does the Solr Output Connector support SolrCloud with Kerberos
>>>>>>>>> authentication and Zookeeper with Kerberos authentication?
>>>>>>>>>
>>>>>>>>> If so, how can this be configured?
>>>>>>>>>
>>>>>>>>> If it is not supported, is there an "easy" way to integrate this?
>>>>>>>>> From a development perspective the Kerberos Authentication with both is not
>>>>>>>>> difficult to achieve, but of course it stil needs to be integrated in the
>>>>>>>>> whole solution.
>>>>>>>>>
>>>>>>>>> Thank you.
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>>
>>>>>>>>
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Karl Wright <da...@gmail.com>.
Found the problem: needed to update a pom dependency.
Everything passes now.
Karl
On Tue, Dec 17, 2019 at 8:07 PM Karl Wright <da...@gmail.com> wrote:
> I just created a plugin directory at
> https://svn.apache.org/repos/asf/manifoldcf/integration/solr-8.x/trunk .
> Code committed there builds but it doesn't test properly because of the
> following exception:
>
> >>>>>>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test)
> on project apache-manifoldcf-solr-8.x-plugin: Execution default-test of
> goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test failed:
> There was an error in the forked process
> [ERROR] org.apache.maven.surefire.util.SurefireReflectionException:
> java.lang.ClassNotFoundException:
> org.apache.maven.surefire.junit4.JUnit4Provider
> [ERROR] at
> org.apache.maven.surefire.util.ReflectionUtils.loadClass(ReflectionUtils.java:252)
> [ERROR] at
> org.apache.maven.surefire.util.ReflectionUtils.instantiateOneArg(ReflectionUtils.java:128)
> [ERROR] at
> org.apache.maven.surefire.booter.ForkedBooter.createProviderInCurrentClassloader(ForkedBooter.java:230)
> [ERROR] at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:199)
> [ERROR] at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> [ERROR] at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
> [ERROR] Caused by: java.lang.ClassNotFoundException:
> org.apache.maven.surefire.junit4.JUnit4Provider
> [ERROR] at
> java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> [ERROR] at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> [ERROR] at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> [ERROR] at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> [ERROR] at
> org.apache.maven.surefire.util.ReflectionUtils.loadClass(ReflectionUtils.java:244)
> [ERROR] ... 5 more
> [ERROR]
> <<<<<<
>
> This is odd because the plugin depends on solr and solr has a transitive
> maven dependency on junit. I'll see if a direct dependency works...
>
> Karl
>
>
> On Tue, Dec 17, 2019 at 3:30 PM Jörn Franke <jo...@gmail.com> wrote:
>
>> Here you find it: https://issues.apache.org/jira/browse/CONNECTORS-1629
>> I will try it out this year I hope.
>> I will try it though with Solr 8.3.1 and will take into account
>> https://issues.apache.org/jira/browse/CONNECTORS-1586
>>
>> On Tue, Dec 17, 2019 at 1:09 PM Karl Wright <da...@gmail.com> wrote:
>>
>>> Please do!
>>> Karl
>>>
>>>
>>> On Tue, Dec 17, 2019 at 7:06 AM Jörn Franke <jo...@gmail.com>
>>> wrote:
>>>
>>>> Thanks a lot Karl for your feedback. Do you mind if I create a Jira
>>>> where I report on the progress?
>>>>
>>>> Am 17.12.2019 um 12:22 schrieb Karl Wright <da...@gmail.com>:
>>>>
>>>>
>>>> Well, you can certainly attempt this simply enough then if you build
>>>> from source. I'd prefer that you validate the approach before we make
>>>> permanent commits.
>>>>
>>>> Please let me know what works and what doesn't.
>>>>
>>>> Karl
>>>>
>>>>
>>>> On Tue, Dec 17, 2019 at 1:22 AM Jörn Franke <jo...@gmail.com>
>>>> wrote:
>>>>
>>>>> I agree.
>>>>> The delegation part is not relevant for me. I also do not believe it
>>>>> makes sense at the ETL level.
>>>>> I think still we need add the one line of code that allows to use
>>>>> Kerberos (second line in the example).
>>>>>
>>>>> Am 17.12.2019 um 01:35 schrieb Karl Wright <da...@gmail.com>:
>>>>>
>>>>>
>>>>> Hi Jorn,
>>>>>
>>>>> The code referenced cannot be set up differently from connection to
>>>>> connection so there is no point in having this be anything other than
>>>>> global. In that case you can point at the config file with
>>>>> -D<parameter>=value and it will do the same thing as setting a system
>>>>> property.
>>>>>
>>>>> The token delegation with HttpClient I'll have to study to confirm
>>>>> that we're doing this right in the connector.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Dec 16, 2019 at 6:15 PM Jörn Franke <jo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks a lot for the quick reply. Actually it is here:
>>>>>> https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
>>>>>> It is also available in the previous versions of Solr.
>>>>>> I wonder how easy it would be to add a configuration to the Manifold
>>>>>> UI to point to a jaas-client.conf. However, it is also not strictly
>>>>>> necessary that this one is configurable in the UI. It could be also a
>>>>>> checkbox yes/no Kerberos authentication and the jaas-client.conf could be
>>>>>> put in a certain folder.
>>>>>>
>>>>>> Interesting would be also if Solr 8.x can be made work in this
>>>>>> setting.
>>>>>>
>>>>>> On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> The Solr Output Connector uses a patched HttpComponents/HttpClient
>>>>>>> for communication with the various Solr Cloud replicas, along with custom
>>>>>>> versions of some of the SolrJ classes which allow multipart posts to work.
>>>>>>> Other than that it's standard SolrJ. Whatever SolrJ needs to work with
>>>>>>> Kerberos, therefore, should work with the ManifoldCF Solr Output
>>>>>>> Connector. So if you can point me at the SolrJ documentation for this
>>>>>>> configuration I can perhaps review it and give you my opinion as to the
>>>>>>> difficulty involved.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hallo,
>>>>>>>>
>>>>>>>> does the Solr Output Connector support SolrCloud with Kerberos
>>>>>>>> authentication and Zookeeper with Kerberos authentication?
>>>>>>>>
>>>>>>>> If so, how can this be configured?
>>>>>>>>
>>>>>>>> If it is not supported, is there an "easy" way to integrate this?
>>>>>>>> From a development perspective the Kerberos Authentication with both is not
>>>>>>>> difficult to achieve, but of course it stil needs to be integrated in the
>>>>>>>> whole solution.
>>>>>>>>
>>>>>>>> Thank you.
>>>>>>>>
>>>>>>>> Best regards
>>>>>>>>
>>>>>>>
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Karl Wright <da...@gmail.com>.
I just created a plugin directory at
https://svn.apache.org/repos/asf/manifoldcf/integration/solr-8.x/trunk .
Code committed there builds but it doesn't test properly because of the
following exception:
>>>>>>
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test)
on project apache-manifoldcf-solr-8.x-plugin: Execution default-test of
goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test failed:
There was an error in the forked process
[ERROR] org.apache.maven.surefire.util.SurefireReflectionException:
java.lang.ClassNotFoundException:
org.apache.maven.surefire.junit4.JUnit4Provider
[ERROR] at
org.apache.maven.surefire.util.ReflectionUtils.loadClass(ReflectionUtils.java:252)
[ERROR] at
org.apache.maven.surefire.util.ReflectionUtils.instantiateOneArg(ReflectionUtils.java:128)
[ERROR] at
org.apache.maven.surefire.booter.ForkedBooter.createProviderInCurrentClassloader(ForkedBooter.java:230)
[ERROR] at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:199)
[ERROR] at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
[ERROR] at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
[ERROR] Caused by: java.lang.ClassNotFoundException:
org.apache.maven.surefire.junit4.JUnit4Provider
[ERROR] at
java.net.URLClassLoader.findClass(URLClassLoader.java:381)
[ERROR] at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
[ERROR] at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
[ERROR] at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
[ERROR] at
org.apache.maven.surefire.util.ReflectionUtils.loadClass(ReflectionUtils.java:244)
[ERROR] ... 5 more
[ERROR]
<<<<<<
This is odd because the plugin depends on solr and solr has a transitive
maven dependency on junit. I'll see if a direct dependency works...
Karl
On Tue, Dec 17, 2019 at 3:30 PM Jörn Franke <jo...@gmail.com> wrote:
> Here you find it: https://issues.apache.org/jira/browse/CONNECTORS-1629
> I will try it out this year I hope.
> I will try it though with Solr 8.3.1 and will take into account
> https://issues.apache.org/jira/browse/CONNECTORS-1586
>
> On Tue, Dec 17, 2019 at 1:09 PM Karl Wright <da...@gmail.com> wrote:
>
>> Please do!
>> Karl
>>
>>
>> On Tue, Dec 17, 2019 at 7:06 AM Jörn Franke <jo...@gmail.com> wrote:
>>
>>> Thanks a lot Karl for your feedback. Do you mind if I create a Jira
>>> where I report on the progress?
>>>
>>> Am 17.12.2019 um 12:22 schrieb Karl Wright <da...@gmail.com>:
>>>
>>>
>>> Well, you can certainly attempt this simply enough then if you build
>>> from source. I'd prefer that you validate the approach before we make
>>> permanent commits.
>>>
>>> Please let me know what works and what doesn't.
>>>
>>> Karl
>>>
>>>
>>> On Tue, Dec 17, 2019 at 1:22 AM Jörn Franke <jo...@gmail.com>
>>> wrote:
>>>
>>>> I agree.
>>>> The delegation part is not relevant for me. I also do not believe it
>>>> makes sense at the ETL level.
>>>> I think still we need add the one line of code that allows to use
>>>> Kerberos (second line in the example).
>>>>
>>>> Am 17.12.2019 um 01:35 schrieb Karl Wright <da...@gmail.com>:
>>>>
>>>>
>>>> Hi Jorn,
>>>>
>>>> The code referenced cannot be set up differently from connection to
>>>> connection so there is no point in having this be anything other than
>>>> global. In that case you can point at the config file with
>>>> -D<parameter>=value and it will do the same thing as setting a system
>>>> property.
>>>>
>>>> The token delegation with HttpClient I'll have to study to confirm that
>>>> we're doing this right in the connector.
>>>>
>>>> Karl
>>>>
>>>>
>>>>
>>>> On Mon, Dec 16, 2019 at 6:15 PM Jörn Franke <jo...@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks a lot for the quick reply. Actually it is here:
>>>>> https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
>>>>> It is also available in the previous versions of Solr.
>>>>> I wonder how easy it would be to add a configuration to the Manifold
>>>>> UI to point to a jaas-client.conf. However, it is also not strictly
>>>>> necessary that this one is configurable in the UI. It could be also a
>>>>> checkbox yes/no Kerberos authentication and the jaas-client.conf could be
>>>>> put in a certain folder.
>>>>>
>>>>> Interesting would be also if Solr 8.x can be made work in this setting.
>>>>>
>>>>> On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> The Solr Output Connector uses a patched HttpComponents/HttpClient
>>>>>> for communication with the various Solr Cloud replicas, along with custom
>>>>>> versions of some of the SolrJ classes which allow multipart posts to work.
>>>>>> Other than that it's standard SolrJ. Whatever SolrJ needs to work with
>>>>>> Kerberos, therefore, should work with the ManifoldCF Solr Output
>>>>>> Connector. So if you can point me at the SolrJ documentation for this
>>>>>> configuration I can perhaps review it and give you my opinion as to the
>>>>>> difficulty involved.
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hallo,
>>>>>>>
>>>>>>> does the Solr Output Connector support SolrCloud with Kerberos
>>>>>>> authentication and Zookeeper with Kerberos authentication?
>>>>>>>
>>>>>>> If so, how can this be configured?
>>>>>>>
>>>>>>> If it is not supported, is there an "easy" way to integrate this?
>>>>>>> From a development perspective the Kerberos Authentication with both is not
>>>>>>> difficult to achieve, but of course it stil needs to be integrated in the
>>>>>>> whole solution.
>>>>>>>
>>>>>>> Thank you.
>>>>>>>
>>>>>>> Best regards
>>>>>>>
>>>>>>
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Jörn Franke <jo...@gmail.com>.
Here you find it: https://issues.apache.org/jira/browse/CONNECTORS-1629
I will try it out this year I hope.
I will try it though with Solr 8.3.1 and will take into account
https://issues.apache.org/jira/browse/CONNECTORS-1586
On Tue, Dec 17, 2019 at 1:09 PM Karl Wright <da...@gmail.com> wrote:
> Please do!
> Karl
>
>
> On Tue, Dec 17, 2019 at 7:06 AM Jörn Franke <jo...@gmail.com> wrote:
>
>> Thanks a lot Karl for your feedback. Do you mind if I create a Jira where
>> I report on the progress?
>>
>> Am 17.12.2019 um 12:22 schrieb Karl Wright <da...@gmail.com>:
>>
>>
>> Well, you can certainly attempt this simply enough then if you build from
>> source. I'd prefer that you validate the approach before we make permanent
>> commits.
>>
>> Please let me know what works and what doesn't.
>>
>> Karl
>>
>>
>> On Tue, Dec 17, 2019 at 1:22 AM Jörn Franke <jo...@gmail.com> wrote:
>>
>>> I agree.
>>> The delegation part is not relevant for me. I also do not believe it
>>> makes sense at the ETL level.
>>> I think still we need add the one line of code that allows to use
>>> Kerberos (second line in the example).
>>>
>>> Am 17.12.2019 um 01:35 schrieb Karl Wright <da...@gmail.com>:
>>>
>>>
>>> Hi Jorn,
>>>
>>> The code referenced cannot be set up differently from connection to
>>> connection so there is no point in having this be anything other than
>>> global. In that case you can point at the config file with
>>> -D<parameter>=value and it will do the same thing as setting a system
>>> property.
>>>
>>> The token delegation with HttpClient I'll have to study to confirm that
>>> we're doing this right in the connector.
>>>
>>> Karl
>>>
>>>
>>>
>>> On Mon, Dec 16, 2019 at 6:15 PM Jörn Franke <jo...@gmail.com>
>>> wrote:
>>>
>>>> Thanks a lot for the quick reply. Actually it is here:
>>>> https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
>>>> It is also available in the previous versions of Solr.
>>>> I wonder how easy it would be to add a configuration to the Manifold UI
>>>> to point to a jaas-client.conf. However, it is also not strictly necessary
>>>> that this one is configurable in the UI. It could be also a checkbox yes/no
>>>> Kerberos authentication and the jaas-client.conf could be put in a certain
>>>> folder.
>>>>
>>>> Interesting would be also if Solr 8.x can be made work in this setting.
>>>>
>>>> On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com>
>>>> wrote:
>>>>
>>>>> The Solr Output Connector uses a patched HttpComponents/HttpClient for
>>>>> communication with the various Solr Cloud replicas, along with custom
>>>>> versions of some of the SolrJ classes which allow multipart posts to work.
>>>>> Other than that it's standard SolrJ. Whatever SolrJ needs to work with
>>>>> Kerberos, therefore, should work with the ManifoldCF Solr Output
>>>>> Connector. So if you can point me at the SolrJ documentation for this
>>>>> configuration I can perhaps review it and give you my opinion as to the
>>>>> difficulty involved.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hallo,
>>>>>>
>>>>>> does the Solr Output Connector support SolrCloud with Kerberos
>>>>>> authentication and Zookeeper with Kerberos authentication?
>>>>>>
>>>>>> If so, how can this be configured?
>>>>>>
>>>>>> If it is not supported, is there an "easy" way to integrate this?
>>>>>> From a development perspective the Kerberos Authentication with both is not
>>>>>> difficult to achieve, but of course it stil needs to be integrated in the
>>>>>> whole solution.
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>> Best regards
>>>>>>
>>>>>
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Karl Wright <da...@gmail.com>.
Please do!
Karl
On Tue, Dec 17, 2019 at 7:06 AM Jörn Franke <jo...@gmail.com> wrote:
> Thanks a lot Karl for your feedback. Do you mind if I create a Jira where
> I report on the progress?
>
> Am 17.12.2019 um 12:22 schrieb Karl Wright <da...@gmail.com>:
>
>
> Well, you can certainly attempt this simply enough then if you build from
> source. I'd prefer that you validate the approach before we make permanent
> commits.
>
> Please let me know what works and what doesn't.
>
> Karl
>
>
> On Tue, Dec 17, 2019 at 1:22 AM Jörn Franke <jo...@gmail.com> wrote:
>
>> I agree.
>> The delegation part is not relevant for me. I also do not believe it
>> makes sense at the ETL level.
>> I think still we need add the one line of code that allows to use
>> Kerberos (second line in the example).
>>
>> Am 17.12.2019 um 01:35 schrieb Karl Wright <da...@gmail.com>:
>>
>>
>> Hi Jorn,
>>
>> The code referenced cannot be set up differently from connection to
>> connection so there is no point in having this be anything other than
>> global. In that case you can point at the config file with
>> -D<parameter>=value and it will do the same thing as setting a system
>> property.
>>
>> The token delegation with HttpClient I'll have to study to confirm that
>> we're doing this right in the connector.
>>
>> Karl
>>
>>
>>
>> On Mon, Dec 16, 2019 at 6:15 PM Jörn Franke <jo...@gmail.com> wrote:
>>
>>> Thanks a lot for the quick reply. Actually it is here:
>>> https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
>>> It is also available in the previous versions of Solr.
>>> I wonder how easy it would be to add a configuration to the Manifold UI
>>> to point to a jaas-client.conf. However, it is also not strictly necessary
>>> that this one is configurable in the UI. It could be also a checkbox yes/no
>>> Kerberos authentication and the jaas-client.conf could be put in a certain
>>> folder.
>>>
>>> Interesting would be also if Solr 8.x can be made work in this setting.
>>>
>>> On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com> wrote:
>>>
>>>> The Solr Output Connector uses a patched HttpComponents/HttpClient for
>>>> communication with the various Solr Cloud replicas, along with custom
>>>> versions of some of the SolrJ classes which allow multipart posts to work.
>>>> Other than that it's standard SolrJ. Whatever SolrJ needs to work with
>>>> Kerberos, therefore, should work with the ManifoldCF Solr Output
>>>> Connector. So if you can point me at the SolrJ documentation for this
>>>> configuration I can perhaps review it and give you my opinion as to the
>>>> difficulty involved.
>>>>
>>>> Karl
>>>>
>>>>
>>>> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hallo,
>>>>>
>>>>> does the Solr Output Connector support SolrCloud with Kerberos
>>>>> authentication and Zookeeper with Kerberos authentication?
>>>>>
>>>>> If so, how can this be configured?
>>>>>
>>>>> If it is not supported, is there an "easy" way to integrate this? From
>>>>> a development perspective the Kerberos Authentication with both is not
>>>>> difficult to achieve, but of course it stil needs to be integrated in the
>>>>> whole solution.
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Best regards
>>>>>
>>>>
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Jörn Franke <jo...@gmail.com>.
Thanks a lot Karl for your feedback. Do you mind if I create a Jira where I report on the progress?
> Am 17.12.2019 um 12:22 schrieb Karl Wright <da...@gmail.com>:
>
>
> Well, you can certainly attempt this simply enough then if you build from source. I'd prefer that you validate the approach before we make permanent commits.
>
> Please let me know what works and what doesn't.
>
> Karl
>
>
>> On Tue, Dec 17, 2019 at 1:22 AM Jörn Franke <jo...@gmail.com> wrote:
>> I agree.
>> The delegation part is not relevant for me. I also do not believe it makes sense at the ETL level.
>> I think still we need add the one line of code that allows to use Kerberos (second line in the example).
>>
>>>> Am 17.12.2019 um 01:35 schrieb Karl Wright <da...@gmail.com>:
>>>>
>>>
>>> Hi Jorn,
>>>
>>> The code referenced cannot be set up differently from connection to connection so there is no point in having this be anything other than global. In that case you can point at the config file with -D<parameter>=value and it will do the same thing as setting a system property.
>>>
>>> The token delegation with HttpClient I'll have to study to confirm that we're doing this right in the connector.
>>>
>>> Karl
>>>
>>>
>>>
>>>> On Mon, Dec 16, 2019 at 6:15 PM Jörn Franke <jo...@gmail.com> wrote:
>>>> Thanks a lot for the quick reply. Actually it is here: https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
>>>> It is also available in the previous versions of Solr.
>>>> I wonder how easy it would be to add a configuration to the Manifold UI to point to a jaas-client.conf. However, it is also not strictly necessary that this one is configurable in the UI. It could be also a checkbox yes/no Kerberos authentication and the jaas-client.conf could be put in a certain folder.
>>>>
>>>> Interesting would be also if Solr 8.x can be made work in this setting.
>>>>
>>>>> On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com> wrote:
>>>>> The Solr Output Connector uses a patched HttpComponents/HttpClient for communication with the various Solr Cloud replicas, along with custom versions of some of the SolrJ classes which allow multipart posts to work. Other than that it's standard SolrJ. Whatever SolrJ needs to work with Kerberos, therefore, should work with the ManifoldCF Solr Output Connector. So if you can point me at the SolrJ documentation for this configuration I can perhaps review it and give you my opinion as to the difficulty involved.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>>> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com> wrote:
>>>>>> Hallo,
>>>>>>
>>>>>> does the Solr Output Connector support SolrCloud with Kerberos authentication and Zookeeper with Kerberos authentication?
>>>>>>
>>>>>> If so, how can this be configured?
>>>>>>
>>>>>> If it is not supported, is there an "easy" way to integrate this? From a development perspective the Kerberos Authentication with both is not difficult to achieve, but of course it stil needs to be integrated in the whole solution.
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>> Best regards
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Karl Wright <da...@gmail.com>.
Well, you can certainly attempt this simply enough then if you build from
source. I'd prefer that you validate the approach before we make permanent
commits.
Please let me know what works and what doesn't.
Karl
On Tue, Dec 17, 2019 at 1:22 AM Jörn Franke <jo...@gmail.com> wrote:
> I agree.
> The delegation part is not relevant for me. I also do not believe it makes
> sense at the ETL level.
> I think still we need add the one line of code that allows to use
> Kerberos (second line in the example).
>
> Am 17.12.2019 um 01:35 schrieb Karl Wright <da...@gmail.com>:
>
>
> Hi Jorn,
>
> The code referenced cannot be set up differently from connection to
> connection so there is no point in having this be anything other than
> global. In that case you can point at the config file with
> -D<parameter>=value and it will do the same thing as setting a system
> property.
>
> The token delegation with HttpClient I'll have to study to confirm that
> we're doing this right in the connector.
>
> Karl
>
>
>
> On Mon, Dec 16, 2019 at 6:15 PM Jörn Franke <jo...@gmail.com> wrote:
>
>> Thanks a lot for the quick reply. Actually it is here:
>> https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
>> It is also available in the previous versions of Solr.
>> I wonder how easy it would be to add a configuration to the Manifold UI
>> to point to a jaas-client.conf. However, it is also not strictly necessary
>> that this one is configurable in the UI. It could be also a checkbox yes/no
>> Kerberos authentication and the jaas-client.conf could be put in a certain
>> folder.
>>
>> Interesting would be also if Solr 8.x can be made work in this setting.
>>
>> On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com> wrote:
>>
>>> The Solr Output Connector uses a patched HttpComponents/HttpClient for
>>> communication with the various Solr Cloud replicas, along with custom
>>> versions of some of the SolrJ classes which allow multipart posts to work.
>>> Other than that it's standard SolrJ. Whatever SolrJ needs to work with
>>> Kerberos, therefore, should work with the ManifoldCF Solr Output
>>> Connector. So if you can point me at the SolrJ documentation for this
>>> configuration I can perhaps review it and give you my opinion as to the
>>> difficulty involved.
>>>
>>> Karl
>>>
>>>
>>> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com>
>>> wrote:
>>>
>>>> Hallo,
>>>>
>>>> does the Solr Output Connector support SolrCloud with Kerberos
>>>> authentication and Zookeeper with Kerberos authentication?
>>>>
>>>> If so, how can this be configured?
>>>>
>>>> If it is not supported, is there an "easy" way to integrate this? From
>>>> a development perspective the Kerberos Authentication with both is not
>>>> difficult to achieve, but of course it stil needs to be integrated in the
>>>> whole solution.
>>>>
>>>> Thank you.
>>>>
>>>> Best regards
>>>>
>>>
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Jörn Franke <jo...@gmail.com>.
I agree.
The delegation part is not relevant for me. I also do not believe it makes sense at the ETL level.
I think still we need add the one line of code that allows to use Kerberos (second line in the example).
> Am 17.12.2019 um 01:35 schrieb Karl Wright <da...@gmail.com>:
>
>
> Hi Jorn,
>
> The code referenced cannot be set up differently from connection to connection so there is no point in having this be anything other than global. In that case you can point at the config file with -D<parameter>=value and it will do the same thing as setting a system property.
>
> The token delegation with HttpClient I'll have to study to confirm that we're doing this right in the connector.
>
> Karl
>
>
>
>> On Mon, Dec 16, 2019 at 6:15 PM Jörn Franke <jo...@gmail.com> wrote:
>> Thanks a lot for the quick reply. Actually it is here: https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
>> It is also available in the previous versions of Solr.
>> I wonder how easy it would be to add a configuration to the Manifold UI to point to a jaas-client.conf. However, it is also not strictly necessary that this one is configurable in the UI. It could be also a checkbox yes/no Kerberos authentication and the jaas-client.conf could be put in a certain folder.
>>
>> Interesting would be also if Solr 8.x can be made work in this setting.
>>
>>> On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com> wrote:
>>> The Solr Output Connector uses a patched HttpComponents/HttpClient for communication with the various Solr Cloud replicas, along with custom versions of some of the SolrJ classes which allow multipart posts to work. Other than that it's standard SolrJ. Whatever SolrJ needs to work with Kerberos, therefore, should work with the ManifoldCF Solr Output Connector. So if you can point me at the SolrJ documentation for this configuration I can perhaps review it and give you my opinion as to the difficulty involved.
>>>
>>> Karl
>>>
>>>
>>>> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com> wrote:
>>>> Hallo,
>>>>
>>>> does the Solr Output Connector support SolrCloud with Kerberos authentication and Zookeeper with Kerberos authentication?
>>>>
>>>> If so, how can this be configured?
>>>>
>>>> If it is not supported, is there an "easy" way to integrate this? From a development perspective the Kerberos Authentication with both is not difficult to achieve, but of course it stil needs to be integrated in the whole solution.
>>>>
>>>> Thank you.
>>>>
>>>> Best regards
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Karl Wright <da...@gmail.com>.
Hi Jorn,
The code referenced cannot be set up differently from connection to
connection so there is no point in having this be anything other than
global. In that case you can point at the config file with
-D<parameter>=value and it will do the same thing as setting a system
property.
The token delegation with HttpClient I'll have to study to confirm that
we're doing this right in the connector.
Karl
On Mon, Dec 16, 2019 at 6:15 PM Jörn Franke <jo...@gmail.com> wrote:
> Thanks a lot for the quick reply. Actually it is here:
> https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
> It is also available in the previous versions of Solr.
> I wonder how easy it would be to add a configuration to the Manifold UI to
> point to a jaas-client.conf. However, it is also not strictly necessary
> that this one is configurable in the UI. It could be also a checkbox yes/no
> Kerberos authentication and the jaas-client.conf could be put in a certain
> folder.
>
> Interesting would be also if Solr 8.x can be made work in this setting.
>
> On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com> wrote:
>
>> The Solr Output Connector uses a patched HttpComponents/HttpClient for
>> communication with the various Solr Cloud replicas, along with custom
>> versions of some of the SolrJ classes which allow multipart posts to work.
>> Other than that it's standard SolrJ. Whatever SolrJ needs to work with
>> Kerberos, therefore, should work with the ManifoldCF Solr Output
>> Connector. So if you can point me at the SolrJ documentation for this
>> configuration I can perhaps review it and give you my opinion as to the
>> difficulty involved.
>>
>> Karl
>>
>>
>> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com> wrote:
>>
>>> Hallo,
>>>
>>> does the Solr Output Connector support SolrCloud with Kerberos
>>> authentication and Zookeeper with Kerberos authentication?
>>>
>>> If so, how can this be configured?
>>>
>>> If it is not supported, is there an "easy" way to integrate this? From a
>>> development perspective the Kerberos Authentication with both is not
>>> difficult to achieve, but of course it stil needs to be integrated in the
>>> whole solution.
>>>
>>> Thank you.
>>>
>>> Best regards
>>>
>>
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Jörn Franke <jo...@gmail.com>.
Thanks a lot for the quick reply. Actually it is here:
https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
It is also available in the previous versions of Solr.
I wonder how easy it would be to add a configuration to the Manifold UI to
point to a jaas-client.conf. However, it is also not strictly necessary
that this one is configurable in the UI. It could be also a checkbox yes/no
Kerberos authentication and the jaas-client.conf could be put in a certain
folder.
Interesting would be also if Solr 8.x can be made work in this setting.
On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com> wrote:
> The Solr Output Connector uses a patched HttpComponents/HttpClient for
> communication with the various Solr Cloud replicas, along with custom
> versions of some of the SolrJ classes which allow multipart posts to work.
> Other than that it's standard SolrJ. Whatever SolrJ needs to work with
> Kerberos, therefore, should work with the ManifoldCF Solr Output
> Connector. So if you can point me at the SolrJ documentation for this
> configuration I can perhaps review it and give you my opinion as to the
> difficulty involved.
>
> Karl
>
>
> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com> wrote:
>
>> Hallo,
>>
>> does the Solr Output Connector support SolrCloud with Kerberos
>> authentication and Zookeeper with Kerberos authentication?
>>
>> If so, how can this be configured?
>>
>> If it is not supported, is there an "easy" way to integrate this? From a
>> development perspective the Kerberos Authentication with both is not
>> difficult to achieve, but of course it stil needs to be integrated in the
>> whole solution.
>>
>> Thank you.
>>
>> Best regards
>>
>
Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos
Posted by Karl Wright <da...@gmail.com>.
The Solr Output Connector uses a patched HttpComponents/HttpClient for
communication with the various Solr Cloud replicas, along with custom
versions of some of the SolrJ classes which allow multipart posts to work.
Other than that it's standard SolrJ. Whatever SolrJ needs to work with
Kerberos, therefore, should work with the ManifoldCF Solr Output
Connector. So if you can point me at the SolrJ documentation for this
configuration I can perhaps review it and give you my opinion as to the
difficulty involved.
Karl
On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com> wrote:
> Hallo,
>
> does the Solr Output Connector support SolrCloud with Kerberos
> authentication and Zookeeper with Kerberos authentication?
>
> If so, how can this be configured?
>
> If it is not supported, is there an "easy" way to integrate this? From a
> development perspective the Kerberos Authentication with both is not
> difficult to achieve, but of course it stil needs to be integrated in the
> whole solution.
>
> Thank you.
>
> Best regards
>