Re: Solr Output Connector: SolrCloud with Kerberos / Zookeeper with Kerberos

[Jörn Franke <jo...@gmail.com>]

Hi Karl,

I found a solution for Kerberos authentication and tested it on a
kerberized Solr 8.3 with kerberized Zookeeper 3.5 (you do NOT need to
update the ZK/Solr versions in manifold, those one found there work
perfectly fine in this scenario).
Find here my detailed report:
https://issues.apache.org/jira/browse/CONNECTORS-1629

Please let me know how I can make a contribution to one of the
next Manifold versions, so that it can be integrated.

Thank you.
best regards

On Wed, Dec 18, 2019 at 2:23 AM Karl Wright <da...@gmail.com> wrote:

> Found the problem: needed to update a pom dependency.
> Everything passes now.
>
> Karl
>
>
> On Tue, Dec 17, 2019 at 8:07 PM Karl Wright <da...@gmail.com> wrote:
>
>> I just created a plugin directory at
>> https://svn.apache.org/repos/asf/manifoldcf/integration/solr-8.x/trunk
>> .  Code committed there builds but it doesn't test properly because of the
>> following exception:
>>
>> >>>>>>
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test)
>> on project apache-manifoldcf-solr-8.x-plugin: Execution default-test of
>> goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test failed:
>> There was an error in the forked process
>> [ERROR] org.apache.maven.surefire.util.SurefireReflectionException:
>> java.lang.ClassNotFoundException:
>> org.apache.maven.surefire.junit4.JUnit4Provider
>> [ERROR]         at
>> org.apache.maven.surefire.util.ReflectionUtils.loadClass(ReflectionUtils.java:252)
>> [ERROR]         at
>> org.apache.maven.surefire.util.ReflectionUtils.instantiateOneArg(ReflectionUtils.java:128)
>> [ERROR]         at
>> org.apache.maven.surefire.booter.ForkedBooter.createProviderInCurrentClassloader(ForkedBooter.java:230)
>> [ERROR]         at
>> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:199)
>> [ERROR]         at
>> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
>> [ERROR]         at
>> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
>> [ERROR] Caused by: java.lang.ClassNotFoundException:
>> org.apache.maven.surefire.junit4.JUnit4Provider
>> [ERROR]         at
>> java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> [ERROR]         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> [ERROR]         at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
>> [ERROR]         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> [ERROR]         at
>> org.apache.maven.surefire.util.ReflectionUtils.loadClass(ReflectionUtils.java:244)
>> [ERROR]         ... 5 more
>> [ERROR]
>> <<<<<<
>>
>> This is odd because the plugin depends on solr and solr has a transitive
>> maven dependency on junit.  I'll see if a direct dependency works...
>>
>> Karl
>>
>>
>> On Tue, Dec 17, 2019 at 3:30 PM Jörn Franke <jo...@gmail.com> wrote:
>>
>>> Here you find it: https://issues.apache.org/jira/browse/CONNECTORS-1629
>>> I will try it out this year I hope.
>>> I will try it though with Solr 8.3.1 and will take into account
>>> https://issues.apache.org/jira/browse/CONNECTORS-1586
>>>
>>> On Tue, Dec 17, 2019 at 1:09 PM Karl Wright <da...@gmail.com> wrote:
>>>
>>>> Please do!
>>>> Karl
>>>>
>>>>
>>>> On Tue, Dec 17, 2019 at 7:06 AM Jörn Franke <jo...@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks a lot Karl for your feedback. Do you mind if I create a Jira
>>>>> where I report on the progress?
>>>>>
>>>>> Am 17.12.2019 um 12:22 schrieb Karl Wright <da...@gmail.com>:
>>>>>
>>>>> 
>>>>> Well, you can certainly attempt this simply enough then if you build
>>>>> from source.  I'd prefer that you validate the approach before we make
>>>>> permanent commits.
>>>>>
>>>>> Please let me know what works and what doesn't.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>> On Tue, Dec 17, 2019 at 1:22 AM Jörn Franke <jo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I agree.
>>>>>> The delegation part is not relevant for me. I also do not believe it
>>>>>> makes sense at the ETL level.
>>>>>>  I think still we need add the one line of code that allows to use
>>>>>> Kerberos (second line in the example).
>>>>>>
>>>>>> Am 17.12.2019 um 01:35 schrieb Karl Wright <da...@gmail.com>:
>>>>>>
>>>>>> 
>>>>>> Hi Jorn,
>>>>>>
>>>>>> The code referenced cannot be set up differently from connection to
>>>>>> connection so there is no point in having this be anything other than
>>>>>> global.  In that case you can point at the config file with
>>>>>> -D<parameter>=value and it will do the same thing as setting a system
>>>>>> property.
>>>>>>
>>>>>> The token delegation with HttpClient I'll have to study to confirm
>>>>>> that we're doing this right in the connector.
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Dec 16, 2019 at 6:15 PM Jörn Franke <jo...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks a lot for the quick reply. Actually it is here:
>>>>>>> https://lucene.apache.org/solr/guide/8_3/kerberos-authentication-plugin.html#using-solrj-with-a-kerberized-solr
>>>>>>> It is also available in the previous versions of Solr.
>>>>>>> I wonder how easy it would be to add a configuration to the Manifold
>>>>>>> UI to point to a jaas-client.conf. However, it is also not strictly
>>>>>>> necessary that this one is configurable in the UI. It could be also a
>>>>>>> checkbox yes/no Kerberos authentication and the jaas-client.conf could be
>>>>>>> put in a certain folder.
>>>>>>>
>>>>>>> Interesting would be also if Solr 8.x can be made work in this
>>>>>>> setting.
>>>>>>>
>>>>>>> On Mon, Dec 16, 2019 at 11:47 PM Karl Wright <da...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> The Solr Output Connector uses a patched HttpComponents/HttpClient
>>>>>>>> for communication with the various Solr Cloud replicas, along with custom
>>>>>>>> versions of some of the SolrJ classes which allow multipart posts to work.
>>>>>>>> Other than that it's standard SolrJ.  Whatever SolrJ needs to work with
>>>>>>>> Kerberos, therefore, should work with the ManifoldCF Solr Output
>>>>>>>> Connector.  So if you can point me at the SolrJ documentation for this
>>>>>>>> configuration I can perhaps review it and give you my opinion as to the
>>>>>>>> difficulty involved.
>>>>>>>>
>>>>>>>> Karl
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Dec 16, 2019 at 5:31 PM Jörn Franke <jo...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hallo,
>>>>>>>>>
>>>>>>>>> does the Solr Output Connector support SolrCloud with Kerberos
>>>>>>>>> authentication and Zookeeper with Kerberos authentication?
>>>>>>>>>
>>>>>>>>> If so, how can this be configured?
>>>>>>>>>
>>>>>>>>> If it is not supported, is there an "easy" way to integrate this?
>>>>>>>>> From a development perspective the Kerberos Authentication with both is not
>>>>>>>>> difficult to achieve, but of course it stil needs to be integrated in the
>>>>>>>>> whole solution.
>>>>>>>>>
>>>>>>>>> Thank you.
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>>
>>>>>>>>