You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Natarajan, Rajeswari" <ra...@sap.com> on 2020/07/15 09:52:16 UTC

SSL + Solr 8.5.1 in cloud mode + Java 8

Ok , I looked at the solr script , here is the logic , even if the  SOLR_SSL_CLIENT_KEY_STORE_PASSWORD is not set , it sets the  -Djavax.net.ssl.keyStore=$SOLR_SSL_KEY_STORE" . Let me remove the else part and test it . Thanks again for the pointer

if [ -n "$SOLR_SSL_CLIENT_KEY_STORE" ]; then
    SOLR_SSL_OPTS+=" -Djavax.net.ssl.keyStore=$SOLR_SSL_CLIENT_KEY_STORE"

    if [ -n "$SOLR_SSL_CLIENT_KEY_STORE_PASSWORD" ]; then
      export SOLR_SSL_CLIENT_KEY_STORE_PASSWORD=$SOLR_SSL_CLIENT_KEY_STORE_PASSWORD
    fi
    if [ -n "$SOLR_SSL_CLIENT_KEY_STORE_TYPE" ]; then
      SOLR_SSL_OPTS+=" -Djavax.net.ssl.keyStoreType=$SOLR_SSL_CLIENT_KEY_STORE_TYPE"
    fi
  else
    if [ -n "$SOLR_SSL_KEY_STORE" ]; then
      SOLR_SSL_OPTS+=" -Djavax.net.ssl.keyStore=$SOLR_SSL_KEY_STORE"
    fi
    if [ -n "$SOLR_SSL_KEY_STORE_TYPE" ]; then
      SOLR_SSL_OPTS+=" -Djavax.net.ssl.keyStoreType=$SOLR_SSL_KEY_STORE_TYPE"
    fi
  fi

Thanks,
Rajeswari

On 7/15/20, 1:49 AM, "Natarajan, Rajeswari" <ra...@sap.com> wrote:

    From the <SOLR_HOME>/bin directory I did grep for SOLR_SSL_CLIENT_KEY_STORE , this is what I see . But somehow the option option -Djavax.net.ssl.keyStore is added 
    grep SOLR_SSL_CLIENT_KEY_STORE *
    grep: init.d: Is a directory
    solr:  if [ -n "$SOLR_SSL_CLIENT_KEY_STORE" ]; then
    solr:    SOLR_SSL_OPTS+=" -Djavax.net.ssl.keyStore=$SOLR_SSL_CLIENT_KEY_STORE"
    solr:    if [ -n "$SOLR_SSL_CLIENT_KEY_STORE_PASSWORD" ]; then
    solr:      export SOLR_SSL_CLIENT_KEY_STORE_PASSWORD=$SOLR_SSL_CLIENT_KEY_STORE_PASSWORD
    solr:    if [ -n "$SOLR_SSL_CLIENT_KEY_STORE_TYPE" ]; then
    solr:      SOLR_SSL_OPTS+=" -Djavax.net.ssl.keyStoreType=$SOLR_SSL_CLIENT_KEY_STORE_TYPE"
    solr.cmd:  IF DEFINED SOLR_SSL_CLIENT_KEY_STORE (
    solr.cmd:    set "SOLR_SSL_OPTS=!SOLR_SSL_OPTS! -Djavax.net.ssl.keyStore=%SOLR_SSL_CLIENT_KEY_STORE%"
    solr.cmd:    IF DEFINED SOLR_SSL_CLIENT_KEY_STORE_TYPE (
    solr.cmd:      set "SOLR_SSL_OPTS=!SOLR_SSL_OPTS! -Djavax.net.ssl.keyStoreType=%SOLR_SSL_CLIENT_KEY_STORE_TYPE%"
    solr.in.cmd:REM set SOLR_SSL_CLIENT_KEY_STORE=
    solr.in.cmd:REM set SOLR_SSL_CLIENT_KEY_STORE_PASSWORD=
    solr.in.cmd:REM set SOLR_SSL_CLIENT_KEY_STORE_TYPE=
    solr.in.sh:#SOLR_SSL_CLIENT_KEY_STORE=
    solr.in.sh:#SOLR_SSL_CLIENT_KEY_STORE_PASSWORD=
    solr.in.sh:#SOLR_SSL_CLIENT_KEY_STORE_TYPE=

    Thanks,
    Rajeswari
    On 7/15/20, 12:46 AM, "Natarajan, Rajeswari" <ra...@sap.com> wrote:

        Thank you for your reply. I looked at solr.in.sh I see that  SOLR_SSL_CLIENT_KEY_STORE  is already commented out by default. But you are right I looked at the running solr,  I see the option -Djavax.net.ssl.keyStore pointing to solr-ssl.keystore.p12 , not sure how it is getting that value. Let me dig more. Thanks for the pointer. Also if you have a pointer how it get's populated  other than SOLR_SSL_CLIENT_KEY_STORE config in solr.in.sh , please let me know

        #SOLR_SSL_CLIENT_KEY_STORE=
        #SOLR_SSL_CLIENT_KEY_STORE_PASSWORD=
        #SOLR_SSL_CLIENT_KEY_STORE_TYPE=
        #SOLR_SSL_CLIENT_TRUST_STORE=
        #SOLR_SSL_CLIENT_TRUST_STORE_PASSWORD=
        #SOLR_SSL_CLIENT_TRUST_STORE_TYPE=

        Yes we are not using Solr client auth.

        Thanks,
        Rajeswari

        On 7/14/20, 5:55 PM, "Kevin Risden" <kr...@apache.org> wrote:

            Hmmm so I looked closer - it looks like a side effect of the default
            passthrough of the keystore being passed to the client keystore.

            https://github.com/apache/lucene-solr/blob/master/solr/bin/solr#L229

            Can you remove or commout the entire SOLR_SSL_CLIENT_KEY_STORE section from
            bin/solr or bin/solr.cmd depending on which version you are using? The key
            being to make sure to not set "-Djavax.net.ssl.keyStore".

            This assumes that you aren't using Solr client auth (which based on your
            config you aren't) and you aren't trying to use Solr to connect to anything
            that is secured via clientAuth (most likely you aren't).

            If you can try this and report back that would be awesome. I think this
            will fix the issue and it would be possible to make client auth opt in
            instead of default fall back.
            Kevin Risden



            On Tue, Jul 14, 2020 at 1:46 AM Natarajan, Rajeswari <
            rajeswari.natarajan@sap.com> wrote:

            > Thank you so much for the response.  Below are the configs I have in
            > solr.in.sh and I followed
            > https://lucene.apache.org/solr/guide/8_5/enabling-ssl.html documentation
            >
            > # Enables HTTPS. It is implicitly true if you set SOLR_SSL_KEY_STORE. Use
            > this config
            > # to enable https module with custom jetty configuration.
            > SOLR_SSL_ENABLED=true
            > # Uncomment to set SSL-related system properties
            > # Be sure to update the paths to the correct keystore for your environment
            > SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.p12
            > SOLR_SSL_KEY_STORE_PASSWORD=secret
            > SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.p12
            > SOLR_SSL_TRUST_STORE_PASSWORD=secret
            > # Require clients to authenticate
            > SOLR_SSL_NEED_CLIENT_AUTH=false
            > # Enable clients to authenticate (but not require)
            > SOLR_SSL_WANT_CLIENT_AUTH=false
            > # SSL Certificates contain host/ip "peer name" information that is
            > validated by default. Setting
            > # this to false can be useful to disable these checks when re-using a
            > certificate on many hosts
            > SOLR_SSL_CHECK_PEER_NAME=true
            >
            > In local , with the below certificate it works
            > ---------------------------------------
            >
            > keytool -list -keystore solr-ssl.keystore.p12
            > Enter keystore password:
            > Keystore type: PKCS12
            > Keystore provider: SUN
            >
            > Your keystore contains 1 entry
            >
            > solr-18, Jun 26, 2020, PrivateKeyEntry,
            > Certificate fingerprint (SHA1):
            > AB:F2:C8:84:E8:E7:A2:BF:2D:0D:2F:D3:95:4A:98:5B:2A:88:81:50
            > C02W48C6HTD6:solr-8.5.1 i843100$ keytool -list -v -keystore
            > solr-ssl.keystore.p12
            > Enter keystore password:
            > Keystore type: PKCS12
            > Keystore provider: SUN
            >
            > Your keystore contains 1 entry
            >
            > Alias name: solr-18
            > Creation date: Jun 26, 2020
            > Entry type: PrivateKeyEntry
            > Certificate chain length: 1
            > Certificate[1]:
            > Owner: CN=localhost, OU=Organizational Unit, O=Organization, L=Location,
            > ST=State, C=Country
            > Issuer: CN=localhost, OU=Organizational Unit, O=Organization, L=Location,
            > ST=State, C=Country
            > Serial number: 45a822c8
            > Valid from: Fri Jun 26 00:13:03 PDT 2020 until: Sun Nov 10 23:13:03 PST
            > 2047
            > Certificate fingerprints:
            >          MD5:  0B:80:54:89:44:65:93:07:1F:81:88:8D:EC:BD:38:41
            >          SHA1: AB:F2:C8:84:E8:E7:A2:BF:2D:0D:2F:D3:95:4A:98:5B:2A:88:81:50
            >          SHA256:
            > 9D:65:A6:55:D7:22:B2:72:C2:20:55:66:F8:0C:9C:48:B1:F6:48:40:A4:FB:CB:26:77:DE:C4:97:34:69:25:42
            > Signature algorithm name: SHA256withRSA
            > Subject Public Key Algorithm: 2048-bit RSA key
            > Version: 3
            >
            > Extensions:
            >
            > #1: ObjectId: 2.5.29.17 Criticality=false
            > SubjectAlternativeName [
            >   DNSName: localhost
            >   IPAddress: 172.20.10.4
            >   IPAddress: 127.0.0.1
            > ]
            >
            > #2: ObjectId: 2.5.29.14 Criticality=false
            > SubjectKeyIdentifier [
            > KeyIdentifier [
            > 0000: 1B 6F BB 65 A4 3C 6A F4   C9 05 08 89 88 0E 9E 76  .o.e.<j........v
            > 0010: A1 B7 28 BE                                        ..(.
            > ]
            >
            > /////////////////////////////////////////////////////////////////
            > In a cluster env , where the deployment  , keystore everything is
            > automated  (used by  multiple teams) keystore generated is as below. As you
            > can see the  keystore has 2 certificates , in which case I get the
            > exception  below.
            >
            > java.lang.UnsupportedOperationException: X509ExtendedKeyManager only
            >     > supported on Server
            >     >               at
            >     >
            > org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:223)
            >     >
            >
            > In both cases , the config is same except the keystore certificates . In
            > the JIRA (https://issues.apache.org/jira/browse/SOLR-14105) , I see the
            > fix says it supports multiple DNS and multiple certificates. So I thought
            > it should be ok. Please let me know .
            >
            > keytool -list -keystore  /etc/nginx/certs/sidecar.p12
            > Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
            > Enter keystore password:
            > Keystore type: PKCS12
            > Keystore provider: SUN
            >
            > Your keystore contains 1 entry
            >
            > 1, Jul 7, 2020, PrivateKeyEntry,
            > Certificate fingerprint (SHA1):
            > E2:3B:4B:4A:0E:05:CF:DA:59:09:55:8D:4E:6D:8A:1D:4E:DD:D4:62
            > bash-5.0#
            > ————————-
            >
            > bash-5.0#  keytool -list -v -keystore /etc/nginx/certs/sidecar.p12
            > Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
            > Enter keystore password:
            > Keystore type: PKCS12
            > Keystore provider: SUN
            >
            > Your keystore contains 1 entry
            >
            > Alias name: 1
            > Creation date: Jul 7, 2020
            > Entry type: PrivateKeyEntry
            > Certificate chain length: 2
            > Certificate[1]:
            > Owner: OU=Cobalt, O=SAP, L=Walldorf, ST=Walldorf, C=DE
            > Issuer: CN=SAP Ariba Cobalt Sidecar Intermediate CA, OU=COBALT, O=SAP
            > Ariba, ST=CA, C=US
            > Serial number: 1000
            > Valid from: Tue Jul 07 05:14:37 GMT 2020 until: Thu Jul 07 05:14:37 GMT
            > 2022
            > Certificate fingerprints:
            >          MD5:  C0:13:87:37:96:C2:E2:DD:B9:D7:B4:E3:6B:73:A0:EC
            >          SHA1: E2:3B:4B:4A:0E:05:CF:DA:59:09:55:8D:4E:6D:8A:1D:4E:DD:D4:62
            >          SHA256:
            > 89:AB:8E:3B:D4:EC:A6:D0:0E:D7:CB:65:8C:92:13:32:F2:FD:7E:41:C9:39:F5:66:D5:7D:F1:04:13:8A:4E:92
            > Signature algorithm name: SHA256withRSA
            > Subject Public Key Algorithm: 2048-bit RSA key
            > Version: 3
            >
            > Extensions:
            >
            > #1: ObjectId: 2.16.840.1.113730.1.13 Criticality=false
            > 0000: 16 24 4F 70 65 6E 53 53   4C 20 47 65 6E 65 72 61  .$OpenSSL Genera
            > 0010: 74 65 64 20 53 65 72 76   65 72 20 43 65 72 74 69  ted Server Certi
            > 0020: 66 69 63 61 74 65                                  ficate
            >
            >
            > #2: ObjectId: 2.5.29.35 Criticality=false
            > AuthorityKeyIdentifier [
            > KeyIdentifier [
            > 0000: E9 5C 42 72 5E 70 D9 02   05 AA 11 BA 0D 4D 8D 0D  .\Br^p.......M..
            > 0010: F3 37 2C 95                                        .7,.
            > ]
            > [CN=SAP Ariba Cobalt CA, OU=ES, O=SAP Ariba, L=Palo Alto, ST=CA, C=US]
            > SerialNumber: [    1001]
            > ]
            >
            > #3: ObjectId: 2.5.29.19 Criticality=false
            > BasicConstraints:[
            >   CA:false
            >   PathLen: undefined
            > ]
            >
            > #4: ObjectId: 2.5.29.37 Criticality=false
            > ExtendedKeyUsages [
            >   serverAuth
            > ]
            >
            > #5: ObjectId: 2.5.29.15 Criticality=true
            > KeyUsage [
            >   DigitalSignature
            >   Key_Encipherment
            > ]
            >
            > #6: ObjectId: 2.16.840.1.113730.1.1 Criticality=false
            > NetscapeCertType [
            >    SSL server
            > ]
            >
            > #7: ObjectId: 2.5.29.17 Criticality=false
            > SubjectAlternativeName [
            >   DNSName: search-solrcloud-solrcloud.service
            >   DNSName: search-solrcloud-solrcloud.service.mu.aws.ariba.com
            >   DNSName: *.query.mu.aws.ariba.com
            >   DNSName: *.query
            >   DNSName: *.service
            >   DNSName:
            > e046469b-1bb0-55f6-913f-bd6d52b238a8.search-solrcloud-solrcloud.service.mu.aws.ariba.com
            >   DNSName:
            > e046469b-1bb0-55f6-913f-bd6d52b238a8.search-solrcloud-solrcloud.service
            >   DNSName: *.service.mu.aws.ariba.com
            >   DNSName: 1.search-solrcloud-solrcloud.service.mu.aws.ariba.com
            >   DNSName: 1.search-solrcloud-solrcloud.service
            >   DNSName: localhost
            >   IPAddress: 10.1.56.9
            >   IPAddress: 10.169.50.16
            >   IPAddress: 127.0.0.1
            > ]
            >
            > #8: ObjectId: 2.5.29.14 Criticality=false
            > SubjectKeyIdentifier [
            > KeyIdentifier [
            > 0000: 3F 9D 3D 24 48 1E 61 3C   BD C0 A4 07 8B 64 51 0D  ?.=$H.a<.....dQ.
            > 0010: A2 B2 FE 89                                        ....
            > ]
            > ]
            >
            > Certificate[2]:
            > Owner: CN=SAP Ariba Cobalt Sidecar Intermediate CA, OU=COBALT, O=SAP
            > Ariba, ST=CA, C=US
            > Issuer: CN=SAP Ariba Cobalt CA, OU=ES, O=SAP Ariba, L=Palo Alto, ST=CA,
            > C=US
            > Serial number: 1001
            > Valid from: Thu Apr 16 07:18:55 GMT 2020 until: Sun Apr 14 07:18:55 GMT
            > 2030
            > Certificate fingerprints:
            >          MD5:  FA:70:2F:DB:63:36:66:71:A6:7B:0F:46:F3:52:0B:3C
            >          SHA1: 4F:27:D3:E3:12:24:64:18:B5:97:D0:BF:94:37:2D:5C:33:EA:1E:40
            >          SHA256:
            > 15:28:F4:DB:B3:D5:2E:21:6A:2E:56:47:E3:6B:D3:16:96:18:06:96:DA:5D:28:6B:34:CB:6D:FA:E8:FA:85:13
            > Signature algorithm name: SHA256withRSA
            > Subject Public Key Algorithm: 4096-bit RSA key
            > Version: 3
            >
            > Extensions:
            >
            > #1: ObjectId: 2.5.29.35 Criticality=false
            > AuthorityKeyIdentifier [
            > KeyIdentifier [
            > 0000: D8 A1 D1 11 50 8C 1C 2A   67 69 82 40 DF B5 68 6A  ....P..*gi.@..hj
            > 0010: E4 97 6E 32                                        ..n2
            > ]
            > ]
            >
            > #2: ObjectId: 2.5.29.19 Criticality=true
            > BasicConstraints:[
            >   CA:true
            >   PathLen:0
            > ]
            >
            > #3: ObjectId: 2.5.29.15 Criticality=true
            > KeyUsage [
            >   DigitalSignature
            >   Key_CertSign
            >   Crl_Sign
            > ]
            >
            > #4: ObjectId: 2.5.29.14 Criticality=false
            > SubjectKeyIdentifier [
            > KeyIdentifier [
            > 0000: E9 5C 42 72 5E 70 D9 02   05 AA 11 BA 0D 4D 8D 0D  .\Br^p.......M..
            > 0010: F3 37 2C 95                                        .7,.
            > ]
            > ]
            >
            >
            > Thanks,
            > Rajeswari
            >
            > On 7/13/20, 2:16 PM, "Kevin Risden" <kr...@apache.org> wrote:
            >
            >     >
            >     > In local with just certificate and one domain name  the SSL
            > communication
            >     > worked. With multiple DNS and 2 certificates SSL fails with below
            > exception.
            >     >
            >
            >     A client keystore by definition can only have a single certificate. A
            >     server keystore can have multiple certificates. The reason being is
            > that a
            >     client can only be identified by a single certificate.
            >
            >     Can you share more details about specifically what your solr.in.sh
            > configs
            >     look like related to keystore/truststore and which files? Specifically
            >     highlight which files have multiple certificates in them.
            >
            >     It looks like for the Solr internal http client, the client keystore
            > has
            >     more than one certificate in it and the error is correct. This is more
            >     strict with recent versions of Jetty 9.4.x. Previously this would
            > silently
            >     fail, but was still incorrect. Now the error is bubbled up so that
            > there is
            >     no silent misconfigurations.
            >
            >     Kevin Risden
            >
            >
            >     On Mon, Jul 13, 2020 at 4:54 PM Natarajan, Rajeswari <
            >     rajeswari.natarajan@sap.com> wrote:
            >
            >     > I looked at the patch mentioned in the JIRA
            >     > https://issues.apache.org/jira/browse/SOLR-14105  reporting the
            > below
            >     > issue. I looked at the solr 8.5.1 code base , I see the patch is
            > applied.
            >     > But still seeing the same  exception with different stack trace. The
            >     > initial excsption stacktrace was at
            >     >
            >     > at
            >     >
            > org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245)
            >     >
            >     >
            >     > Now the exception we encounter is at httpsolrclient creation
            >     >
            >     >
            >     >         Caused by: java.lang.RuntimeException:
            >     > java.lang.UnsupportedOperationException: X509ExtendedKeyManager only
            >     > supported on Server
            >     >               at
            >     >
            > org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:223)
            >     >
            >     > I commented the JIRA also. Let me know if this is still an issue.
            >     >
            >     > Thanks,
            >     > Rajeswari
            >     >
            >     > On 7/13/20, 2:03 AM, "Natarajan, Rajeswari" <
            > rajeswari.natarajan@sap.com>
            >     > wrote:
            >     >
            >     >     Re-sending to see if anyone encountered  had this combination and
            >     > encountered this issue. In local with just certificate and one
            > domain name
            >     > the SSL communication worked. With multiple DNS and 2 certificates
            > SSL
            >     > fails with below exception.  Below JIRA says it is fixed for
            >     > Http2SolrClient , wondering if this is fixed for http1 solr client
            > as we
            >     > pass -Dsolr.http1=true .
            >     >
            >     >     Thanks,
            >     >     Rajeswari
            >     >
            >     >     https://issues.apache.org/jira/browse/SOLR-14105
            >     >
            >     >     On 7/6/20, 10:02 PM, "Natarajan, Rajeswari" <
            >     > rajeswari.natarajan@sap.com> wrote:
            >     >
            >     >         Hi,
            >     >
            >     >         We are using Solr 8.5.1 in cloud mode  with Java 8. We are
            >     > enabling  TLS  with http1  (as we get a warning java 8 + solr 8.5
            > SSL can’t
            >     > be enabled) and we get below exception
            >     >
            >     >
            >     >
            >     >         2020-07-07 03:58:53.078 ERROR (main) [   ] o.a.s.c.SolrCore
            >     > null:org.apache.solr.common.SolrException: Error instantiating
            >     > shardHandlerFactory class [HttpShardHandlerFactory]:
            >     > java.lang.UnsupportedOperationException: X509ExtendedKeyManager only
            >     > supported on Server
            >     >               at
            >     >
            > org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:56)
            >     >               at
            >     > org.apache.solr.core.CoreContainer.load(CoreContainer.java:647)
            >     >               at
            >     >
            > org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:263)
            >     >               at
            >     >
            > org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:183)
            >     >               at
            >     >
            > org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:134)
            >     >               at
            >     >
            > org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:751)
            >     >               at
            >     >
            > java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
            >     >               at
            >     >
            > java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742)
            >     >               at
            >     >
            > java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742)
            >     >               at
            >     >
            > java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
            >     >               at
            >     >
            > org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744)
            >     >               at
            >     >
            > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:360)
            >     >               at
            >     >
            > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1445)
            >     >               at
            >     >
            > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1409)
            >     >               at
            >     >
            > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:822)
            >     >               at
            >     >
            > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:275)
            >     >               at
            >     >
            > org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
            >     >               at
            >     >
            > org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:46)
            >     >               at
            >     >
            > org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:188)
            >     >               at
            >     >
            > org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:513)
            >     >               at
            >     >
            > org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:154)
            >     >               at
            >     >
            > org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:173)
            >     >               at
            >     >
            > org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:447)
            >     >               at
            >     >
            > org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:66)
            >     >               at
            >     > org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:784)
            >     >               at
            >     > org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:753)
            >     >               at
            > org.eclipse.jetty.util.Scanner.scan(Scanner.java:641)
            >     >               at
            > org.eclipse.jetty.util.Scanner.doStart(Scanner.java:540)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
            >     >               at
            >     >
            > org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:146)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
            >     >               at
            >     >
            > org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:599)
            >     >               at
            >     >
            > org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:249)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
            >     >               at
            > org.eclipse.jetty.server.Server.start(Server.java:407)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
            >     >               at
            >     >
            > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:100)
            >     >               at
            > org.eclipse.jetty.server.Server.doStart(Server.java:371)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
            >     >               at
            >     >
            > org.eclipse.jetty.xml.XmlConfiguration.lambda$main$0(XmlConfiguration.java:1888)
            >     >               at java.security.AccessController.doPrivileged(Native
            > Method)
            >     >               at
            >     >
            > org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1837)
            >     >               at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
            >     > Method)
            >     >               at
            >     >
            > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            >     >               at
            >     >
            > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            >     >               at java.lang.reflect.Method.invoke(Method.java:498)
            >     >               at
            > org.eclipse.jetty.start.Main.invokeMain(Main.java:218)
            >     >               at org.eclipse.jetty.start.Main.start(Main.java:491)
            >     >               at org.eclipse.jetty.start.Main.main(Main.java:77)
            >     >         Caused by: java.lang.RuntimeException:
            >     > java.lang.UnsupportedOperationException: X509ExtendedKeyManager only
            >     > supported on Server
            >     >               at
            >     >
            > org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:223)
            >     >               at
            >     >
            > org.apache.solr.client.solrj.impl.Http2SolrClient.<init>(Http2SolrClient.java:153)
            >     >               at
            >     >
            > org.apache.solr.client.solrj.impl.Http2SolrClient$Builder.build(Http2SolrClient.java:832)
            >     >               at
            >     >
            > org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:321)
            >     >               at
            >     >
            > org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:51)
            >     >               ... 50 more
            >     >         Caused by: java.lang.UnsupportedOperationException:
            >     > X509ExtendedKeyManager only supported on Server
            >     >               at
            >     >
            > org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1273)
            >     >               at
            >     >
            > org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1255)
            >     >               at
            >     >
            > org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374)
            >     >               at
            >     >
            > org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
            >     >               at
            >     > org.eclipse.jetty.client.HttpClient.doStart(HttpClient.java:244)
            >     >               at
            >     >
            > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
            >     >               at
            >     >
            > org.apache.solr.client.solrj.impl.Http2SolrClient.createHttpClient(Http2SolrClient.java:221)
            >     >               ... 54 more
            >     >
            >     >
            >     >         I see that there is a  below bug for this issue and is
            > resolved.
            >     > So I am not sure what will the cause of the issue.
            >     >
            >     >         https://issues.apache.org/jira/browse/SOLR-14105
            >     >
            >     >
            >     >         Thanks,
            >     >         Rajeswari
            >     >
            >     >
            >     >
            >
            >