Re: grpc request error -- JMeter 4.0

[Prateek Dua <du...@gmail.com>]

Hi Everyone,

Many thanks for your continuous support & help and due to that, I'am able
to publish a blog on said problem on both linked in & medium..

https://www.linkedin.com/posts/prateek-dua-5304aaa_handling-grpc-proto-requests-with-ssl-certificate-activity-6816342777505320960-el2E

Thanks,
Prateek

On Mon, May 31, 2021 at 9:14 PM Prateek Dua <du...@gmail.com> wrote:

> Hi Felix,
>
> Many thanks for your help & valuable suggestions as that really helped in
> resolving the problem in sending grpc requests to JMeter. So as you rightly
> said, ca.cert was enough to be shared in truststore entry of JMeter.
>
> But problem was further getting created due to token format getting passed
> in metadata field of grpc request. It was getting passed in string format (
> " ") + carrying curly '{ }' braces scoping which was failing request to be
> triggered to Server. So post giving token in this format *authorization:Token
> 070eeb***** , request is giving required response and we can say current
> GRPC plugin is defect free to be used for sending grpc requests  :)
>
> And thanks to you Scott Dean too for your quick inputs related to Keystore
> handling.
>
> Regards,
> Prateek
>
> On Thu, May 27, 2021 at 12:37 AM Felix Schumacher <
> felix.schumacher@internetallee.de> wrote:
>
>> Hi Prateek,
>>
>> given the list of keys/certs below, I think the only cert you need is the
>> ca.crt. That cert should be placed in the truststore.
>>
>> The server key/cert is probably used on the server side and has no
>> relevance for JMeter (as far as the installation/usage is concerned).
>>
>> It is normal, that only the server has a fixed private key (server.key
>> below) and a corresponding certificate (server.crt below), that has been
>> signed by a CA (ca.key below) and can be verified by a CA cert (ca.crt
>> below).
>>
>> The clients will often use other means to authenticate themselves to the
>> server, for example by providing a username and password (over the secured
>> connection). The client can be sure to talk to a server securely, if
>>
>>  * the hostname information in the server certificate matches the
>> hostname of the server it wanted to talk to
>>  * the server certificate was signed by a CA, that the client trusts
>>
>> A client *can* authenticate itself to a server by using a client
>> certificate (which is basically the same as a server certificate, a pair of
>> private and public keys with the public key signed by a CA). However that
>> scenario is rare.
>>
>> It is probably best to find someone administering the server, who can
>> tell you, what triggered the 500 response or can tell you, which
>> authentication method is required.
>>
>> Felix
>> Am 26.05.21 um 13:05 schrieb Prateek Dua:
>>
>> Hi Felix,
>>
>> Sharing below keys I've , please help in signifying the required keys to
>> be configured at JMeter's end. I'm aware of 2 keys -- one is '*ca.crt' *to
>> be given in JMeter's truststore as .jks and another is *'server.key' *to
>> be configured at keystore as PKCS12..
>>
>> Is there any other key to be configured at JMeter client ... Please help
>> in knowing the same.
>>
>> [image: Screenshot 2021-05-26 at 3.15.47 PM.png]
>>
>>
>> Thanks,
>> Prateek
>>
>> On Mon, May 24, 2021 at 6:58 PM Prateek Dua <du...@gmail.com> wrote:
>>
>>> Hi Felix,
>>>
>>> My apologies here for the delayed response -- wasn't doing much well in
>>> terms of health last week. So to summarise the things per your mail, I
>>> would require following things to make my request to hit server without any
>>> miss..
>>>
>>> 1. CA Certificate ( truststore certificate) to be
>>> configured in javax.net.ssl.trustStore entries.
>>>
>>> 2. Private key to use the cert for authentication.
>>>
>>> 3*.* Unable to understand this as mentioned in your mail ....
>>>
>>> "*<< If you want to go a step further and sign in with a certificate,
>>> you have to give the client (JMeter) the private key and the certificate of
>>> the users cert and configure the client to actually use it. >>"*
>>>
>>> *"and the certificate of the users cert"*  -- *which certificate
>>> you're referring to here ?*
>>>
>>>
>>> Thanks,
>>> Prateek
>>>
>>> On Sun, May 16, 2021 at 4:58 PM Felix Schumacher <
>>> felix.schumacher@internetallee.de> wrote:
>>>
>>>> Hi Prateek,
>>>>
>>>> I think that you still mix the different types of certificates and
>>>> their usage.
>>>>
>>>> Certificates (or rather public certificates) always have a counterpart,
>>>> which is their private key. That pair can be used to encrypt data, sign
>>>> stuff (which is really encrypting known data) or assure identities (which
>>>> can be thought of as possessing the private key to the cert).
>>>>
>>>> In your scenario, JMeter can be thought of as a *client*, which you -
>>>> as far as I have understood - want to use with a *client* certificate
>>>> (which - in this szenario - should be a private key and a public cert
>>>> stored in a keystore).
>>>>
>>>> When you place the certificates (the public part) in the trust store,
>>>> you could verify, if your communication partner is in possession of the
>>>> private key and therefore assure their identity. That is probably not what
>>>> you wanted.
>>>>
>>>> Usually you place a CA certificate into your trust store (on the client
>>>> and on the server side) and use the CA certificate to verify the root chain
>>>> of the certificate presented by the server to validate the hostname of that
>>>> server. In JMeter you put such a CA certificate into the truststore, that
>>>> you configured per javax.net.ssl.trustStore entries.
>>>>
>>>> If you want to go a step further and sign in with a certificate, you
>>>> have to give the client (JMeter) the private key and the certificate of the
>>>> users cert and configure the client to actually use it.
>>>>
>>>> So make sure, that you know, which of the two you need. If you want to
>>>> use the cert for authentication, you *need* the corresponding private key.
>>>> If you want the make sure, you reached the correct server and no one can
>>>> sniff your traffic, you have to have the CA certificate or the server
>>>> certificate in your trust store.
>>>>
>>>> Maybe someone who gave you the task to test the service can help you
>>>> with sorting those questions out?
>>>>
>>>> Felix
>>>> Am 11.05.21 um 20:28 schrieb Prateek Dua:
>>>>
>>>> Hi Felix,
>>>>
>>>> Thanks for your inputs and per that I did following changes from my end
>>>> here. ..
>>>>
>>>> 1. Switched to JMeter latest version 5.4.1.
>>>>
>>>> 2. Since original certificate is of truststore type, I provided
>>>> converted .jks to *Truststore properties* (* instead of Keystore
>>>> properties*) in *'System.properties'* file of JMeter as per below..
>>>>
>>>>
>>>> javax.net.ssl.trustStore=/Users/prateekdua/Downloads/crt/JMeterjks
>>>> javax.net.ssl.trustStorePassword=******
>>>> javax.net.ssl.trustStoreProvider
>>>> javax.net.ssl.trustStoreType=jks
>>>>
>>>> 3.Enabled javax.net.debug=ssl to trace the certificate changes.. And
>>>> per that, following output is coming --  *https://imgur.com/a/SwueHMY
>>>> <https://imgur.com/a/SwueHMY>*
>>>>
>>>> [image: Screenshot 2021-05-11 at 7.34.59 PM.png]
>>>>
>>>> 4. Output shows certificate got added with message *"trigger seeding
>>>> of SecureRandom..done seeding SecureRandom*" but still grpc is showing
>>>> error (500, Exception: io.grpc.StatusRuntimeException: UNKNOWN).
>>>>
>>>> 5. For error logs, I'll check with Dev team if they can help but before
>>>> that just want to confirm -- if output shown in shared screenshot with
>>>> messages is validating certificate or still there is some problem".
>>>>
>>>> Thanks,
>>>> Prateek
>>>>
>>>> On Thu, Apr 29, 2021 at 1:12 AM Felix Schumacher <
>>>> felix.schumacher@internetallee.de> wrote:
>>>>
>>>>>
>>>>> Am 28.04.21 um 19:59 schrieb Prateek Dua:
>>>>> > Hi Felix,
>>>>> >
>>>>> > Please find response to your queries.
>>>>> >
>>>>> > How did you add the certificate (note, that it has to include the
>>>>> private
>>>>> > key, too) into your (configured) keystore?
>>>>> >
>>>>> > 1. I used tool keystore Explorer to convert original client
>>>>> certificate (
>>>>> > abc.crt) to 'mycert.PKCS#12' . Further I converted mycert.PKCS#12'
>>>>> to .jks
>>>>> > as 'JMeterjks.jks' using below cmd
>>>>> > keytool -importkeystore -srckeystore certificate.p12 -srcstoretype
>>>>> PKCS12
>>>>> > -srcstorepass <certificate_password> -keystore <keystore_filename>
>>>>> > -storepass <stored_password>
>>>>> >
>>>>> > 2. Then I configured below lines of 'system.properties' file to input
>>>>> > JMeterjks.jks details
>>>>> >
>>>>> > # Keystore properties (client certificates)
>>>>> > # Location
>>>>> > javax.net.ssl.keyStore=/Users/prateekdua/Downloads/crt/JMeterjks.jks'
>>>>> > #The password to your keystore
>>>>> > javax.net.ssl.keyStorePassword=^^^^^@^^^^
>>>>> >
>>>>> > 3. Added user defined variable *'*certi_val' bearing alias
>>>>> 'jmeter_certi'
>>>>> > as value of it under Test plan. And giving the same variable name
>>>>> > 'certi_val' into 'Variable name holding certificate alias' of
>>>>> Keystore
>>>>> > config component of JMeter
>>>>> >
>>>>> >
>>>>> > xCan you list the entries in the keystore and show it to us?
>>>>> >
>>>>> > It's basically a public key. Please find the key details.
>>>>> >
>>>>> >
>>>>> > GO-MAC-PRATEEKDUA:~ prateekdua$ keytool -list -keystore
>>>>> > /Users/prateekdua/Downloads/crt/JMeterjks
>>>>> >
>>>>> > Enter keystore password:
>>>>> >
>>>>> >
>>>>> > Keystore type: JKS
>>>>> >
>>>>> > Keystore provider: SUN
>>>>> >
>>>>> >
>>>>> > Your keystore contains 1 entry
>>>>> >
>>>>> >
>>>>> > jmeter_certi, 28 Apr, 2021, trustedCertEntry,
>>>>> >
>>>>> > Certificate fingerprint (SHA1):
>>>>> > E8:AB:4D:7B:46:98:8E:5E:FD:36:83:C4:A9:D0:83:C3:E1:76:FD:84
>>>>>
>>>>> I am not sure, you are looking at the right end of the problem.
>>>>>
>>>>> If you have a trusted cert (and not a private key plus its
>>>>> corresponding
>>>>> cert), you don't need a key store, but a trust store.
>>>>>
>>>>> Having a client cert without a private key is almost useless (as a
>>>>> client), as you can't use it to authenticate yourself with it.
>>>>>
>>>>> The name of the cert you posted in some mails earlier indicated, that
>>>>> it
>>>>> was a server (wildcard) certificate and not a client certificate.
>>>>>
>>>>> Are you sure, that you need a cert to authenticate? Maybe you (only)
>>>>> want to certify, that you are talking to the right server. (then you
>>>>> would need either the public key of the server or better the public
>>>>> cert
>>>>> of the CA that signed the servers public key)
>>>>>
>>>>> To get back to you original problem. You showed an error (500,
>>>>> Exception: io.grpc.StatusRuntimeException: UNKNOWN) that came from the
>>>>> server. Did you have a look at the server logs to find out more about
>>>>> the problem the server had with your request?
>>>>>
>>>>> Felix
>>>>>
>>>>> >
>>>>> >
>>>>> >
>>>>> > Thanks,
>>>>> > Prateek
>>>>> >
>>>>> > On Wed, Apr 28, 2021 at 6:57 PM Felix Schumacher <
>>>>> > felix.schumacher@internetallee.de> wrote:
>>>>> >
>>>>> >> Am 25.04.21 um 18:27 schrieb Prateek Dua:
>>>>> >>
>>>>> >> Hi Felix,
>>>>> >>
>>>>> >> Thanks for your valuable suggestion. I did same earlier by updating
>>>>> to
>>>>> >> guava.23.0 jar and resolved this error. But now I'm getting new
>>>>> error while
>>>>> >> hitting grpc request -- good thing is that grpc based requests are
>>>>> getting
>>>>> >> created now..
>>>>> >>
>>>>> >> Response code: 500
>>>>> >> Response message: Exception: io.grpc.StatusRuntimeException: UNKNOWN
>>>>> >> Caught exception while waiting for rpc
>>>>> >>
>>>>> >> My request is unable to reach out to server handling grpc request
>>>>> via
>>>>> >> JMeter. I cross-checked it using another application *BloomRPC
>>>>> clien*t
>>>>> >> for validating grpc based requests and it's working fine through it.
>>>>> >>
>>>>> >> Is there any problem related to Client side Certificate ( .crt file
>>>>> needed
>>>>> >> to validate server & request for client) which is not getting
>>>>> correctly
>>>>> >> handled in my case via JMeter.* I did following stuff to
>>>>> >> use certificate using JMeter*
>>>>> >>
>>>>> >> Converted .crt based certificate to .jks and sconfigured in below
>>>>> paths
>>>>> >> with required password..
>>>>> >>
>>>>> >>  *1. system.properties file of JMeter*
>>>>> >>
>>>>> >> # Keystore properties (client certificates)
>>>>> >> # Location
>>>>> >>
>>>>> >>
>>>>> *javax.net.ssl.keyStore=/Users/prateekdua/Downloads/proto_certificate/test.jks*
>>>>> >> #
>>>>> >> #The password to your keystore
>>>>> >> *javax.net.ssl.keyStorePassword=^^^^^@^^^^*
>>>>> >> #
>>>>> >> #javax.net.ssl.keyStoreProvider
>>>>> >> #javax.net.ssl.keyStoreType [default = KeyStore.getDefaultType()]
>>>>> >>
>>>>> >> 2. Added .jks certificate as *'jssecacerts'* file under
>>>>> >> lib/security/jssecacerts path of installed java.
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> /Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home/jre/lib/security/jssecacerts
>>>>> >>
>>>>> >> 3. Configured the same *'jssecacerts' *file path too in *s*
>>>>> >> *ystem.properties* file of JMeter for following lines..
>>>>> >>
>>>>> >> # Truststore properties (trusted certificates)
>>>>> >>
>>>>> >>
>>>>> *javax.net.ssl.trustStore=/Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home/jre/lib/security/jssecacerts
>>>>> >> javax.net.ssl.trustStorePassword= *^^^^^@^^^^
>>>>> >> #javax.net.ssl.trustStoreProvider
>>>>> >> #javax.net.ssl.trustStoreType [default = KeyStore.getDefaultType()]
>>>>> >>
>>>>> >> But still getting above highlighted error in red..
>>>>> >>
>>>>> >> How did you add the certificate (note, that it has to include the
>>>>> private
>>>>> >> key, too) into your (configured) keystore?
>>>>> >>
>>>>> >> Can you list the entries in the keystore and show it to us?
>>>>> >>
>>>>> >> For example, when I use the keystore for RMI that can be generated
>>>>> by a
>>>>> >> shell script in the bin folder, I would use the following command
>>>>> and
>>>>> >> expect some output like
>>>>> >>
>>>>> >> $ keytool -list -keystore bin/rmi_keystore.jks
>>>>> >> Enter keystore password:
>>>>> >> Keystore type: PKCS12
>>>>> >> Keystore provider: SUN
>>>>> >>
>>>>> >> Your keystore contains 1 entry
>>>>> >>
>>>>> >> rmi, Jan 23, 2018, PrivateKeyEntry,
>>>>> >> Certificate fingerprint (SHA1):
>>>>> >> D7:E4:FA:61:8F:52:8D:BA:09:10:D5:4F:F1:21:C6:AA:5C:10:0D:38
>>>>> >>
>>>>> >>
>>>>> >> The alias of the client certifficate (Note the PrivateKeyEntry) is
>>>>> "rmi".
>>>>> >>
>>>>> >> If you want to use the KeyStore Configuration, that value (rmi) has
>>>>> to be
>>>>> >> put into a variable (for example "cert_name") and the string
>>>>> "cert_name"
>>>>> >> would have to placed in the KeyStore Configuration field "Variable
>>>>> name
>>>>> >> holding certificate alias". That configuration is meant to be used,
>>>>> if you
>>>>> >> have more than one alias (read key/certificate) in your keystore
>>>>> and want
>>>>> >> to change the usage of the certificate based on variables in the
>>>>> plan.
>>>>> >>
>>>>> >> If you have only one certificate, that might be a bit overkill.
>>>>> >>
>>>>> >> Also note, that the KeyStore Configuration is meant to be used with
>>>>> the
>>>>> >> HTTP Sampler from JMeter (especially the HTTPClient 4
>>>>> implementation). Make
>>>>> >> sure, that it works with your sampler (by asking the third party
>>>>> plugin
>>>>> >> developer?).
>>>>> >>
>>>>> >> Regards
>>>>> >>
>>>>> >>  Felix
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> Thanks,
>>>>> >> Prateek
>>>>> >>
>>>>> >> On Sat, Apr 24, 2021 at 4:40 PM Felix Schumacher <
>>>>> >> felix.schumacher@internetallee.de> wrote:
>>>>> >>
>>>>> >>> Am 20.04.21 um 09:34 schrieb Prateek Dua:
>>>>> >>>
>>>>> >>> Hi Team,
>>>>> >>>
>>>>> >>> Need your help here in sorting this out problem related to hitting
>>>>> grpc
>>>>> >>> based requests where I'm receiving below error...
>>>>> >>>
>>>>> >>> 2021-04-19 14:07:41,552 ERROR o.a.j.JMeter: Uncaught exception:
>>>>> >>> *java.lang.NoSuchMethodError:
>>>>> >>> com.google.common.net.HostAndPort.getHost()Ljava/lang/String;*
>>>>> >>> at
>>>>> >>>
>>>>> vn.zalopay.benchmark.core.grpc.ChannelFactory.createChannelBuilder(ChannelFactory.java:34)
>>>>> >>> ~[jmeter-grpc-request-1.1.2.jar:?]
>>>>> >>>
>>>>> >>>
>>>>> >>> This is a third party plugin, so you should try to get help by the
>>>>> >>> authors of that plugin.
>>>>> >>>
>>>>> >>> That said. If you google for "java.lang.NoSuchMethodError:
>>>>> >>> com.google.common.net.HostAndPort.getHost()Ljava/lang/String",you
>>>>> >>> probably would have found links similar to the bug report
>>>>> >>> https://github.com/lightbody/browsermob-proxy/issues/398. The
>>>>> plugin
>>>>> >>> seems to depend on a newer version of guava, than is available in
>>>>> your
>>>>> >>> classpath.
>>>>> >>>
>>>>> >>> I would try to locate a jar named guava*.jar in lib/** and update
>>>>> it to a
>>>>> >>> newer one. Beware, I haven't used that plugin and haven't tried the
>>>>> >>> solution.
>>>>> >>>
>>>>> >>> Felix
>>>>> >>>
>>>>> >>>
>>>>> >>> Please find attached my test plan and error details..
>>>>> >>>
>>>>> >>> Screenshots uploaded to https://imgur.com/a/toJHQNR
>>>>> >>>
>>>>> >>> [image: Screenshot 2021-04-19 at 2.26.04 PM.png]
>>>>> >>>
>>>>> >>> *Following steps I've done for setting up grpc based requests for
>>>>> my
>>>>> >>> JMeter 4.0*
>>>>> >>>
>>>>> >>> 1. Added jar file  'jmeter-grpc-request-1.1.2.jar' under ext/lib
>>>>> folder.
>>>>> >>> Got this jar file from *
>>>>> https://github.com/zalopay-oss/jmeter-grpc-request/tree/master/dist/bin
>>>>> >>> <
>>>>> https://github.com/zalopay-oss/jmeter-grpc-request/tree/master/dist/bin>.
>>>>> *
>>>>> >>>
>>>>> >>> 2*.  *grpc-client-1.2.0.jar , grpc-protobuf-1.1.2.jar
>>>>> >>> , grpc-stub-1.1.2.jar are the pred added grpc jar files present
>>>>> >>> under ext/lib folder.
>>>>> >>>
>>>>> >>> 3. There is extra client side certificate *"**abc.crt"* shared by
>>>>> client
>>>>> >>> that is needed for authenticating current grpc requests. So I
>>>>> converted it
>>>>> >>> to *.jks *using below *keytool *command..
>>>>> >>>
>>>>> >>> *keytool -*import -alias alias -file
>>>>> >>> /Users/prateekdua/Downloads/abc.crt  -keypass keypass -keystore
>>>>> >>> yourkeystore.jks -storepass sub
>>>>> >>>
>>>>> >>> 4. And I've stored converted *.jks *certificate inside bin folder
>>>>> of
>>>>> >>> JMeter.
>>>>> >>>
>>>>> >>> 5. Configured *system.properties* file of JMeter for following
>>>>> lines
>>>>> >>>
>>>>> >>> # Keystore properties (client certificates)
>>>>> >>> # Location
>>>>> >>>
>>>>> >>>
>>>>> javax.net.ssl.keyStore=/Users/prateekdua/Downloads/apache-jmeter-4.0/bin/aws-ecs-mmt.jks
>>>>> >>>
>>>>> >>> #The password to your keystore
>>>>> >>> javax.net.ssl.keyStorePassword=mypassword
>>>>> >>>
>>>>> >>>
>>>>> >>> Thanks,
>>>>> >>> Prateek
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>
>>>>>
>>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> Prateek Dua
>>>>
>>>>
>>>
>>> --
>>> Thanks,
>>> Prateek Dua
>>>
>>
>>
>> --
>> Thanks,
>> Prateek Dua
>>
>>
>
> --
> Thanks,
> Prateek Dua
>


-- 
Thanks,
Prateek Dua