RE: Livy GET requests work but POST requests fail with "401 Authentication required"

["Partridge, Lucas (GE Aviation)" <Lu...@ge.com>]

For anyone who’s interested I posted a solution at https://stackoverflow.com/a/47079736/1843329

From: Partridge, Lucas (GE Aviation)
Sent: 25 October 2017 14:53
To: user@livy.incubator.apache.org
Subject: EXT: Livy GET requests work but POST requests fail with "401 Authentication required"

Thanks for the pointers, Alex. Unfortunately we appear to be using Livy 0.3.0. I agree that it’s unlikely to be a Kerberos issue if the GET calls are working.  Still debugging… but I’ll add an update if we get to the bottom of it…

Thanks again, Lucas.

From: Alex Bozarth [mailto:ajbozart@us.ibm.com]
Sent: 24 October 2017 23:26
To: user@livy.incubator.apache.org<ma...@livy.incubator.apache.org>
Subject: EXT: Re: Livy GET requests work but POST requests fail with "401 Authentication required"


So it's been a few months since I played with a Kerberized Livy cluster but I'll see if I can help. First what version of Livy are you using? Authentication was updated in Livy 0.4 and it may change how to address this. ( https://github.com/apache/incubator-livy/pull/15 ) GET and POST calls can have different levels of auth depending on how it's set up and that may be why you're seeing the difference. In 0.4 GET calls check livy.server.access-control.view-users and POST calls check livy.server.access-control.modify-users. Also I'm a bit foggy on it, but IIRC Kerberos auth is used on all calls, so if you're getting responses on GETS its should not Kerberos causing the issue on the POSTs.

Hopefully that give you a launching point at trouble shooting.
Alex Bozarth
Software Engineer
Spark Technology Center




________________________________

E-mail: ajbozart@us.ibm.com<ma...@us.ibm.com>
GitHub: github.com/ajbozarth<https://github.com/ajbozarth>



505 Howard Street
San Francisco, CA 94105
United States




[Inactive hide details for "Partridge, Lucas (GE Aviation)" ---10/23/2017 08:27:18 AM---I've written a Java client for parts of]"Partridge, Lucas (GE Aviation)" ---10/23/2017 08:27:18 AM---I've written a Java client for parts of Livy's REST API at https://urldefense.proofpoint.com/v2/url?

From: "Partridge, Lucas (GE Aviation)" <Lu...@ge.com>>
To: "user@livy.incubator.apache.org<ma...@livy.incubator.apache.org>" <us...@livy.incubator.apache.org>>
Date: 10/23/2017 08:27 AM
Subject: Livy GET requests work but POST requests fail with "401 Authentication required"

________________________________



I’ve written a Java client for parts of Livy’s REST API at https://github.com/apache/incubator-livy/blob/master/docs/rest-api.md#response-body<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_incubator-2Dlivy_blob_master_docs_rest-2Dapi.md-23response-2Dbody&d=DwMFAg&c=jf_iaSHvJObTbx-siA1ZOg&r=S1_S7Dymu4ZL6g7L21O78VQZ53vEnAyZ-cx37DPYDyo&m=t80CtBH011YjL6EMsWz08NO-N15lAYv16fDRDxjPPXA&s=tFufJNt3BeUsmOQ5gNRd0Snak-_i_QtS7_gc7YHwWS4&e=> . The client uses Spring’s RestTemplate class and the Livy server is secured with Kerberos.

Get /sessions and GET /batches requests work fine. But both POST /sessions and POST /batches requests fail with:
org.springframework.web.client.HttpClientErrorException: 401 Authentication required

Has anyone else seen something like this where the GET requests work but POST requests don’t? Does my username need adding to some Livy config setting perhaps, e.g., to livy.server.access_control.users as described at http://henning.kropponline.de/2016/11/06/connecting-livy-to-a-secured-kerberized-hdp-cluster/<https://urldefense.proofpoint.com/v2/url?u=http-3A__henning.kropponline.de_2016_11_06_connecting-2Dlivy-2Dto-2Da-2Dsecured-2Dkerberized-2Dhdp-2Dcluster_&d=DwMFAg&c=jf_iaSHvJObTbx-siA1ZOg&r=S1_S7Dymu4ZL6g7L21O78VQZ53vEnAyZ-cx37DPYDyo&m=t80CtBH011YjL6EMsWz08NO-N15lAYv16fDRDxjPPXA&s=96eelCDFXq8_SmS_kxRaLTcwLmxjIlyOqbtMF9mya6I&e=> ?

Thanks, Lucas.