[jira] [Commented] (LIVY-1062) Livy Java HTTP client does not follow redirects for POST/DELETE in HA mode

["Nilesh Rathi (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LIVY-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18099922#comment-18099922 ] 

Nilesh Rathi commented on LIVY-1062:
------------------------------------

PR available at https://github.com/apache/livy/pull/535

> Livy Java HTTP client does not follow redirects for POST/DELETE in HA mode
> --------------------------------------------------------------------------
>
>                 Key: LIVY-1062
>                 URL: https://issues.apache.org/jira/browse/LIVY-1062
>             Project: Livy
>          Issue Type: Bug
>            Reporter: Nilesh Rathi
>            Assignee: Nilesh Rathi
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> When Apache Livy runs in High Availability (HA) mode, passive (standby) Livy server instances redirect incoming client requests to the active leader using HTTP {{307 Temporary Redirect}} responses with a {{Location}} header pointing to the active server.
> The Livy Java HTTP client ({{{}LivyConnection{}}} in {{{}client-http{}}}) does not handle these redirects for {{POST}} and {{DELETE}} requests. Apache HttpClient does not automatically follow redirects for non-idempotent methods, and {{LivyConnection.sendRequest()}} treats any non-200 response as a fatal error. As a result, applications using the Livy Job API (Java/Scala programmatic client) fail when their initial connection lands on a standby node.
> h3. Problem
> In HA deployments, clients often connect to a load-balanced or round-robin Livy endpoint. When a {{POST}} (e.g., session creation, job submission) or {{DELETE}} (e.g., session/job cancellation) request reaches a passive Livy instance, the server responds with HTTP 307 and a {{Location}} header redirecting to the active leader. {{LivyConnection}} does not follow this redirect and instead throws an {{{}IOException{}}}.
> h3. Root Cause
> In {{{}client-http/src/main/java/org/apache/livy/client/http/LivyConnection.java{}}}, the {{sendRequest()}} method executes the HTTP request and only accepts {{HTTP 200 (SC_OK)}} as success. Any other status code (including {{{}307 Temporary Redirect{}}}) causes an {{IOException}} to be thrown:
> {code:java}
> int status = (res.getStatusLine().getStatusCode() / 100) * 100;
> if (status == HttpStatus.SC_OK) {
>   // success
> } else {
>   throw new IOException(String.format("%s: %s", res.getStatusLine().getReasonPhrase(), error));
> }
> {code}
> Apache HttpClient follows redirects automatically for {{GET}} requests but *not* for {{POST}} or {{{}DELETE{}}}. Since the Livy Job API relies heavily on {{POST}} (submit jobs, create sessions) and {{DELETE}} (cancel jobs, stop sessions), HA redirect handling is broken for the primary programmatic client use cases.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)