[jira] [Created] (ARROW-9802) [FlightRPC][Java] CallStatus.metadata should get copied into the gRPC StatusRuntimeException

["David Li (Jira)" <ji...@apache.org>]

David Li created ARROW-9802:
-------------------------------

             Summary: [FlightRPC][Java] CallStatus.metadata should get copied into the gRPC StatusRuntimeException
                 Key: ARROW-9802
                 URL: https://issues.apache.org/jira/browse/ARROW-9802
             Project: Apache Arrow
          Issue Type: Bug
          Components: FlightRPC, Java
    Affects Versions: 1.0.0
            Reporter: David Li


Reported on the mailing list.
{noformat}
I had been doing it that way, but I found that If I was sending something
along the lines of:

ErrorFlightMetadata errorFlightMetadata = new ErrorFlightMetadata();
errorFlightMetadata.insert("my_custom_key", "hello");
listener.error(CallStatus.INTERNAL
    .withDescription("Testing description")
    .withCause(new RuntimeException("My cause"))
    .withMetadata(errorFlightMetadata)
    .toRuntimeException());


I would only receive the description back by the time it was throwing a
Python error:

FlightInternalError: gRPC returned internal error, with message:
Testing description. Client context: IOError: Server never sent a data
message. Detail: Internal. gRPC client debug context:
{"created":"@1597858462.086707300","description":"Error received from
peer ipv6:[::1]:12233","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Testing
description","grpc_status":13}

(catching the exception and checking FlightInternalError.extra_info returns
nothing as well)

However if I manually create a grpc status exception like so:

private static Metadata.Key<byte[]> arrowStatusDetail =
    Metadata.Key.of("x-arrow-status-detail-bin",
Metadata.BINARY_BYTE_MARSHALLER);
private static Metadata.Key<byte[]> grpcStatusDetail =
    Metadata.Key.of("grpc-status-details-bin", Metadata.BINARY_BYTE_MARSHALLER);
private static Metadata.Key<String> statusCode =
    Metadata.Key.of("x-arrow-status", Metadata.ASCII_STRING_MARSHALLER);
private static Metadata.Key<byte[]> message =
    Metadata.Key.of("x-arrow-status-message-bin",
Metadata.BINARY_BYTE_MARSHALLER);

Metadata metadata = new Metadata();
metadata.put(arrowStatusDetail, "my_internal_details".getBytes());
metadata.put(grpcStatusDetail, "this_is_in_extra_info".getBytes());
metadata.put(statusCode, "1");
metadata.put(message, "my_internal_message".getBytes());
return new StatusRuntimeException(Status.INTERNAL, metadata);
StatusUtils.toGrpcStatus doesn't copy the metadata from the CallStatus into the gRPC status it constructs. 
Then I receive this back on the Python side:

FlightInternalError: my_internal_message. Detail: my_internal_details.
Client context: IOError: Server never sent a data message. Detail:
Internal. gRPC client debug context:
{"created":"@1597859023.244971700","description":"Error received from
peer ipv6:[::1]:12233","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"","grpc_status":13}

and FlightInternalError.extra_info contains the bytes for
"this_is_in_extra_info"  - I can pretty much put whatever I need in there
to add richer metadata and utilize it on the client.

It feels a bit awkward / maybe incorrect to dive into the C++ code to
hijack those metadata keys just to transport extra metadata from Java ->
C++. If the above approach with CallStatus is incorrect for bringing extra
data to the client then let me know.

{noformat}
StatusUtils.toGrpcStatus doesn't copy the metadata from the CallStatus into the gRPC status it constructs; I think that's the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)