[jira] [Created] (AVRO-3037) py3 ipc.Responder swallows schema errors

["Owen Healy (Jira)" <ji...@apache.org>]

Owen Healy created AVRO-3037:
--------------------------------

             Summary: py3 ipc.Responder swallows schema errors
                 Key: AVRO-3037
                 URL: https://issues.apache.org/jira/browse/AVRO-3037
             Project: Apache Avro
          Issue Type: Bug
          Components: python
    Affects Versions: 1.10.1
            Reporter: Owen Healy


In the Python3 Avro IPC library, if the responder returns a response that doesn't match the schema, this error is swallowed and the response data is truncated. This results in a malformed response and a parsing error at the client end.

 

An [example showing the problem is on GitHub|https://github.com/ellbur/avro-error-example/blob/main/main.py].

 

The problem appears to come down to [these lines in ipc.py|https://github.com/apache/avro/blob/branch-1.10/lang/py3/avro/ipc.py#L393]:
{code:java}
except schema.AvroException as exn:
    error = AvroRemoteException(str(exn))
    buffer_encoder = avro_io.BinaryEncoder(io.StringIO())
    META_WRITER.write(response_metadata, buffer_encoder)
    buffer_encoder.write_boolean(True)
    self._WriteError(SYSTEM_ERROR_SCHEMA, error, buffer_encoder)
{code}
I can think of two ways to fix this:
 # Error on the server side; or
 # Pass the error through to the client.

 

Choice (1) seems to make logical sense since sending a non-conforming response is a bug in the server, not the client, but (2) might be easier to detect in real applications. 

 

The way to do (2) would be something like the following, which I am happy to make a pull request for if it would be helpful:
{code:java}
# write response using local protocol
META_WRITER.write(response_metadata, buffer_encoder)
saved_pos = buffer_writer.tell()
buffer_encoder.write_boolean(error is not None)
if error is None:
    writers_schema = local_message.response
    try:
        self.write_response(writers_schema, response, buffer_encoder)
    except schema.AvroException as e:
        buffer_writer.seek(saved_pos)
        buffer_writer.truncate(saved_pos)
        buffer_encoder.write_boolean(True)
        writers_schema = local_message.errors
        self.write_error(writers_schema, e, buffer_encoder)
else:
    writers_schema = local_message.errors
    self.write_error(writers_schema, error, buffer_encoder)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)