[jira] [Comment Edited] (AVRO-3037) py3 ipc.Responder swallows schema errors

["Michael A. Smith (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284753#comment-17284753 ] 

Michael A. Smith edited comment on AVRO-3037 at 2/15/21, 2:27 PM:
------------------------------------------------------------------

Hi, Owen, are you able to reproduce this using the avro library instead of avro-python3? avro-python3 is obsolete and no longer maintained. https://pypi.org/project/avro/ is python3 compatible.


was (Author: kojiromike):
Hi, Owen, are you able to reproduce this using the avro library instead of avro-python3? avro-python3 is obsolete and no longer maintained.

> py3 ipc.Responder swallows schema errors
> ----------------------------------------
>
>                 Key: AVRO-3037
>                 URL: https://issues.apache.org/jira/browse/AVRO-3037
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: python
>    Affects Versions: 1.10.1
>            Reporter: Owen Healy
>            Priority: Minor
>              Labels: ipc, python3
>
> In the Python3 Avro IPC library, if the responder returns a response that doesn't match the schema, this error is swallowed and the response data is truncated. This results in a malformed response and a parsing error at the client end.
>  
> An [example showing the problem is on GitHub|https://github.com/ellbur/avro-error-example/blob/main/main.py].
>  
> The problem appears to come down to [these lines in ipc.py|https://github.com/apache/avro/blob/branch-1.10/lang/py3/avro/ipc.py#L393]:
> {code:java}
> except schema.AvroException as exn:
>     error = AvroRemoteException(str(exn))
>     buffer_encoder = avro_io.BinaryEncoder(io.StringIO())
>     META_WRITER.write(response_metadata, buffer_encoder)
>     buffer_encoder.write_boolean(True)
>     self._WriteError(SYSTEM_ERROR_SCHEMA, error, buffer_encoder)
> {code}
> I can think of two ways to fix this:
>  # Error on the server side; or
>  # Pass the error through to the client.
>  
> Choice (1) seems to make logical sense since sending a non-conforming response is a bug in the server, not the client, but (2) might be easier to detect in real applications. 
>  
> The way to do (2) would be something like the following, which I am happy to make a pull request for if it would be helpful:
> {code:java}
> # write response using local protocol
> META_WRITER.write(response_metadata, buffer_encoder)
> saved_pos = buffer_writer.tell()
> buffer_encoder.write_boolean(error is not None)
> if error is None:
>     writers_schema = local_message.response
>     try:
>         self.write_response(writers_schema, response, buffer_encoder)
>     except schema.AvroException as e:
>         buffer_writer.seek(saved_pos)
>         buffer_writer.truncate(saved_pos)
>         buffer_encoder.write_boolean(True)
>         writers_schema = local_message.errors
>         self.write_error(writers_schema, e, buffer_encoder)
> else:
>     writers_schema = local_message.errors
>     self.write_error(writers_schema, error, buffer_encoder)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)