You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by GitBox <gi...@apache.org> on 2022/02/09 16:48:20 UTC
[GitHub] [avro] jjaakola-aiven opened a new pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
jjaakola-aiven opened a new pull request #1529:
URL: https://github.com/apache/avro/pull/1529
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Avro Jira-3380](https://issues.apache.org/jira/browse/AVRO-3380/) issues and references them in the PR title. For example, "AVRO-1234: My Avro PR"
- https://issues.apache.org/jira/browse/AVRO-3380
- In case you are adding a dependency, check if the license complies with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
* lang/py/avro/test/test_io.py::TestIncompatibleSchemaReading
### Commits
- [x] My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](https://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes how to use it.
- All the public functions and the classes in the PR contain Javadoc that explain what it does
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] netlify[bot] commented on pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
netlify[bot] commented on pull request #1529:
URL: https://github.com/apache/avro/pull/1529#issuecomment-1033971695
👷 Deploy Preview for *avro-website-preview* processing.
🔨 Explore the source changes: f10214d10ebecdf14ddcaf2a9ae235246b1d9ebd
🔍 Inspect the deploy log: [https://app.netlify.com/sites/avro-website-preview/deploys/6203f04ba011360007831ed9](https://app.netlify.com/sites/avro-website-preview/deploys/6203f04ba011360007831ed9)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] RyanSkraba commented on a change in pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
RyanSkraba commented on a change in pull request #1529:
URL: https://github.com/apache/avro/pull/1529#discussion_r803974606
##########
File path: lang/py/avro/io.py
##########
@@ -222,7 +222,10 @@ def read(self, n: int) -> bytes:
"""
Read n bytes.
"""
- return self.reader.read(n)
+ read_bytes = self.reader.read(n)
Review comment:
You mentioned that the original `avro-python3` code also checked that n is non-negative. I think that's a good idea too!
##########
File path: lang/py/avro/io.py
##########
@@ -663,63 +666,69 @@ def read_data(self, writers_schema: avro.schema.Schema, readers_schema: avro.sch
# This shouldn't happen because of the match check at the start of this method.
raise avro.errors.SchemaResolutionException("Schemas do not match.", writers_schema, readers_schema)
- if writers_schema.type == "null":
- return None
- if writers_schema.type == "boolean":
- return decoder.read_boolean()
- if writers_schema.type == "string":
- return decoder.read_utf8()
- if writers_schema.type == "int":
- if logical_type == avro.constants.DATE:
- return decoder.read_date_from_int()
- if logical_type == avro.constants.TIME_MILLIS:
- return decoder.read_time_millis_from_int()
- return decoder.read_int()
- if writers_schema.type == "long":
- if logical_type == avro.constants.TIME_MICROS:
- return decoder.read_time_micros_from_long()
- if logical_type == avro.constants.TIMESTAMP_MILLIS:
- return decoder.read_timestamp_millis_from_long()
- if logical_type == avro.constants.TIMESTAMP_MICROS:
- return decoder.read_timestamp_micros_from_long()
- return decoder.read_long()
- if writers_schema.type == "float":
- return decoder.read_float()
- if writers_schema.type == "double":
- return decoder.read_double()
- if writers_schema.type == "bytes":
- if logical_type == "decimal":
- precision = writers_schema.get_prop("precision")
- if not (isinstance(precision, int) and precision > 0):
- warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal precision {precision}. Must be a positive integer."))
- return decoder.read_bytes()
- scale = writers_schema.get_prop("scale")
- if not (isinstance(scale, int) and scale > 0):
- warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal scale {scale}. Must be a positive integer."))
- return decoder.read_bytes()
- return decoder.read_decimal_from_bytes(precision, scale)
- return decoder.read_bytes()
- if isinstance(writers_schema, avro.schema.FixedSchema) and isinstance(readers_schema, avro.schema.FixedSchema):
- if logical_type == "decimal":
- precision = writers_schema.get_prop("precision")
- if not (isinstance(precision, int) and precision > 0):
- warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal precision {precision}. Must be a positive integer."))
- return self.read_fixed(writers_schema, readers_schema, decoder)
- scale = writers_schema.get_prop("scale")
- if not (isinstance(scale, int) and scale > 0):
- warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal scale {scale}. Must be a positive integer."))
- return self.read_fixed(writers_schema, readers_schema, decoder)
- return decoder.read_decimal_from_fixed(precision, scale, writers_schema.size)
- return self.read_fixed(writers_schema, readers_schema, decoder)
- if isinstance(writers_schema, avro.schema.EnumSchema) and isinstance(readers_schema, avro.schema.EnumSchema):
- return self.read_enum(writers_schema, readers_schema, decoder)
- if isinstance(writers_schema, avro.schema.ArraySchema) and isinstance(readers_schema, avro.schema.ArraySchema):
- return self.read_array(writers_schema, readers_schema, decoder)
- if isinstance(writers_schema, avro.schema.MapSchema) and isinstance(readers_schema, avro.schema.MapSchema):
- return self.read_map(writers_schema, readers_schema, decoder)
- if isinstance(writers_schema, avro.schema.RecordSchema) and isinstance(readers_schema, avro.schema.RecordSchema):
- # .type in ["record", "error", "request"]:
- return self.read_record(writers_schema, readers_schema, decoder)
+ try:
+ if writers_schema.type == "null":
+ return None
+ if writers_schema.type == "boolean":
+ return decoder.read_boolean()
+ if writers_schema.type == "string":
+ return decoder.read_utf8()
+ if writers_schema.type == "int":
+ if logical_type == avro.constants.DATE:
+ return decoder.read_date_from_int()
+ if logical_type == avro.constants.TIME_MILLIS:
+ return decoder.read_time_millis_from_int()
+ return decoder.read_int()
+ if writers_schema.type == "long":
+ if logical_type == avro.constants.TIME_MICROS:
+ return decoder.read_time_micros_from_long()
+ if logical_type == avro.constants.TIMESTAMP_MILLIS:
+ return decoder.read_timestamp_millis_from_long()
+ if logical_type == avro.constants.TIMESTAMP_MICROS:
+ return decoder.read_timestamp_micros_from_long()
+ return decoder.read_long()
+ if writers_schema.type == "float":
+ return decoder.read_float()
+ if writers_schema.type == "double":
+ return decoder.read_double()
+ if writers_schema.type == "bytes":
+ if logical_type == "decimal":
+ precision = writers_schema.get_prop("precision")
+ if not (isinstance(precision, int) and precision > 0):
+ warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal precision {precision}. Must be a positive integer."))
+ return decoder.read_bytes()
+ scale = writers_schema.get_prop("scale")
+ if not (isinstance(scale, int) and scale > 0):
+ warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal scale {scale}. Must be a positive integer."))
+ return decoder.read_bytes()
+ return decoder.read_decimal_from_bytes(precision, scale)
+ return decoder.read_bytes()
+ if isinstance(writers_schema, avro.schema.FixedSchema) and isinstance(readers_schema, avro.schema.FixedSchema):
+ if logical_type == "decimal":
+ precision = writers_schema.get_prop("precision")
+ if not (isinstance(precision, int) and precision > 0):
+ warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal precision {precision}. Must be a positive integer."))
+ return self.read_fixed(writers_schema, readers_schema, decoder)
+ scale = writers_schema.get_prop("scale")
+ if not (isinstance(scale, int) and scale > 0):
+ warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal scale {scale}. Must be a positive integer."))
+ return self.read_fixed(writers_schema, readers_schema, decoder)
+ return decoder.read_decimal_from_fixed(precision, scale, writers_schema.size)
+ return self.read_fixed(writers_schema, readers_schema, decoder)
+ if isinstance(writers_schema, avro.schema.EnumSchema) and isinstance(readers_schema, avro.schema.EnumSchema):
+ return self.read_enum(writers_schema, readers_schema, decoder)
+ if isinstance(writers_schema, avro.schema.ArraySchema) and isinstance(readers_schema, avro.schema.ArraySchema):
+ return self.read_array(writers_schema, readers_schema, decoder)
+ if isinstance(writers_schema, avro.schema.MapSchema) and isinstance(readers_schema, avro.schema.MapSchema):
+ return self.read_map(writers_schema, readers_schema, decoder)
+ if isinstance(writers_schema, avro.schema.RecordSchema) and isinstance(readers_schema, avro.schema.RecordSchema):
+ # .type in ["record", "error", "request"]:
+ return self.read_record(writers_schema, readers_schema, decoder)
+ except avro.errors.InvalidBytesRead as e:
+ decoder.reader.seek(0)
Review comment:
I'm not sure this is the right thing to do here --> I like the original message in `InvalidBytesRead` more than the one constructed by AvroTypeException.
Is there something nice that we can do in the avro.error hierarchy to make this useful?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] jjaakola-aiven commented on a change in pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
jjaakola-aiven commented on a change in pull request #1529:
URL: https://github.com/apache/avro/pull/1529#discussion_r804430842
##########
File path: lang/py/avro/errors.py
##########
@@ -32,6 +32,10 @@ class AvroException(Exception):
"""The base class for exceptions in avro."""
+class InvalidBytesRead(AvroException):
Review comment:
Renamed to `InvalidAvroBinaryEncoding`.
##########
File path: lang/py/avro/io.py
##########
@@ -222,7 +222,10 @@ def read(self, n: int) -> bytes:
"""
Read n bytes.
"""
- return self.reader.read(n)
+ read_bytes = self.reader.read(n)
Review comment:
Added, raises `InvalidAvroBinaryEncoding`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] RyanSkraba merged pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
RyanSkraba merged pull request #1529:
URL: https://github.com/apache/avro/pull/1529
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] netlify[bot] edited a comment on pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
netlify[bot] edited a comment on pull request #1529:
URL: https://github.com/apache/avro/pull/1529#issuecomment-1033971695
👷 Deploy Preview for *avro-website-preview* processing.
🔨 Explore the source changes: a19db77f4caea4ff156007251f631712dd45bf47
🔍 Inspect the deploy log: [https://app.netlify.com/sites/avro-website-preview/deploys/6204170eebd3eb00077c9bef](https://app.netlify.com/sites/avro-website-preview/deploys/6204170eebd3eb00077c9bef)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] netlify[bot] edited a comment on pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
netlify[bot] edited a comment on pull request #1529:
URL: https://github.com/apache/avro/pull/1529#issuecomment-1033971695
❌ Deploy Preview for *avro-website-preview* failed.
🔨 Explore the source changes: 83698458c0c185d1f0172c6a805d98ec3f2a803f
🔍 Inspect the deploy log: [https://app.netlify.com/sites/avro-website-preview/deploys/62061b0fdf74c900070927c1](https://app.netlify.com/sites/avro-website-preview/deploys/62061b0fdf74c900070927c1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] netlify[bot] edited a comment on pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
netlify[bot] edited a comment on pull request #1529:
URL: https://github.com/apache/avro/pull/1529#issuecomment-1033971695
👷 Deploy Preview for *avro-website-preview* processing.
🔨 Explore the source changes: dc1093b3ef0ed7b838fbab48a78e5fb586cae39d
🔍 Inspect the deploy log: [https://app.netlify.com/sites/avro-website-preview/deploys/62041ea5cfbb910008050ba3](https://app.netlify.com/sites/avro-website-preview/deploys/62041ea5cfbb910008050ba3)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] netlify[bot] edited a comment on pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
netlify[bot] edited a comment on pull request #1529:
URL: https://github.com/apache/avro/pull/1529#issuecomment-1033971695
👷 Deploy Preview for *avro-website-preview* processing.
🔨 Explore the source changes: 83698458c0c185d1f0172c6a805d98ec3f2a803f
🔍 Inspect the deploy log: [https://app.netlify.com/sites/avro-website-preview/deploys/62061b0fdf74c900070927c1](https://app.netlify.com/sites/avro-website-preview/deploys/62061b0fdf74c900070927c1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] netlify[bot] edited a comment on pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
netlify[bot] edited a comment on pull request #1529:
URL: https://github.com/apache/avro/pull/1529#issuecomment-1033971695
❌ Deploy Preview for *avro-website-preview* failed.
🔨 Explore the source changes: f10214d10ebecdf14ddcaf2a9ae235246b1d9ebd
🔍 Inspect the deploy log: [https://app.netlify.com/sites/avro-website-preview/deploys/6203f04ba011360007831ed9](https://app.netlify.com/sites/avro-website-preview/deploys/6203f04ba011360007831ed9)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] netlify[bot] edited a comment on pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
netlify[bot] edited a comment on pull request #1529:
URL: https://github.com/apache/avro/pull/1529#issuecomment-1033971695
❌ Deploy Preview for *avro-website-preview* failed.
🔨 Explore the source changes: a19db77f4caea4ff156007251f631712dd45bf47
🔍 Inspect the deploy log: [https://app.netlify.com/sites/avro-website-preview/deploys/6204170eebd3eb00077c9bef](https://app.netlify.com/sites/avro-website-preview/deploys/6204170eebd3eb00077c9bef)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] jjaakola-aiven commented on a change in pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
jjaakola-aiven commented on a change in pull request #1529:
URL: https://github.com/apache/avro/pull/1529#discussion_r804408932
##########
File path: lang/py/avro/io.py
##########
@@ -663,63 +666,69 @@ def read_data(self, writers_schema: avro.schema.Schema, readers_schema: avro.sch
# This shouldn't happen because of the match check at the start of this method.
raise avro.errors.SchemaResolutionException("Schemas do not match.", writers_schema, readers_schema)
- if writers_schema.type == "null":
- return None
- if writers_schema.type == "boolean":
- return decoder.read_boolean()
- if writers_schema.type == "string":
- return decoder.read_utf8()
- if writers_schema.type == "int":
- if logical_type == avro.constants.DATE:
- return decoder.read_date_from_int()
- if logical_type == avro.constants.TIME_MILLIS:
- return decoder.read_time_millis_from_int()
- return decoder.read_int()
- if writers_schema.type == "long":
- if logical_type == avro.constants.TIME_MICROS:
- return decoder.read_time_micros_from_long()
- if logical_type == avro.constants.TIMESTAMP_MILLIS:
- return decoder.read_timestamp_millis_from_long()
- if logical_type == avro.constants.TIMESTAMP_MICROS:
- return decoder.read_timestamp_micros_from_long()
- return decoder.read_long()
- if writers_schema.type == "float":
- return decoder.read_float()
- if writers_schema.type == "double":
- return decoder.read_double()
- if writers_schema.type == "bytes":
- if logical_type == "decimal":
- precision = writers_schema.get_prop("precision")
- if not (isinstance(precision, int) and precision > 0):
- warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal precision {precision}. Must be a positive integer."))
- return decoder.read_bytes()
- scale = writers_schema.get_prop("scale")
- if not (isinstance(scale, int) and scale > 0):
- warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal scale {scale}. Must be a positive integer."))
- return decoder.read_bytes()
- return decoder.read_decimal_from_bytes(precision, scale)
- return decoder.read_bytes()
- if isinstance(writers_schema, avro.schema.FixedSchema) and isinstance(readers_schema, avro.schema.FixedSchema):
- if logical_type == "decimal":
- precision = writers_schema.get_prop("precision")
- if not (isinstance(precision, int) and precision > 0):
- warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal precision {precision}. Must be a positive integer."))
- return self.read_fixed(writers_schema, readers_schema, decoder)
- scale = writers_schema.get_prop("scale")
- if not (isinstance(scale, int) and scale > 0):
- warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal scale {scale}. Must be a positive integer."))
- return self.read_fixed(writers_schema, readers_schema, decoder)
- return decoder.read_decimal_from_fixed(precision, scale, writers_schema.size)
- return self.read_fixed(writers_schema, readers_schema, decoder)
- if isinstance(writers_schema, avro.schema.EnumSchema) and isinstance(readers_schema, avro.schema.EnumSchema):
- return self.read_enum(writers_schema, readers_schema, decoder)
- if isinstance(writers_schema, avro.schema.ArraySchema) and isinstance(readers_schema, avro.schema.ArraySchema):
- return self.read_array(writers_schema, readers_schema, decoder)
- if isinstance(writers_schema, avro.schema.MapSchema) and isinstance(readers_schema, avro.schema.MapSchema):
- return self.read_map(writers_schema, readers_schema, decoder)
- if isinstance(writers_schema, avro.schema.RecordSchema) and isinstance(readers_schema, avro.schema.RecordSchema):
- # .type in ["record", "error", "request"]:
- return self.read_record(writers_schema, readers_schema, decoder)
+ try:
+ if writers_schema.type == "null":
+ return None
+ if writers_schema.type == "boolean":
+ return decoder.read_boolean()
+ if writers_schema.type == "string":
+ return decoder.read_utf8()
+ if writers_schema.type == "int":
+ if logical_type == avro.constants.DATE:
+ return decoder.read_date_from_int()
+ if logical_type == avro.constants.TIME_MILLIS:
+ return decoder.read_time_millis_from_int()
+ return decoder.read_int()
+ if writers_schema.type == "long":
+ if logical_type == avro.constants.TIME_MICROS:
+ return decoder.read_time_micros_from_long()
+ if logical_type == avro.constants.TIMESTAMP_MILLIS:
+ return decoder.read_timestamp_millis_from_long()
+ if logical_type == avro.constants.TIMESTAMP_MICROS:
+ return decoder.read_timestamp_micros_from_long()
+ return decoder.read_long()
+ if writers_schema.type == "float":
+ return decoder.read_float()
+ if writers_schema.type == "double":
+ return decoder.read_double()
+ if writers_schema.type == "bytes":
+ if logical_type == "decimal":
+ precision = writers_schema.get_prop("precision")
+ if not (isinstance(precision, int) and precision > 0):
+ warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal precision {precision}. Must be a positive integer."))
+ return decoder.read_bytes()
+ scale = writers_schema.get_prop("scale")
+ if not (isinstance(scale, int) and scale > 0):
+ warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal scale {scale}. Must be a positive integer."))
+ return decoder.read_bytes()
+ return decoder.read_decimal_from_bytes(precision, scale)
+ return decoder.read_bytes()
+ if isinstance(writers_schema, avro.schema.FixedSchema) and isinstance(readers_schema, avro.schema.FixedSchema):
+ if logical_type == "decimal":
+ precision = writers_schema.get_prop("precision")
+ if not (isinstance(precision, int) and precision > 0):
+ warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal precision {precision}. Must be a positive integer."))
+ return self.read_fixed(writers_schema, readers_schema, decoder)
+ scale = writers_schema.get_prop("scale")
+ if not (isinstance(scale, int) and scale > 0):
+ warnings.warn(avro.errors.IgnoredLogicalType(f"Invalid decimal scale {scale}. Must be a positive integer."))
+ return self.read_fixed(writers_schema, readers_schema, decoder)
+ return decoder.read_decimal_from_fixed(precision, scale, writers_schema.size)
+ return self.read_fixed(writers_schema, readers_schema, decoder)
+ if isinstance(writers_schema, avro.schema.EnumSchema) and isinstance(readers_schema, avro.schema.EnumSchema):
+ return self.read_enum(writers_schema, readers_schema, decoder)
+ if isinstance(writers_schema, avro.schema.ArraySchema) and isinstance(readers_schema, avro.schema.ArraySchema):
+ return self.read_array(writers_schema, readers_schema, decoder)
+ if isinstance(writers_schema, avro.schema.MapSchema) and isinstance(readers_schema, avro.schema.MapSchema):
+ return self.read_map(writers_schema, readers_schema, decoder)
+ if isinstance(writers_schema, avro.schema.RecordSchema) and isinstance(readers_schema, avro.schema.RecordSchema):
+ # .type in ["record", "error", "request"]:
+ return self.read_record(writers_schema, readers_schema, decoder)
+ except avro.errors.InvalidBytesRead as e:
+ decoder.reader.seek(0)
Review comment:
The proposed idea here is to map the specific read error to something that helps out when encoding is off. Logging the data and schema used would help finding the issue.
But I'll remove this. I think logging the data is wrong, as there are environments where data in the datum is sensitive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] RyanSkraba commented on a change in pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
RyanSkraba commented on a change in pull request #1529:
URL: https://github.com/apache/avro/pull/1529#discussion_r803965181
##########
File path: lang/py/avro/errors.py
##########
@@ -32,6 +32,10 @@ class AvroException(Exception):
"""The base class for exceptions in avro."""
+class InvalidBytesRead(AvroException):
Review comment:
This is a good exception, but we could probably extend it to be a bit more useful: `InvalidAvroBinaryEncoding` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [avro] netlify[bot] edited a comment on pull request #1529: AVRO-3380: Raise an exception if invalid number of bytes read
Posted by GitBox <gi...@apache.org>.
netlify[bot] edited a comment on pull request #1529:
URL: https://github.com/apache/avro/pull/1529#issuecomment-1033971695
❌ Deploy Preview for *avro-website-preview* failed.
🔨 Explore the source changes: dc1093b3ef0ed7b838fbab48a78e5fb586cae39d
🔍 Inspect the deploy log: [https://app.netlify.com/sites/avro-website-preview/deploys/62041ea5cfbb910008050ba3](https://app.netlify.com/sites/avro-website-preview/deploys/62041ea5cfbb910008050ba3)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org