You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/02/09 18:49:51 UTC
[GitHub] [pulsar] BewareMyPower opened a new pull request #9548: [Python] Fix nested Map or Array in schema doesn't work
BewareMyPower opened a new pull request #9548:
URL: https://github.com/apache/pulsar/pull/9548
Fixes #9483
### Motivation
Currently Python client doesn't handle nested `Map` or `Array` well, the generated schema string is invalid. It's because when `Map`/`Array`'s `schema()` method set the `values` field of schema string, it only ignores the `Record` type but not `Map` and `Array`.
### Modifications
- Fix nested Map or Array in schema doesn't work.
- Add 4 tests for `Map<Map>`, `Map<Array>`, `Array<Array>`, `Array<Map>` to cover all nested cases that involve `Map` or `Array`.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change added tests and can be verified as follows:
- *Added SchemaTest.test_avro_map_array to schema_test.py*
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #9548: [Python] Fix nested Map or Array in schema doesn't work
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #9548:
URL: https://github.com/apache/pulsar/pull/9548#issuecomment-777333435
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #9548: [Python] Fix nested Map or Array in schema doesn't work
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #9548:
URL: https://github.com/apache/pulsar/pull/9548#issuecomment-777365251
Now all tests passed, PTAL again @sijie @codelipenghui @eolivelli
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #9548: [Python] Fix nested Map or Array in schema doesn't work
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #9548:
URL: https://github.com/apache/pulsar/pull/9548#issuecomment-776405039
@wolfstudy It's caused by broken master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower edited a comment on pull request #9548: [Python] Fix nested Map or Array in schema doesn't work
Posted by GitBox <gi...@apache.org>.
BewareMyPower edited a comment on pull request #9548:
URL: https://github.com/apache/pulsar/pull/9548#issuecomment-777365251
Now all tests passed, PTAL again @sijie @codelipenghui @eolivelli
There's a bug in Python2 that was exposed in my tests accidentally and I fixed it in this PR by the way. The bug is that Python2 doesn't differ bytes and string well and has the `unicode` and `str` type. The existed code only checks if the key of map is `str` but it could be `unicode` during deserialization.
In Python3, this bug doesn't exist.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #9548: [Python] Fix nested Map or Array in schema doesn't work
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #9548:
URL: https://github.com/apache/pulsar/pull/9548#issuecomment-776532709
It looks like the tests failed because some Python 2.7 issues (I tested with Python 3.x in my local env). I'll take a look.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat merged pull request #9548: [Python] Fix nested Map or Array in schema doesn't work
Posted by GitBox <gi...@apache.org>.
merlimat merged pull request #9548:
URL: https://github.com/apache/pulsar/pull/9548
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #9548: [Python] Fix nested Map or Array in schema doesn't work
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #9548:
URL: https://github.com/apache/pulsar/pull/9548#issuecomment-777218456
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #9548: [Python] Fix nested Map or Array in schema doesn't work
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #9548:
URL: https://github.com/apache/pulsar/pull/9548#issuecomment-777272042
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #9548: [Python] Fix nested Map or Array in schema doesn't work
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #9548:
URL: https://github.com/apache/pulsar/pull/9548#issuecomment-776405498
OK, it looks like the problem is fixed, It'll rebase.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org