You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/05/15 00:55:28 UTC
[GitHub] [pulsar] ta1meng opened a new issue #10596: Python client: broken/missing Avro schema support
ta1meng opened a new issue #10596:
URL: https://github.com/apache/pulsar/issues/10596
**Describe the bug**
There are many issues in the Python client library regarding Avro schema support:
* The documented [example](https://pulsar.apache.org/docs/fr/client-libraries-python/#complex-types) does not work.
* No support for `default` values.
* No support for the `doc` attribute.
* No support for the `namespace` attribute.
* No support for required subrecords (required fields of type `record`)
* Ordering bug when generating schema definition that makes schema incompatible with topic's registered schema.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to https://github.com/ta1meng/pulsar-python-avro-schema-examples
2. Read the README files
3. All issues above except one can be reproduced using the Python example applications in that repo
4. The repo does not contain an example of the lack of support of explicit default values, but it's easy to see that shortcoming in code -- there is no code for `default` values inside the schema generation code.
5. Apply the changes in the `schema_fix` directory to the Pulsar Python client library, so that all example applications will succeed.
**Expected behavior**
The [documented example](https://pulsar.apache.org/docs/fr/client-libraries-python/#complex-types) should work out of the box. There should be support for common Avro keywords like `default`, `doc`, and `namespace`.
**Screenshots**
N/A
**Desktop (please complete the following information):**
- OS: MacOS
**Additional context**
Tested agains Pulsar 2.7.1. Using Python Pulsar client 2.7.1 under Python 3.7. Though issues and fixes should be appliable to Python 2.7 as well (untested).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff edited a comment on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff edited a comment on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-947972294
UPDATE ON BUG STATUS AFTER TESTING AGAINST PULSAR 2.7.2 and Pulsar 2.8.1:
The documented [example](https://pulsar.apache.org/docs/fr/client-libraries-python/#complex-types)
* No support for `default` values. [NOT FIXED]
* No support for the `doc` attribute. [NOT FIXED]
* No support for the `namespace` attribute. [NOT FIXED]
* No support for subrecords (fields of type `record`) [FIXED]
* Ordering bug when generating schema definition that makes schema incompatible with topic's registered schema. [NOT FIXED]
Additional context
*The support for subrecords (item 4) was fixed for Pulsar 2.7.2 and flows into 2.8.1. The remaining identified issues are not fixed through 2.8.1.*
Tested against Pulsar 2.7.2 and Pulsar 2.8.1. Using both Python Pulsar client 2.7.2 and Pulsar 2.8.1 under Python 3.7.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-1058890333
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff commented on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff commented on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-947972294
UPDATE ON BUG STATUS AFTER TESTING AGAINST PULSAR 2.8.1:
The documented [example](https://pulsar.apache.org/docs/fr/client-libraries-python/#complex-types)
* No support for `default` values. [NOT FIXED]
* No support for the `doc` attribute. [NOT FIXED]
* No support for the `namespace` attribute. [NOT FIXED]
* No support for subrecords (fields of type record) [FIXED]
* Ordering bug when generating schema definition that makes schema incompatible with topic's registered schema. [TO TRIAGE]
Additional context
Tested against Pulsar 2.81. Using Python Pulsar client 2.8.1 under Python 3.7.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff commented on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff commented on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-956769752
Separate tickets filed for remaining issues:
https://github.com/apache/pulsar/issues/12571
https://github.com/apache/pulsar/issues/12572
https://github.com/apache/pulsar/issues/12573
https://github.com/apache/pulsar/issues/12574
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff commented on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff commented on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-956769752
Separate tickets filed for remaining issues:
https://github.com/apache/pulsar/issues/12571
https://github.com/apache/pulsar/issues/12572
https://github.com/apache/pulsar/issues/12573
https://github.com/apache/pulsar/issues/12574
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff edited a comment on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff edited a comment on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-947972294
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff edited a comment on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff edited a comment on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-956769752
Separate tickets filed for remaining issues:
No support for `default` values: https://github.com/apache/pulsar/issues/12571
No support for `doc` attribute: https://github.com/apache/pulsar/issues/12572
No support for the `namespace` attribute: https://github.com/apache/pulsar/issues/12573
Ordering bug when generating schema definition : https://github.com/apache/pulsar/issues/12574
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff edited a comment on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff edited a comment on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-956769752
Separate tickets filed for remaining issues:
No support for `default` values: https://github.com/apache/pulsar/issues/12571
No support for `doc` attribute: https://github.com/apache/pulsar/issues/12572
No support for the `namespace` attribute: https://github.com/apache/pulsar/issues/12573
Ordering bug when generating schema definition : https://github.com/apache/pulsar/issues/12574
This ticket is now only related to the sub-records bug, which has been fixed. This ticket can now be closed and remaining issues can be addressed in their respective tickets..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff commented on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff commented on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-956769752
Separate tickets filed for remaining issues:
https://github.com/apache/pulsar/issues/12571
https://github.com/apache/pulsar/issues/12572
https://github.com/apache/pulsar/issues/12573
https://github.com/apache/pulsar/issues/12574
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ta1meng commented on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
ta1meng commented on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-841574871
This ticket is a superset of https://github.com/apache/pulsar/issues/9571
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff edited a comment on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff edited a comment on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-947972294
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff edited a comment on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff edited a comment on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-947972294
UPDATE ON BUG STATUS AFTER TESTING AGAINST PULSAR 2.7.2 and Pulsar 2.8.1:
The documented [example](https://pulsar.apache.org/docs/fr/client-libraries-python/#complex-types)
* No support for `default` values. [NOT FIXED]
* No support for the `doc` attribute. [NOT FIXED]
* No support for the `namespace` attribute. [NOT FIXED]
* No support for subrecords (fields of type `record`) [FIXED]
* Ordering bug when generating schema definition that makes schema incompatible with topic's registered schema. [TO TRIAGE]
Additional context
*The support for subrecords (item 4) was fixed for Pulsar 2.7.2 and flows into 2.8.1. The remaining identified issues are not fixed through 2.8.1.*
Tested against Pulsar 2.7.2 and Pulsar 2.8.1. Using both Python Pulsar client 2.7.2 and Pulsar 2.8.1 under Python 3.7.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] branhoff edited a comment on issue #10596: Python client: broken/missing Avro schema support
Posted by GitBox <gi...@apache.org>.
branhoff edited a comment on issue #10596:
URL: https://github.com/apache/pulsar/issues/10596#issuecomment-947972294
UPDATE ON BUG STATUS AFTER TESTING AGAINST PULSAR 2.8.1:
The documented [example](https://pulsar.apache.org/docs/fr/client-libraries-python/#complex-types)
* No support for `default` values. [NOT FIXED]
* No support for the `doc` attribute. [NOT FIXED]
* No support for the `namespace` attribute. [NOT FIXED]
* No support for subrecords (fields of type `record`) [FIXED]
* Ordering bug when generating schema definition that makes schema incompatible with topic's registered schema. [TO TRIAGE]
Additional context
Tested against Pulsar 2.81. Using Python Pulsar client 2.8.1 under Python 3.7.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org