You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/12/21 03:25:11 UTC
[GitHub] [pulsar] jack-tristan opened a new issue #13419: python sdk bug "Pulsar error: IncompatibleSchema"
jack-tristan opened a new issue #13419:
URL: https://github.com/apache/pulsar/issues/13419
**Describe the bug**
python sdk bug "Pulsar error: IncompatibleSchema".
**To Reproduce**
Steps to reproduce the behavior:
1. schema v1 (mind the key `mmid`)
```
import pulsar
from pulsar.schema import *
class TaskMsgSchema(Record):
#mmid = Long()
uri = String() # 媒体uri地址
bi = Integer() # 媒体业务信息 - 业务方
db = Integer() # 媒资业务信息 - 业务数据
itemid = Integer() # 媒资数据id - itemid
tasks = Array(String()) # 检查表
redl = Boolean() # 是否需要下载：新下载或重新下载（由驱动层决策完毕，mediadl任务只执行）
bk = String()
if __name__ == '__main__':
msg = TaskMsgSchema()
msg.uri = 'http://127.0.0.1'
msg.bi = 83586 # 80908399640577
msg.db = 1 # 媒资业务信息 - 业务数据
msg.itemid = 14966527 # 媒资数据id - itemid
msg.tasks = ["star"] # 检查表
msg.redl = True # 是否需要下载：新下载或重新下载（由驱动层决策完毕，mediadl任务只执行）
msg.bk = "test_bucket"
#msg.mmid = 8358680908399640577
client = pulsar.Client('pulsar://10.200.19.39:6650,10.200.19.40:6650,10.200.19.41:6650')
producer = client.create_producer('persistent://mgtv-algorithm/app-meizi/pt-ai-mzxz-dev-v1', schema=AvroSchema(TaskMsgSchema))
```
2. schema v2 (mind the key `mmid`)
```
import pulsar
from pulsar.schema import *
class TaskMsgSchema(Record):
mmid = Long()
uri = String() # 媒体uri地址
bi = Integer() # 媒体业务信息 - 业务方
db = Integer() # 媒资业务信息 - 业务数据
itemid = Integer() # 媒资数据id - itemid
tasks = Array(String()) # 检查表
redl = Boolean() # 是否需要下载：新下载或重新下载（由驱动层决策完毕，mediadl任务只执行）
bk = String()
if __name__ == '__main__':
msg = TaskMsgSchema()
msg.uri = 'http://127.0.0.1'
msg.bi = 83586 # 80908399640577
msg.db = 1 # 媒资业务信息 - 业务数据
msg.itemid = 14966527 # 媒资数据id - itemid
msg.tasks = ["star"] # 检查表
msg.redl = True # 是否需要下载：新下载或重新下载（由驱动层决策完毕，mediadl任务只执行）
msg.bk = "test_bucket"
msg.mmid = 8358680908399640577
client = pulsar.Client('pulsar://10.200.19.39:6650,10.200.19.40:6650,10.200.19.41:6650')
producer = client.create_producer('persistent://mgtv-algorithm/app-meizi/pt-ai-mzxz-dev-v1', schema=AvroSchema(TaskMsgSchema))
```
3. Exception: Pulsar error: IncompatibleSchema
4.
![Uploading pulsar-python3-error.png…]()
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
Error:
```
2021-12-21 10:26:47.133 WARN [140086447261440] ClientConnection:1034 | [10.200.19.32:36558 -> 10.200.19.41:6650] Received error response from server: IncompatibleSchema (org.apache.pulsar.broker.service.schema.exceptions.IncompatibleSchemaException: org.apache.avro.SchemaValidationException: Unable to read schema:
{
"type" : "record",
"name" : "TaskMsgSchema",
"fields" : [ {
"name" : "bi",
"type" : [ "null", "int" ]
}, {
"name" : "bk",
"type" : [ "null", "string" ]
}, {
"name" : "db",
"type" : [ "null", "int" ]
}, {
"name" : "itemid",
"type" : [ "null", "int" ]
}, {
"name" : "redl",
"type" : [ "null", "boolean" ]
}, {
"name" : "tasks",
"type" : [ "null", {
"type" : "array",
"items" : "string"
} ]
}, {
"name" : "uri",
"type" : [ "null", "string" ]
} ]
}
using schema:
{
"type" : "record",
"name" : "TaskMsgSchema",
"fields" : [ {
"name" : "bi",
"type" : [ "null", "int" ]
}, {
"name" : "bk",
"type" : [ "null", "string" ]
}, {
"name" : "db",
"type" : [ "null", "int" ]
}, {
"name" : "itemid",
"type" : [ "null", "int" ]
}, {
"name" : "mmid",
"type" : [ "null", "long" ]
}, {
"name" : "redl",
"type" : [ "null", "boolean" ]
}, {
"name" : "tasks",
"type" : [ "null", {
"type" : "array",
"items" : "string"
} ]
}, {
"name" : "uri",
"type" : [ "null", "string" ]
} ]
}) -- req_id: 0
2021-12-21 10:26:47.133 ERROR [140086447261440] ProducerImpl:239 | [persistent://public/default/test-aero-v1, ] Failed to create producer: IncompatibleSchema
2021-12-21 10:26:47.133 INFO [140086447261440] ProducerImpl:564 | Producer - [persistent://public/default/test-aero-v1, ] , [batching = off]
Traceback (most recent call last):
File "ttt.py", line 28, in <module>
producer = client.create_producer('test-aero-v1', schema=AvroSchema(TaskMsgSchema))
File "/usr/local/python3/lib/python3.7/site-packages/pulsar/__init__.py", line 589, in create_producer
p._producer = self._client.create_producer(topic, conf)
_pulsar.IncompatibleSchema: Pulsar error: IncompatibleSchema
```
**Desktop (please complete the following information):**
- OS: centos7
**Additional context**
pulsar version is 2.6.3.
python pulsar-client error version eg:2.6.3/2.7.2/2.8.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #13419: python sdk bug "Pulsar error: IncompatibleSchema"
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #13419:
URL: https://github.com/apache/pulsar/issues/13419#issuecomment-1051438426
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jack-tristan commented on issue #13419: python sdk bug "Pulsar error: IncompatibleSchema"
Posted by GitBox <gi...@apache.org>.
jack-tristan commented on issue #13419:
URL: https://github.com/apache/pulsar/issues/13419#issuecomment-998441536
AutoUpdate Strategy = Full
Schema Validation Enforced = Disabled
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org