You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/02/22 00:32:36 UTC
[GitHub] [pulsar] codelipenghui opened a new issue #9666: BufferOverflowException occurs while query data by Pulsar SQL and compression enabled.
codelipenghui opened a new issue #9666:
URL: https://github.com/apache/pulsar/issues/9666
**Describe the bug**
Here is the error log:
```
2021-02-22T08:29:11.873+0800 INFO derserialize-thread-split-1 stderr java.nio.BufferOverflowException
2021-02-22T08:29:11.873+0800 INFO derserialize-thread-split-1 stderr at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:206)
2021-02-22T08:29:11.873+0800 INFO derserialize-thread-split-1 stderr at org.apache.pulsar.common.compression.AirliftUtils.ensureAirliftSupported(AirliftUtils.java:32)
2021-02-22T08:29:11.873+0800 INFO derserialize-thread-split-1 stderr at org.apache.pulsar.common.compression.CompressionCodecZstd.decode(CompressionCodecZstd.java:100)
2021-02-22T08:29:11.873+0800 INFO derserialize-thread-split-1 stderr at org.apache.pulsar.common.api.raw.MessageParser.uncompressPayloadIfNeeded(MessageParser.java:140)
2021-02-22T08:29:11.873+0800 INFO derserialize-thread-split-1 stderr at org.apache.pulsar.common.api.raw.MessageParser.parseMessage(MessageParser.java:85)
2021-02-22T08:29:11.873+0800 INFO derserialize-thread-split-1 stderr at org.apache.pulsar.sql.presto.PulsarRecordCursor$DeserializeEntries$1.accept(PulsarRecordCursor.java:243)
2021-02-22T08:29:11.873+0800 INFO derserialize-thread-split-1 stderr at org.apache.pulsar.sql.presto.PulsarRecordCursor$DeserializeEntries$1.accept(PulsarRecordCursor.java:224)
2021-02-22T08:29:11.873+0800 INFO derserialize-thread-split-1 stderr at org.jctools.queues.SpscArrayQueue.drain(SpscArrayQueue.java:266)
2021-02-22T08:29:11.873+0800 INFO derserialize-thread-split-1 stderr at org.jctools.queues.SpscArrayQueue.drain(SpscArrayQueue.java:239)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-1 stderr at org.apache.pulsar.sql.presto.PulsarRecordCursor$DeserializeEntries.run(PulsarRecordCursor.java:224)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-1 stderr at java.lang.Thread.run(Thread.java:748)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr Exception in thread "derserialize-thread-split-0"
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr java.nio.BufferOverflowException
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:206)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr at org.apache.pulsar.common.compression.AirliftUtils.ensureAirliftSupported(AirliftUtils.java:32)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr at org.apache.pulsar.common.compression.CompressionCodecZstd.decode(CompressionCodecZstd.java:100)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr at org.apache.pulsar.common.api.raw.MessageParser.uncompressPayloadIfNeeded(MessageParser.java:140)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr at org.apache.pulsar.common.api.raw.MessageParser.parseMessage(MessageParser.java:85)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr at org.apache.pulsar.sql.presto.PulsarRecordCursor$DeserializeEntries$1.accept(PulsarRecordCursor.java:243)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr at org.apache.pulsar.sql.presto.PulsarRecordCursor$DeserializeEntries$1.accept(PulsarRecordCursor.java:224)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr at org.jctools.queues.SpscArrayQueue.drain(SpscArrayQueue.java:266)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr at org.jctools.queues.SpscArrayQueue.drain(SpscArrayQueue.java:239)
2021-02-22T08:29:11.874+0800 INFO derserialize-thread-split-0 stderr at org.apache.pulsar.sql.presto.PulsarRecordCursor$DeserializeEntries.run(PulsarRecordCursor.java:224)
2021-02-22T08:29:11.875+0800 INFO derserialize-thread-split-0 stderr at java.lang.Thread.run(Thread.java:748)
```
The root cause of this issue is the compression data size greater than the original data size.
**To Reproduce**
Steps to reproduce the behavior:
1. Send message `Pulsar` to a topic with Zstd compression
2. Query data from topic
3. See error
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on issue #9666: BufferOverflowException occurs while query data by Pulsar SQL and compression enabled.
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on issue #9666:
URL: https://github.com/apache/pulsar/issues/9666#issuecomment-782989791
Ok, I'll fix this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9666: BufferOverflowException occurs while query data by Pulsar SQL and compression enabled.
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9666:
URL: https://github.com/apache/pulsar/issues/9666#issuecomment-782966161
@gaoran10 Please help take a look at this issue, thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #9666: BufferOverflowException occurs while query data by Pulsar SQL and compression enabled.
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #9666:
URL: https://github.com/apache/pulsar/issues/9666
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9666: BufferOverflowException occurs while query data by Pulsar SQL and compression enabled.
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9666:
URL: https://github.com/apache/pulsar/issues/9666#issuecomment-782964375
Related to #9658
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org