You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "ktpktr0 (via GitHub)" <gi...@apache.org> on 2024/03/08 12:31:23 UTC
[I] [Bug] topic message size exceeds limit [pulsar]
ktpktr0 opened a new issue, #22225:
URL: https://github.com/apache/pulsar/issues/22225
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
java: 21
pulsar: 3.2.0
### Minimal reproduce step
log
```
2024-03-08T13:43:43,369+0800 [GarbageCollectorThread-6-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Do compaction to compact those files lower than 0.2
2024-03-08T13:43:43,369+0800 [GarbageCollectorThread-6-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Compaction: entry log usage buckets before compaction [10% 20% 30% 40% 50% 60% 70% 80% 90% 100%] = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2024-03-08T13:43:43,369+0800 [GarbageCollectorThread-6-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Compaction: entry log usage buckets[10% 20% 30% 40% 50% 60% 70% 80% 90% 100%] = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], compacted [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2024-03-08T13:49:12,105+0800 [bookie-io-8-1] INFO org.apache.bookkeeper.proto.AuthHandler - Authentication success on server side
2024-03-08T13:49:12,105+0800 [bookie-io-8-1] INFO org.apache.bookkeeper.proto.BookieRequestHandler - Channel connected [id: 0x3087e422, L:/192.168.0.150:3181 - R:/192.168.0.250:53602]
2024-03-08T13:49:12,106+0800 [bookie-io-8-1] ERROR org.apache.bookkeeper.proto.BookieRequestHandler - Unhandled exception occurred in I/O thread or handler on [id: 0x3087e422, L:/192.168.0.150:3181 - R:/192.168.0.250:53602]
io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 5253120: 1195725860 - discarded
at io.netty.handler.codec.LengthFieldBasedFrameDecoder.fail(LengthFieldBasedFrameDecoder.java:507) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final]
at io.netty.handler.codec.LengthFieldBasedFrameDecoder.failIfNecessary(LengthFieldBasedFrameDecoder.java:493) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final]
at io.netty.handler.codec.LengthFieldBasedFrameDecoder.exceededFrameLength(LengthFieldBasedFrameDecoder.java:377) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final]
at io.netty.handler.codec.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:423) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final]
at io.netty.handler.codec.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:333) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final]
at io.netty.handler.flush.FlushConsolidationHandler.channelRead(FlushConsolidationHandler.java:152) ~[io.netty-netty-handler-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[io.netty-netty-transport-classes-epoll-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[io.netty-netty-transport-classes-epoll-4.1.104.Final.jar:4.1.104.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[io.netty-netty-transport-classes-epoll-4.1.104.Final.jar:4.1.104.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[io.netty-netty-common-4.1.104.Final.jar:4.1.104.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[io.netty-netty-common-4.1.104.Final.jar:4.1.104.Final]
```
### What did you expect to see?
Message size does not exceed the limit
### What did you see instead?
The message size exceeds the limit by too much, too much
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] topic message size exceeds limit [pulsar]
Posted by "dao-jun (via GitHub)" <gi...@apache.org>.
dao-jun commented on issue #22225:
URL: https://github.com/apache/pulsar/issues/22225#issuecomment-2031024495
@ktpktr0
I noticed the remote peer is 192.168.0.250, what is the remote service? Pulsar or bookkeeper?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] topic message size exceeds limit [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #22225: [Bug] topic message size exceeds limit
URL: https://github.com/apache/pulsar/issues/22225
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] topic message size exceeds limit [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #22225:
URL: https://github.com/apache/pulsar/issues/22225#issuecomment-6381525176
Thanks for the report @ktpktr0. This isn't a message that is too large. The "frame length" in the error, 1195725860, minus the 4-byte length header is 0x47455420, the ASCII bytes `GET `. Something at 192.168.0.250 is sending an HTTP request to the bookie's binary protocol port (3181), and the bookie drops that connection as it should. Check which process on that host connects to port 3181, for example a health check, a metrics scraper or a service URL with the wrong port, and point it at the bookie HTTP port (8000 by default) instead. If you see this from a real Pulsar or BookKeeper client on a currently supported release (4.0.x, 4.2.x or 5.0.x), please reopen with those details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] topic message size exceeds limit [pulsar]
Posted by "dao-jun (via GitHub)" <gi...@apache.org>.
dao-jun commented on issue #22225:
URL: https://github.com/apache/pulsar/issues/22225#issuecomment-2031028696
so the remote service is... ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] topic message size exceeds limit [pulsar]
Posted by "dao-jun (via GitHub)" <gi...@apache.org>.
dao-jun commented on issue #22225:
URL: https://github.com/apache/pulsar/issues/22225#issuecomment-2030984013
@ktpktr0 It should not happen, is there error log on pulsar side?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] topic message size exceeds limit [pulsar]
Posted by "ktpktr0 (via GitHub)" <gi...@apache.org>.
ktpktr0 commented on issue #22225:
URL: https://github.com/apache/pulsar/issues/22225#issuecomment-2030989732
> @ktpktr0 It should not happen, is there error log on pulsar side?
There are no obvious errors, it looks normal. I wonder if this error is caused by the size generated during GC compression
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] topic message size exceeds limit [pulsar]
Posted by "ktpktr0 (via GitHub)" <gi...@apache.org>.
ktpktr0 commented on issue #22225:
URL: https://github.com/apache/pulsar/issues/22225#issuecomment-2031026279
Pulsar has three nodes, broker and cookie
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org