You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by z11373 <z1...@outlook.com> on 2017/01/31 20:39:37 UTC
accumulo thrift server error
We have C++ client talking to Accumulo server via Thrift, so there are
Accumulo proxy/thrift instances running on the server. The client got
exception, and upon investigation on Accumulo thrift server, this error line
is found:
2017-01-31 18:00:11,806 [rpc.CustomNonBlockingServer$CustomFrameBuffer]
ERROR: Read a frame size of 18986583, which is bigger than the maximum
allowable buffer size for ALL connections.
This issue is likely the client is trying to write data bigger than 16MB to
Accumulo thrift, which the max frame size is set to 16MB on the server side.
Therefore, I made change on client side to explicitly call
apache::thrift::transport::TFramedTransport::setMaxFrameSize and pass 16 *
1024 * 1024 value (or 16MB).
Unfortunately I still hit same issue, so either I missed something, or what
I did was wrong. If you have experience on Thrift, or ever ran into this
issue, let me know how to workaround this issue.
Thanks,
Z
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/accumulo-thrift-server-error-tp19786.html
Sent from the Developers mailing list archive at Nabble.com.
Re: accumulo thrift server error
Posted by z11373 <z1...@outlook.com>.
Thanks Josh and Keith!
Just to be clear, it is Accumulo Thrift server, not Accumulo Tablet server.
The fix is to increase the value of maxFrameSize config in
conf/proxy.properties, and that fixes the issue. Thanks for the explanation
too, Josh!
Z
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/accumulo-thrift-server-error-tp19786p19790.html
Sent from the Developers mailing list archive at Nabble.com.
Re: accumulo thrift server error
Posted by Christopher <ct...@apache.org>.
Possibly general.server.message.size.max ? Docs say default is 1G, but that
doesn't seem right to me.
https://accumulo.apache.org/1.8/accumulo_user_manual.html#_general_server_message_size_max
On Tue, Jan 31, 2017 at 6:07 PM Josh Elser <jo...@gmail.com> wrote:
> The maximum frame size is being validated by the Thrift Proxy server, not
> your client. This is to prevent clients from performing a denial of service
> attack against the (shared) service.
>
> There should be a configuration property to control this, but I don't know
> what it is off the top of my head (and am on my phone). I would guess that
> it's documented on the Accumulo user manual.
>
> On Jan 31, 2017 15:44, "z11373" <z1...@outlook.com> wrote:
>
> > We have C++ client talking to Accumulo server via Thrift, so there are
> > Accumulo proxy/thrift instances running on the server. The client got
> > exception, and upon investigation on Accumulo thrift server, this error
> > line
> > is found:
> >
> > 2017-01-31 18:00:11,806 [rpc.CustomNonBlockingServer$CustomFrameBuffer]
> > ERROR: Read a frame size of 18986583, which is bigger than the maximum
> > allowable buffer size for ALL connections.
> >
> > This issue is likely the client is trying to write data bigger than 16MB
> to
> > Accumulo thrift, which the max frame size is set to 16MB on the server
> > side.
> > Therefore, I made change on client side to explicitly call
> > apache::thrift::transport::TFramedTransport::setMaxFrameSize and pass 16
> *
> > 1024 * 1024 value (or 16MB).
> >
> > Unfortunately I still hit same issue, so either I missed something, or
> what
> > I did was wrong. If you have experience on Thrift, or ever ran into this
> > issue, let me know how to workaround this issue.
> >
> > Thanks,
> > Z
> >
> >
> >
> > --
> > View this message in context: http://apache-accumulo.
> > 1065345.n5.nabble.com/accumulo-thrift-server-error-tp19786.html
> > Sent from the Developers mailing list archive at Nabble.com.
> >
>
--
Christopher
Re: accumulo thrift server error
Posted by Josh Elser <jo...@gmail.com>.
The maximum frame size is being validated by the Thrift Proxy server, not
your client. This is to prevent clients from performing a denial of service
attack against the (shared) service.
There should be a configuration property to control this, but I don't know
what it is off the top of my head (and am on my phone). I would guess that
it's documented on the Accumulo user manual.
On Jan 31, 2017 15:44, "z11373" <z1...@outlook.com> wrote:
> We have C++ client talking to Accumulo server via Thrift, so there are
> Accumulo proxy/thrift instances running on the server. The client got
> exception, and upon investigation on Accumulo thrift server, this error
> line
> is found:
>
> 2017-01-31 18:00:11,806 [rpc.CustomNonBlockingServer$CustomFrameBuffer]
> ERROR: Read a frame size of 18986583, which is bigger than the maximum
> allowable buffer size for ALL connections.
>
> This issue is likely the client is trying to write data bigger than 16MB to
> Accumulo thrift, which the max frame size is set to 16MB on the server
> side.
> Therefore, I made change on client side to explicitly call
> apache::thrift::transport::TFramedTransport::setMaxFrameSize and pass 16 *
> 1024 * 1024 value (or 16MB).
>
> Unfortunately I still hit same issue, so either I missed something, or what
> I did was wrong. If you have experience on Thrift, or ever ran into this
> issue, let me know how to workaround this issue.
>
> Thanks,
> Z
>
>
>
> --
> View this message in context: http://apache-accumulo.
> 1065345.n5.nabble.com/accumulo-thrift-server-error-tp19786.html
> Sent from the Developers mailing list archive at Nabble.com.
>