You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by z11373 <z1...@outlook.com> on 2016/04/25 16:49:37 UTC
c++ thrift client crash
Hi,
Anyone here has experience writing C++ client code using Accumulo Thrift
proxy.
My C++ code crash when trying to add mutation, and the proxy spits out this
callstack:
2016-04-25 14:28:33,952 [server.AbstractNonblockingServer$FrameBuffer] WARN
: Got an IOException in internalRead!
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
at
org.apache.thrift.transport.TNonblockingSocket.read(TNonblockingSocket.java:141)
at
org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.internalRead(AbstractNonblockingServer.java:537)
at
org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.read(AbstractNonblockingServer.java:338)
at
org.apache.thrift.server.AbstractNonblockingServer$AbstractSelectThread.handleRead(AbstractNonblockingServer.java:203)
at
org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.select(TNonblockingServer.java:202)
at
org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.run(TNonblockingServer.java:158)
I can't figure out what's wrong with the function 'addMutation' in our c++
code below. Any idea or pointer?
BatchWriter::BatchWriter(std::shared_ptr<::accumulo::AccumuloProxyClient>
proxyClient,
const std::string& login,
const std::string& tableName,
const int64_t maxMemory,
const int64_t latencyMs,
const int64_t timeoutMs,
const int32_t numThreads)
: d_client(proxyClient)
, d_login(login)
, d_tableName(tableName)
{
::accumulo::WriterOptions writerOptions;
writerOptions.__set_maxMemory(maxMemory);
writerOptions.__set_latencyMs(latencyMs);
writerOptions.__set_timeoutMs(timeoutMs);
writerOptions.__set_threads(numThreads);
BOOST_ASSERT_MSG(d_client, "Client must be valid pointer");
d_client->createWriter(d_writerToken, login, tableName, writerOptions);
}
void BatchWriter::addMutation(Mutation& mutation)
{
std::map<std::string, std::vector<::accumulo::ColumnUpdate> > cells;
const std::vector<::accumulo::ColumnUpdate> updates =
mutation.getUpdates();
const std::string rowId = mutation.getRowId();
cells.insert(std::make_pair(rowId, updates));
d_client->update(d_writerToken, cells);
mutation.clear();
}
Thanks,
Z
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/c-thrift-client-crash-tp16741.html
Sent from the Developers mailing list archive at Nabble.com.
Re: c++ thrift client crash
Posted by z11373 <z1...@outlook.com>.
Weird, it looks like somehow my object's close method being called twice (I
guess maybe I did instance copy). I managed to find a way to fix it, and no
longer seeing that exception. Thanks again.
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/c-thrift-client-crash-tp16741p16779.html
Sent from the Developers mailing list archive at Nabble.com.
Re: c++ thrift client crash
Posted by "Marc P." <ma...@gmail.com>.
I'm sorry, I'm not terribly familiar to the proxy as I was making the RPC
calls in my client. I did not come across that exception. Do you know if
that is your local client or an exception generated in the proxy and
returned to you?
On Tue, Apr 26, 2016 at 11:30 AM z11373 <z1...@outlook.com> wrote:
> Marc, have you seen this exception before?
> The write was successful, the code closes the writer and connection when
> the
> object is destroyed, which I believe throwing this exception somehow.
>
> terminate called after throwing an instance of
> 'apache::thrift::transport::TTransportException'
> what(): Called write on non-open socket
> Abort(coredump)
>
>
> Thanks,
> Z
>
>
>
> --
> View this message in context:
> http://apache-accumulo.1065345.n5.nabble.com/c-thrift-client-crash-tp16741p16767.html
> Sent from the Developers mailing list archive at Nabble.com.
>
Re: c++ thrift client crash
Posted by z11373 <z1...@outlook.com>.
Marc, have you seen this exception before?
The write was successful, the code closes the writer and connection when the
object is destroyed, which I believe throwing this exception somehow.
terminate called after throwing an instance of
'apache::thrift::transport::TTransportException'
what(): Called write on non-open socket
Abort(coredump)
Thanks,
Z
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/c-thrift-client-crash-tp16741p16767.html
Sent from the Developers mailing list archive at Nabble.com.
Re: c++ thrift client crash
Posted by z11373 <z1...@outlook.com>.
Thanks Marc! It's actually my bad in this case :-(
The client crash, hence the proxy reporting that 'connection reset by peer'
message. Earlier I thought the proxy threw the error and causing my client
crash, but it's actually another way around. I'll keep in mind your other
tips in case I encounter any of them in the future.
Thanks,
Z
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/c-thrift-client-crash-tp16741p16756.html
Sent from the Developers mailing list archive at Nabble.com.
Re: c++ thrift client crash
Posted by "Marc P." <ma...@gmail.com>.
I ran into this a lot when performing operations like bulk load, writing,
and scanning accumulo from c++. I wasn't using the proxy, but I ran into
this a lot
1) My first likely cause was issues of endianness. I assume the proxy takes
care of this....so perhaps this is moot.
2) The next most likely culprit was pointer mismanagement or scope
problems. Does d_client's update call occur asynchronously or will it
complete the write before that function returns?
3) Invalid thrift operation. Did you do a startUpdate somewhere? I don't
see it. Doesn't mean it didn't occur...but can I see the code where it
occurred?
Hi,
> Anyone here has experience writing C++ client code using Accumulo Thrift
> proxy.
> My C++ code crash when trying to add mutation, and the proxy spits out this
> callstack:
>
> 2016-04-25 14:28:33,952 [server.AbstractNonblockingServer$FrameBuffer] WARN
> : Got an IOException in internalRead!
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> at sun.nio.ch.IOUtil.read(IOUtil.java:197)
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
> at
>
> org.apache.thrift.transport.TNonblockingSocket.read(TNonblockingSocket.java:141)
> at
>
> org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.internalRead(AbstractNonblockingServer.java:537)
> at
>
> org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.read(AbstractNonblockingServer.java:338)
> at
>
> org.apache.thrift.server.AbstractNonblockingServer$AbstractSelectThread.handleRead(AbstractNonblockingServer.java:203)
> at
>
> org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.select(TNonblockingServer.java:202)
> at
>
> org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.run(TNonblockingServer.java:158)
>
> I can't figure out what's wrong with the function 'addMutation' in our c++
> code below. Any idea or pointer?
>
> BatchWriter::BatchWriter(std::shared_ptr<::accumulo::AccumuloProxyClient>
> proxyClient,
> const std::string& login,
> const std::string& tableName,
> const int64_t maxMemory,
> const int64_t latencyMs,
> const int64_t timeoutMs,
> const int32_t numThreads)
> : d_client(proxyClient)
> , d_login(login)
> , d_tableName(tableName)
> {
> ::accumulo::WriterOptions writerOptions;
> writerOptions.__set_maxMemory(maxMemory);
> writerOptions.__set_latencyMs(latencyMs);
> writerOptions.__set_timeoutMs(timeoutMs);
> writerOptions.__set_threads(numThreads);
> BOOST_ASSERT_MSG(d_client, "Client must be valid pointer");
> d_client->createWriter(d_writerToken, login, tableName,
> writerOptions);
> }
>
> void BatchWriter::addMutation(Mutation& mutation)
> {
> std::map<std::string, std::vector<::accumulo::ColumnUpdate> >
> cells;
>
> const std::vector<::accumulo::ColumnUpdate> updates =
> mutation.getUpdates();
> const std::string rowId = mutation.getRowId();
>
> cells.insert(std::make_pair(rowId, updates));
>
> d_client->update(d_writerToken, cells);
>
> mutation.clear();
> }
>
>
> Thanks,
> Z
>
>
>
>
> --
> View this message in context:
>
> http://apache-accumulo.1065345.n5.nabble.com/c-thrift-client-crash-tp16741.html
> Sent from the Developers mailing list archive at Nabble.com.
>