Exception at inserting big amount of data

[Juan Pablo Briganti <ju...@globant.com>]

Hi!!

  We are facing some errors trying to insert multiple records into the
database through the Java API.
  We have a simple cluster composed of 1 master node and 1 slave node. 1
table with 1 bigint primary key and 1 string. The String length is more or
less 30 characters. The table has a replication of 2.
  We are using 0.8 on both the cluster and java API and performing Manual
Flush mode with a maximum of 10.000 rows per flush (it is only a maximum,
not all flush operations insert the same amount of data).
  After 2 or 3 successful flushes (more or less 6000 records) we are
receiving the error attached to this email.
  We started receiving this error a few weeks ago, when we were using the
0.7.1 version.
  Did this happen to other people? any ideas on what can be wrong? Any help
would be appreciated.
  If you need more info let me know.

Thanks, Juan Pablo.

-- 
*Juan Pablo Briganti* | Data Architect
*GLOBANT* | AR: +54 11 4109 1700 ext. 19508 | US: +1 877 215 5230 ext. 19508
|
[image: Facebook] <https://www.facebook.com/Globant> [image: Twitter]
<http://www.twitter.com/globant> [image: Youtube]
<http://www.youtube.com/Globant> [image: Linkedin]
<http://www.linkedin.com/company/globant> [image: Pinterest]
<http://pinterest.com/globant/> [image: Globant] <http://www.globant.com>

-- 


The information contained in this e-mail may be confidential. It has been 
sent for the sole use of the intended recipient(s). If the reader of this 
message is not an intended recipient, you are hereby notified that any 
unauthorized review, use, disclosure, dissemination, distribution or 
copying of this communication, or any of its contents, 
is strictly prohibited. If you have received it by mistake please let us 
know by e-mail immediately and delete it from your system. Many thanks.

 

La información contenida en este mensaje puede ser confidencial. Ha sido 
enviada para el uso exclusivo del destinatario(s) previsto. Si el lector de 
este mensaje no fuera el destinatario previsto, por el presente queda Ud. 
notificado que cualquier lectura, uso, publicación, diseminación, 
distribución o copiado de esta comunicación o su contenido está 
estrictamente prohibido. En caso de que Ud. hubiera recibido este mensaje 
por error le agradeceremos notificarnos por e-mail inmediatamente y 
eliminarlo de su sistema. Muchas gracias.

Re: Exception at inserting big amount of data

[Jean-Daniel Cryans <jd...@apache.org>]

Hi Juan Pablo,

The error basically means that the client didn't hear from the server after
sending the data, even after retrying a few times, and reached the default
10 seconds timeout. Can you run your insert again and then capture the
output of this command?

curl -s http://10.0.6.157:8050/metrics | gzip - > metrics.gz

Then post that file somewhere we can download. I you have more than one
tablet server, it might be a different node, basically I want the one that
ends up listed in this exception on the right:

Caused by: org.kududb.client.ConnectionResetException: [Peer
f7e2936b040d4c58b52d90ae50ad6d5a] Connection reset on [id: 0x323019c2, /
10.0.6.6:58930 :> /10.0.6.157:7050]

Also, can we see the logs from that node around 10AM on 16/04/26?

Finally, I'm surprised you're even able to create your table if you only
have one tablet server and a replication of 2 (unless you meant to say that
your master node has both a master and a tablet server).

J-D

On Tue, Apr 26, 2016 at 7:06 AM, Juan Pablo Briganti <
juan.briganti@globant.com> wrote:

> Hi!!
>
>   We are facing some errors trying to insert multiple records into the
> database through the Java API.
>   We have a simple cluster composed of 1 master node and 1 slave node. 1
> table with 1 bigint primary key and 1 string. The String length is more or
> less 30 characters. The table has a replication of 2.
>   We are using 0.8 on both the cluster and java API and performing Manual
> Flush mode with a maximum of 10.000 rows per flush (it is only a maximum,
> not all flush operations insert the same amount of data).
>   After 2 or 3 successful flushes (more or less 6000 records) we are
> receiving the error attached to this email.
>   We started receiving this error a few weeks ago, when we were using the
> 0.7.1 version.
>   Did this happen to other people? any ideas on what can be wrong? Any
> help would be appreciated.
>   If you need more info let me know.
>
> Thanks, Juan Pablo.
>
> --
> *Juan Pablo Briganti* | Data Architect
> *GLOBANT* | AR: +54 11 4109 1700 ext. 19508 | US: +1 877 215 5230 ext.
> 19508 |
> [image: Facebook] <https://www.facebook.com/Globant> [image: Twitter]
> <http://www.twitter.com/globant> [image: Youtube]
> <http://www.youtube.com/Globant> [image: Linkedin]
> <http://www.linkedin.com/company/globant> [image: Pinterest]
> <http://pinterest.com/globant/> [image: Globant] <http://www.globant.com>
>
> The information contained in this e-mail may be confidential. It has been
> sent for the sole use of the intended recipient(s). If the reader of this
> message is not an intended recipient, you are hereby notified that any
> unauthorized review, use, disclosure, dissemination, distribution or
> copying of this communication, or any of its contents,
> is strictly prohibited. If you have received it by mistake please let us
> know by e-mail immediately and delete it from your system. Many thanks.
>
>
>
> La información contenida en este mensaje puede ser confidencial. Ha sido
> enviada para el uso exclusivo del destinatario(s) previsto. Si el lector de
> este mensaje no fuera el destinatario previsto, por el presente queda Ud.
> notificado que cualquier lectura, uso, publicación, diseminación,
> distribución o copiado de esta comunicación o su contenido está
> estrictamente prohibido. En caso de que Ud. hubiera recibido este mensaje
> por error le agradeceremos notificarnos por e-mail inmediatamente y
> eliminarlo de su sistema. Muchas gracias.
>
>