You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Davorin Vukelić <da...@crossing-technologies.com> on 2016/10/25 12:16:25 UTC
HIveBolt
HI,
can someone explain to me how to adjust HiveBolt? I use storm-hive
0.10.1 library.
We made topology which prepares 500 messages per second for storing
into Hive.
I frequently get this ERROR:
org.apache.storm.hive.common.HiveWriter$TxnFailure: Failed switching to
next Txn in TxnBatch TxnIds=[21862479...21862481] on endPoint
={metaStoreUri='thrift://hostname:9083', database='database',
table='table', partitionVals=[xxxxx, 2016-10-18] }
.......
Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable
to acquire lock on {metaStoreUri='.......
.........
Caused by: org.apache.hadoop.hive.metastore.api.TxnAbortedException:
Transaction txnid:21862481 already aborted
.........
What happened with transaction 21862481? Will tuples/rows in this
transaction be deleted or HiveBolt will try to commit this transaction
again?
Lp,
*Davorin Vukelic!*
Re: HIveBolt
Posted by Harsha Chintalapani <st...@harsha.io>.
HI Davorin,
I recommend using hive 1.x or higher release. They had quite
few issues in hive streaming api and also we had issues on heartbeating
from storm hive side. These are all
fixed in storm 1.0 bolt and hive 1.0 had streaming api fixes as well.
Thanks,
Harsha
On Tue, Oct 25, 2016 at 5:06 PM Aaron Niskodé-Dossett <do...@gmail.com>
wrote:
> If you are using a current version of the HiveBolt then those tuples will
> not be acknowledged and will eventually be retried.
>
> On Tue, Oct 25, 2016 at 7:22 AM Davorin Vukelić <
> davorin@crossing-technologies.com> wrote:
>
> HI,
>
> can someone explain to me how to adjust HiveBolt? I use storm-hive 0.10.1
> library.
> We made topology which prepares 500 messages per second for storing into
> Hive.
>
> I frequently get this ERROR:
>
> org.apache.storm.hive.common.HiveWriter$TxnFailure: Failed switching to
> next Txn in TxnBatch TxnIds=[21862479...21862481] on endPoint
> ={metaStoreUri='thrift://hostname:9083', database='database',
> table='table', partitionVals=[xxxxx, 2016-10-18] }
> .......
> Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable to
> acquire lock on {metaStoreUri='.......
> .........
> Caused by: org.apache.hadoop.hive.metastore.api.TxnAbortedException:
> Transaction txnid:21862481 already aborted
> .........
>
> What happened with transaction 21862481? Will tuples/rows in this
> transaction be deleted or HiveBolt will try to commit this transaction
> again?
>
> Lp,
>
> *Davorin Vukelic!*
>
>
Re: HIveBolt
Posted by Aaron Niskodé-Dossett <do...@gmail.com>.
If you are using a current version of the HiveBolt then those tuples will
not be acknowledged and will eventually be retried.
On Tue, Oct 25, 2016 at 7:22 AM Davorin Vukelić <
davorin@crossing-technologies.com> wrote:
> HI,
>
> can someone explain to me how to adjust HiveBolt? I use storm-hive 0.10.1
> library.
> We made topology which prepares 500 messages per second for storing into
> Hive.
>
> I frequently get this ERROR:
>
> org.apache.storm.hive.common.HiveWriter$TxnFailure: Failed switching to
> next Txn in TxnBatch TxnIds=[21862479...21862481] on endPoint
> ={metaStoreUri='thrift://hostname:9083', database='database',
> table='table', partitionVals=[xxxxx, 2016-10-18] }
> .......
> Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable to
> acquire lock on {metaStoreUri='.......
> .........
> Caused by: org.apache.hadoop.hive.metastore.api.TxnAbortedException:
> Transaction txnid:21862481 already aborted
> .........
>
> What happened with transaction 21862481? Will tuples/rows in this
> transaction be deleted or HiveBolt will try to commit this transaction
> again?
>
> Lp,
>
> *Davorin Vukelic!*
>