You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "Nick Allen (JIRA)" <ji...@apache.org> on 2017/10/25 17:57:01 UTC

[jira] [Commented] (METRON-1279) Cannot Add Alert to an Already Large Meta-Alert

    [ https://issues.apache.org/jira/browse/METRON-1279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219187#comment-16219187 ] 

Nick Allen commented on METRON-1279:
------------------------------------

I think the serialized JSON is just too large with these ~500 alerts when it attempts to write them to HBase. I think we impose a limit of 999 currently, but I'm not sure a fixed limit is going to always work. It depends on the size of each alert.

> Cannot Add Alert to an Already Large Meta-Alert
> -----------------------------------------------
>
>                 Key: METRON-1279
>                 URL: https://issues.apache.org/jira/browse/METRON-1279
>             Project: Metron
>          Issue Type: Bug
>    Affects Versions: 0.4.1
>            Reporter: Nick Allen
>              Labels: Alerts-UI
>         Attachments: 31972179-5e89d79c-b8ed-11e7-9e7a-7ac6c0b718c8.png, 31972201-78e06b24-b8ed-11e7-80c5-4bb01a3e72b7.png
>
>
> If I have a metaalert with a large number of alerts (say 525 in this example), then I attempt to add 1 or 2 alerts to that metaalert, the REST API responds back with a 500 error.
> The UI itself never reports an error or acknowledges success/failure.  The open dialogue just remains on-screen.
> But I do find that the metaalert was actually updated, as the number of alerts increased (from 527 to 529 here).
> {code}
> fullMessage":"IOException: class org.apache.metron.indexing.dao.HBaseDao: KeyValue size too large
> java.lang.IllegalArgumentException: KeyValue size too large
> \tat org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:1521)
> \tat org.apache.hadoop.hbase.client.BufferedMutatorImpl.validatePut(BufferedMutatorImpl.java:147)
> \tat org.apache.hadoop.hbase.client.BufferedMutatorImpl.doMutate(BufferedMutatorImpl.java:134)
> \tat org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:105)
> \tat org.apache.hadoop.hbase.client.HTable.put(HTable.java:1050)
> \tat org.apache.metron.indexing.dao.HBaseDao.batchUpdate(HBaseDao.java:140)
> \tat org.apache.metron.indexing.dao.MultiIndexDao.lambda$batchUpdate$3(MultiIndexDao.java:78)
> \tat java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> \tat java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
> \tat java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> \tat java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> \tat java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:747)
> \tat java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:721)
> \tat java.util.stream.AbstractTask.compute(AbstractTask.java:316)
> \tat java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
> \tat java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
> \tat java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
> \tat java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
> \tat java.util.stream.ReduceOps$ReduceOp.evaluateParallel(ReduceOps.java:714)
> \tat java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
> \tat java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> \tat org.apache.metron.indexing.dao.MultiIndexDao.batchUpdate(MultiIndexDao.java:83)
> \tat org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertDao.indexDaoUpdate(ElasticsearchMetaAlertDao.java:374)
> \tat org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertDao.handleMetaUpdate(ElasticsearchMetaAlertDao.java:361)
> \tat org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertDao.update(ElasticsearchMetaAlertDao.java:248)
> \tat org.apache.metron.indexing.dao.IndexDao.patch(IndexDao.java:130)
> \tat org.apache.metron.rest.service.impl.UpdateServiceImpl.patch(UpdateServiceImpl.java:44)
> \tat org.apache.metron.rest.controller.UpdateController.patch(UpdateController.java:52)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)