You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by Kevin Risden <kr...@apache.org> on 2017/12/19 21:52:34 UTC
Ranger and Ambari Infra Solr (or another Solr install) - OOM
Ranger uses Ambari Infra Solr (or another Apache Solr install) for storing
Ranger Audit events for displaying in Ranger Admin. In our case, we have
noticed quite a few Ambari Infra Solr OOM due to Ranger. I've talked with a
few other people who are having very similar problems with OOM errors.
I've typed up some details about how the way Ranger is using Solr requires
a lot of heap. I've also outlined the fix for this which significantly
reduced the amount of heap memory required. I'm an Apache Lucene/Solr
committer so this optimization/usage might not be immediately obvious to
those using Solr especially version 5.x.
https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html
I can open a JIRA issue if this would be helpful to fix the managed-schema
shipped with Ranger. This would also be something that should be fixed
upstream with Ambari if possible when deploying Ranger and Ambari Infra
Solr.
Kevin Risden
Re: Ranger and Ambari Infra Solr (or another Solr install) - OOM
Posted by Sean Roberts <sr...@hortonworks.com>.
Kevin – Good article. Definitely get this to a JIRA!
I have previously done some of the same improvements as you, but not all that you covered.
--
Sean Roberts
@seano
From: Velmurugan Periasamy <vp...@hortonworks.com>
Reply-To: "user@ranger.apache.org" <us...@ranger.apache.org>
Date: Tuesday, December 19, 2017 at 9:58 PM
To: "user@ranger.apache.org" <us...@ranger.apache.org>
Cc: "dev@ranger.apache.org" <de...@ranger.apache.org>
Subject: Re: Ranger and Ambari Infra Solr (or another Solr install) - OOM
This is a good idea. Thanks Kevin. Please go ahead with JIRA. Adding dev list.
From: Kevin Risden <kr...@apache.org>>
Reply-To: "user@ranger.apache.org<ma...@ranger.apache.org>" <us...@ranger.apache.org>>
Date: Tuesday, December 19, 2017 at 4:52 PM
To: "user@ranger.apache.org<ma...@ranger.apache.org>" <us...@ranger.apache.org>>
Subject: Ranger and Ambari Infra Solr (or another Solr install) - OOM
Ranger uses Ambari Infra Solr (or another Apache Solr install) for storing Ranger Audit events for displaying in Ranger Admin. In our case, we have noticed quite a few Ambari Infra Solr OOM due to Ranger. I've talked with a few other people who are having very similar problems with OOM errors.
I've typed up some details about how the way Ranger is using Solr requires a lot of heap. I've also outlined the fix for this which significantly reduced the amount of heap memory required. I'm an Apache Lucene/Solr committer so this optimization/usage might not be immediately obvious to those using Solr especially version 5.x.
https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html
I can open a JIRA issue if this would be helpful to fix the managed-schema shipped with Ranger. This would also be something that should be fixed upstream with Ambari if possible when deploying Ranger and Ambari Infra Solr.
Kevin Risden
Re: Ranger and Ambari Infra Solr (or another Solr install) - OOM
Posted by Kevin Risden <kr...@apache.org>.
Created https://issues.apache.org/jira/browse/RANGER-1938
I'll move more details from the blog post into the ticket and should be
able to come up with the simple patch.
Kevin Risden
On Tue, Dec 19, 2017 at 3:58 PM, Velmurugan Periasamy <
vperiasamy@hortonworks.com> wrote:
> This is a good idea. Thanks Kevin. Please go ahead with JIRA. Adding dev
> list.
>
> From: Kevin Risden <kr...@apache.org>>
> Reply-To: "user@ranger.apache.org<ma...@ranger.apache.org>" <
> user@ranger.apache.org<ma...@ranger.apache.org>>
> Date: Tuesday, December 19, 2017 at 4:52 PM
> To: "user@ranger.apache.org<ma...@ranger.apache.org>" <
> user@ranger.apache.org<ma...@ranger.apache.org>>
> Subject: Ranger and Ambari Infra Solr (or another Solr install) - OOM
>
> Ranger uses Ambari Infra Solr (or another Apache Solr install) for storing
> Ranger Audit events for displaying in Ranger Admin. In our case, we have
> noticed quite a few Ambari Infra Solr OOM due to Ranger. I've talked with a
> few other people who are having very similar problems with OOM errors.
>
> I've typed up some details about how the way Ranger is using Solr requires
> a lot of heap. I've also outlined the fix for this which significantly
> reduced the amount of heap memory required. I'm an Apache Lucene/Solr
> committer so this optimization/usage might not be immediately obvious to
> those using Solr especially version 5.x.
>
> https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html
>
> I can open a JIRA issue if this would be helpful to fix the managed-schema
> shipped with Ranger. This would also be something that should be fixed
> upstream with Ambari if possible when deploying Ranger and Ambari Infra
> Solr.
>
> Kevin Risden
>
Re: Ranger and Ambari Infra Solr (or another Solr install) - OOM
Posted by Kevin Risden <kr...@apache.org>.
Created https://issues.apache.org/jira/browse/RANGER-1938
I'll move more details from the blog post into the ticket and should be
able to come up with the simple patch.
Kevin Risden
On Tue, Dec 19, 2017 at 3:58 PM, Velmurugan Periasamy <
vperiasamy@hortonworks.com> wrote:
> This is a good idea. Thanks Kevin. Please go ahead with JIRA. Adding dev
> list.
>
> From: Kevin Risden <kr...@apache.org>>
> Reply-To: "user@ranger.apache.org<ma...@ranger.apache.org>" <
> user@ranger.apache.org<ma...@ranger.apache.org>>
> Date: Tuesday, December 19, 2017 at 4:52 PM
> To: "user@ranger.apache.org<ma...@ranger.apache.org>" <
> user@ranger.apache.org<ma...@ranger.apache.org>>
> Subject: Ranger and Ambari Infra Solr (or another Solr install) - OOM
>
> Ranger uses Ambari Infra Solr (or another Apache Solr install) for storing
> Ranger Audit events for displaying in Ranger Admin. In our case, we have
> noticed quite a few Ambari Infra Solr OOM due to Ranger. I've talked with a
> few other people who are having very similar problems with OOM errors.
>
> I've typed up some details about how the way Ranger is using Solr requires
> a lot of heap. I've also outlined the fix for this which significantly
> reduced the amount of heap memory required. I'm an Apache Lucene/Solr
> committer so this optimization/usage might not be immediately obvious to
> those using Solr especially version 5.x.
>
> https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html
>
> I can open a JIRA issue if this would be helpful to fix the managed-schema
> shipped with Ranger. This would also be something that should be fixed
> upstream with Ambari if possible when deploying Ranger and Ambari Infra
> Solr.
>
> Kevin Risden
>
Re: Ranger and Ambari Infra Solr (or another Solr install) - OOM
Posted by Velmurugan Periasamy <vp...@hortonworks.com>.
This is a good idea. Thanks Kevin. Please go ahead with JIRA. Adding dev list.
From: Kevin Risden <kr...@apache.org>>
Reply-To: "user@ranger.apache.org<ma...@ranger.apache.org>" <us...@ranger.apache.org>>
Date: Tuesday, December 19, 2017 at 4:52 PM
To: "user@ranger.apache.org<ma...@ranger.apache.org>" <us...@ranger.apache.org>>
Subject: Ranger and Ambari Infra Solr (or another Solr install) - OOM
Ranger uses Ambari Infra Solr (or another Apache Solr install) for storing Ranger Audit events for displaying in Ranger Admin. In our case, we have noticed quite a few Ambari Infra Solr OOM due to Ranger. I've talked with a few other people who are having very similar problems with OOM errors.
I've typed up some details about how the way Ranger is using Solr requires a lot of heap. I've also outlined the fix for this which significantly reduced the amount of heap memory required. I'm an Apache Lucene/Solr committer so this optimization/usage might not be immediately obvious to those using Solr especially version 5.x.
https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html
I can open a JIRA issue if this would be helpful to fix the managed-schema shipped with Ranger. This would also be something that should be fixed upstream with Ambari if possible when deploying Ranger and Ambari Infra Solr.
Kevin Risden
Re: Ranger and Ambari Infra Solr (or another Solr install) - OOM
Posted by Velmurugan Periasamy <vp...@hortonworks.com>.
This is a good idea. Thanks Kevin. Please go ahead with JIRA. Adding dev list.
From: Kevin Risden <kr...@apache.org>>
Reply-To: "user@ranger.apache.org<ma...@ranger.apache.org>" <us...@ranger.apache.org>>
Date: Tuesday, December 19, 2017 at 4:52 PM
To: "user@ranger.apache.org<ma...@ranger.apache.org>" <us...@ranger.apache.org>>
Subject: Ranger and Ambari Infra Solr (or another Solr install) - OOM
Ranger uses Ambari Infra Solr (or another Apache Solr install) for storing Ranger Audit events for displaying in Ranger Admin. In our case, we have noticed quite a few Ambari Infra Solr OOM due to Ranger. I've talked with a few other people who are having very similar problems with OOM errors.
I've typed up some details about how the way Ranger is using Solr requires a lot of heap. I've also outlined the fix for this which significantly reduced the amount of heap memory required. I'm an Apache Lucene/Solr committer so this optimization/usage might not be immediately obvious to those using Solr especially version 5.x.
https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html
I can open a JIRA issue if this would be helpful to fix the managed-schema shipped with Ranger. This would also be something that should be fixed upstream with Ambari if possible when deploying Ranger and Ambari Infra Solr.
Kevin Risden