[jira] [Commented] (METRON-642) Fix ElasticSearch Mpack issues in Metron

["Matt Foley (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/METRON-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15790395#comment-15790395 ] 

Matt Foley commented on METRON-642:
-----------------------------------

Item 1: There is indeed naming confusion about es.ip, es.port, and es_url.  Unfortunately, the UI for this piece of the MPack has been in a state of flux, and in some combinations there is not currently (or no longer?) a bug.  Most importantly, the code that consumes this value (es.ip) actually has parsing logic that does in fact consume URLs rather than just host names here; please check out metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/writer/ElasticsearchWriter.java::getIps()
This issue is not currently in METRON-634 because I concluded the current behavior in master branch is not buggy.
It can and should be enhanced to fix the naming confusion, but there are several ways to do so and care must be taken to be up to date with current master branch, first, and understand the consumer code in ElasticsearchWriter.java.

Item 2: I admit I forgot to include this in METRON-634, but I've added it now.  But in addition, backslashes need to be added to the ends of lines in ES_JAVA_OPTS, as noted in METRON-634.

Item 3: This is a dup of an item in METRON-634.  (Search for "$CONF_DIR/scripts".)  In current master branch, only the scripts subdirectory doesn't yet have permissions set correctly, the logging.yml file is fine.

> Fix ElasticSearch Mpack issues in Metron
> ----------------------------------------
>
>                 Key: METRON-642
>                 URL: https://issues.apache.org/jira/browse/METRON-642
>             Project: Metron
>          Issue Type: Bug
>    Affects Versions: 0.3.0
>            Reporter: Dima Kovalyov
>            Priority: Minor
>
> ==== First ====
> Currently in Metron Mpack es.ip in Metron configuration is set to: es_url.
> Which is incorrect and will fail, instead it should be replaced with:
> "es.ip": "<elasticsearch_master_hostname>",
> "es.port": "9300", 
> But I do not know how to get elasticsearch_master_hostname. Also, value of es.port seems like can be defaulted to es_binary_port or even omitted as it is already specified in es_binary_port.
> This should be chanted in:
> https://github.com/apache/incubator-metron/blob/master/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-env.xml
> On line 208.
> ==== First ====
> ==== Second ====
> In file https://github.com/apache/incubator-metron/blob/master/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/ELASTICSEARCH/2.3.3/configuration/elastic-sysconfig.xml
> On lines 93 and 95 symbols "/" is forgotten. e.g:
> ES_JAVA_OPTS="-verbose:gc -Xloggc:{{log_dir}}/elasticsearch_gc.log -XX:-CMSConcurrentMTEnabled
> should be:
> ES_JAVA_OPTS="-verbose:gc -Xloggc:{{log_dir}}/elasticsearch_gc.log -XX:-CMSConcurrentMTEnabled
> ==== Second ====
> ==== Third ====
> After ES install it will fail to start with error"
> {code}
> Likely root cause: java.nio.file.AccessDeniedException: /etc/elasticsearch/scripts
> {code}
> The reason for the is root permission on these files:
> {code}
>  # ls -la /etc/elasticsearch
> ...
> -rwxr-x---.   1 root          elasticsearch  2571 May 12 09:24 logging.yml
> drwxr-x---.   2 root          elasticsearch  4096 May 17 11:49 scripts
> {code}
> I was not able to trace down origin of these files, so do not know how to fix it.
> ==== Third ====



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)