You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@metron.apache.org by as22323 <gi...@git.apache.org> on 2017/09/13 18:51:41 UTC

[GitHub] metron pull request #754: METRON-1184 EC2 Deployment - Updating control_path...

GitHub user as22323 opened a pull request:

    https://github.com/apache/metron/pull/754

    METRON-1184 EC2 Deployment - Updating control_path to accommodate for Linux

    ## Contributor Comments - EC2 Deployment - Updating control_path to accommodate for Linux
    In Metron 0.4.1-rc4, using "control_path = ~/.ssh/ansible-ssh-%%C"(amazon-ec2/ansible.cfg) in a Linux environment ( such as ClearOS release 7.4.0[ClearOS is based on CentOS] with npm 3.10.10, node 6.11.1, docker 1.12.6, mvn 3.3.9, ansible 2.2.2.0, No Vagrant) could throw the following error:
    ```
    "TASK [setup] *******************************************************************
    fatal: [ec2-xx-yy-0-130.us-west-2.compute.amazonaws.com]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: percent_expand: unknown key %C\r\n", "unreachable": true}" 
    ```
    I noticed the line was formally "control_path = %(directory)s/%%h-%%p-%%r" (which worked for the Linux env above)] was changed to "control_path = ~/.ssh/ansible-ssh-%%C"
    
    This PR is to change the line to "control_path = ~/.ssh/ansbile-ssh-%%h-%%r" to make amazon-ec2 also potentially deployable with Linux while preserving a small path length. 
    
    This is an attempt to resolve the path length issue as described in the following links:
    http://docs.ansible.com/ansible/latest/intro_configuration.html  - see control_path
    https://github.com/ansible/ansible/issues/11536#issuecomment-153030743 - using %%C
    
    ---------------
    Testing method: 
    Made change to ansible.cfg file (in Metron 0.4.1-rc4) as described in this PR then ran ./run.sh (metron/metron-deployment/amazon-ec2/).
    If successful then the following should appear: 
    ```
    ok: [xxxxxxxxx.compute.amazonaws.com] => {
        "Success": [
            "Apache Metron deployed successfully",
            "   Metron          @ http://xxxxxxxxx.compute.amazonaws.com:5000",
            "   Ambari          @ http://xxxxxxxxx.compute.amazonaws.com:8080",
            "   Sensor Status   @ http://xxxxxxxxx.compute.amazonaws.com:2812",
            "   Zookeeper       @ xxxxxxxxx.compute.amazonaws.com:2181",
            "   Kafka           @ xxxxxxxxx.compute.amazonaws.com:6667,xxxxxxxxx..compute.amazonaws.com:6667,xxxxxxxxx.compute.amazonaws.com:6667,xxxxxxxxx.compute.amazonaws.com:6667,xxxxxxxxx.compute.amazonaws.com:6667",
            "For additional information, see https://metron.apache.org/'"
        ]
    }
    PLAY RECAP *********************************************************************
    xxxxxxxxx..compute.amazonaws.com : ok=44   changed=28   unreachable=0    failed=0
    xxxxxxxxx..compute.amazonaws.com : ok=44   changed=28   unreachable=0    failed=0
    xxxxxxxxx..compute.amazonaws.com : ok=44   changed=28   unreachable=0    failed=0
    xxxxxxxxx..compute.amazonaws.com : ok=44   changed=28   unreachable=0    failed=0
    xxxxxxxxx..compute.amazonaws.com : ok=204  changed=112  unreachable=0    failed=0
    xxxxxxxxx..compute.amazonaws.com : ok=50   changed=32   unreachable=0    failed=0
    xxxxxxxxx..compute.amazonaws.com : ok=44   changed=28   unreachable=0    failed=0
    xxxxxxxxx..compute.amazonaws.com : ok=79   changed=26   unreachable=0    failed=0
    xxxxxxxxx..compute.amazonaws.com : ok=44   changed=28   unreachable=0    failed=0
    xxxxxxxxx..compute.amazonaws.com : ok=44   changed=28   unreachable=0    failed=0
    localhost                  : ok=20   changed=16   unreachable=0    failed=0
    
    
    ```
    
    
    
    
    ## Pull Request Checklist
    
    Thank you for submitting a contribution to Apache Metron.  
    Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.  
    Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.  
    
    
    In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
    
    ### For all changes:
    - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). 
    
    https://issues.apache.org/jira/browse/METRON-1184
    
    - [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
    - [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
    
    
    ### For code changes:
    - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
    - [ ] Have you included steps or a guide to how the change may be verified and tested manually?
    - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
      ```
      mvn -q clean integration-test install && build_utils/verify_licenses.sh 
      ```
    
    - [ ] Have you written or updated unit tests and or integration tests to verify your changes?
    - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? 
    - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
    
    ### For documentation related changes:
    - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
    
      ```
      cd site-book
      mvn site
      ```
    
    #### Note:
    Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
    It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/as22323/metron patch-1

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/metron/pull/754.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #754
    
----
commit 9b893959a37631f7cd07bf850c011b9b5b7ac07c
Author: as22323 <a....@ieee.org>
Date:   2017-09-13T17:52:24Z

    Update ansible.cfg
    
    For 0.4.1-rc4, using "control_path = ~/.ssh/ansible-ssh-%%C" for a Linux environment ( such as ClearOS[Distro of CentOS] npm 3.10.10, node 6.11.1, docker 1.12.6, mvn 3.3.9, ansible 2.2.2.0, No Vagrant) could throw the following error:
    ```
    "TASK [setup] *******************************************************************
    fatal: [ec2-xx-yy-0-130.us-west-2.compute.amazonaws.com]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: percent_expand: unknown key %C\r\n", "unreachable": true}" 
    ```
    I noticed the line was formally "control_path = %(directory)s/%%h-%%p-%%r" (which worked for the Linux env above) was changed to "control_path = ~/.ssh/ansible-ssh-%%C"
    
    This PR is to change the line to "control_path = ~/.ssh/ansbile-ssh-%%h-%%r" to make amazon-ec2 also potentially deployable with Linux.
    
    This is an attempt to resolve the path length issue as described in the following links:
    http://docs.ansible.com/ansible/latest/intro_configuration.html  - see control_path
    https://github.com/ansible/ansible/issues/11536#issuecomment-153030743 - using %%C

----


---

[GitHub] metron issue #754: METRON-1184 EC2 Deployment - Updating control_path to acc...

Posted by as22323 <gi...@git.apache.org>.

Github user as22323 commented on the issue:

    https://github.com/apache/metron/pull/754
  
    Could the following approach be taken?: 
    
    In metron-deployment/amazon-ec2/README.md add two new "Errors" in the "Common Error Section"
    
    1) 
    # Error: ''Failed to connect to the host via ssh: percent_expand: unknown key %C\r\n"
    
    This error might occur if you are deploying with Linux OS. The %C in the control_path is compatible with Mac OS but might not be compatible certain Linux operating systems.
    
    ## Solution
    If you see this error update the control_path  in ansble.cfg  to the following. 
    ```
    control_path = ~/.ssh/ansbile-ssh-%%h-%%r
    ```
    
    2) 
    # Error: ''too long for Unix domain socket\rn\n"
    This error might occur if you are deploying with Mac OS.
    
    ## Solution
    If you see this error when deploying with a Mac update the control_path in  ansble.cfg to the following.
    ```
    control_path = ~/.ssh/ansible-ssh-%%C
    ```
    
    -----------------------
    Update ansble.cfg to include comments: 
    ```
    #When deploying with Mac OS use this control_path if you receive the following error "too long for Unix domain socket":
    [ssh_connection]
    control_path = ~/.ssh/ansible-ssh-%%C
    
    #When deploying with Linux OS use this control_path if you receive the following error "Failed to connect to the host via ssh: percent_expand: unknown key %C":
    #[ssh_connection]
    #control_path = ~/.ssh/ansbile-ssh-%%h-%%r
    ```


---

[GitHub] metron issue #754: METRON-1184 EC2 Deployment - Updating control_path to acc...

Posted by ottobackwards <gi...@git.apache.org>.

Github user ottobackwards commented on the issue:

    https://github.com/apache/metron/pull/754
  
    Thanks for the contribution!  Please take care of the Jira


---

[GitHub] metron issue #754: METRON-1184 EC2 Deployment - Updating control_path to acc...

Posted by nickwallen <gi...@git.apache.org>.

Github user nickwallen commented on the issue:

    https://github.com/apache/metron/pull/754
  
    @ottobackwards Did you or anyone else happen to test this on a Mac?  I believe @mmiklavc runs a Mac and it is not working for him.  I am curious if this is working for anyone on a Mac.  


---

[GitHub] metron issue #754: METRON-1184 EC2 Deployment - Updating control_path to acc...

Posted by as22323 <gi...@git.apache.org>.

Github user as22323 commented on the issue:

    https://github.com/apache/metron/pull/754
  
    Just an FYI: This PR is to simply change "control_path = ~/.ssh/ansible-ssh-%%C"(in ../amazon-ec2/ansible.cfg)  to "control_path = ~/.ssh/ansbile-ssh-%%h-%%r" to allow linux systems to deploy to EC2. When I tested the change with Mac it didn't throw any errors at the same point for ""TASK [setup] ***"


---

[GitHub] metron issue #754: METRON-1184 EC2 Deployment - Updating control_path to acc...

Posted by lvets <gi...@git.apache.org>.

Github user lvets commented on the issue:

    https://github.com/apache/metron/pull/754
  
    @ottobackwards I will try this out this week.


---

[GitHub] metron issue #754: METRON-1184 EC2 Deployment - Updating control_path to acc...

Posted by ottobackwards <gi...@git.apache.org>.

Github user ottobackwards commented on the issue:

    https://github.com/apache/metron/pull/754
  
    @as22323 I need a real name to use in the commit


---

[GitHub] metron issue #754: METRON-1184 EC2 Deployment - Updating control_path to acc...

Posted by as22323 <gi...@git.apache.org>.

Github user as22323 commented on the issue:

    https://github.com/apache/metron/pull/754
  
    Hello, 
    Just wondering if what needs to be done next to get this committed.
    



---

[GitHub] metron issue #754: METRON-1184 EC2 Deployment - Updating control_path to acc...

Posted by nickwallen <gi...@git.apache.org>.

Github user nickwallen commented on the issue:

    https://github.com/apache/metron/pull/754
  
    @JonZeolla What do you think about this one?  I remember you fought this specific issue a few months back.


---