You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by 2xyo <gi...@git.apache.org> on 2016/08/21 12:14:46 UTC
[GitHub] incubator-metron pull request #219: Fix support of ansible 2.1 in metron_ela...
GitHub user 2xyo opened a pull request:
https://github.com/apache/incubator-metron/pull/219
Fix support of ansible 2.1 in metron_elasticsearch_templates - Fix #153
This pull request add the option `body_format: "json"` to support the upload of json.
Fix the following error during the installation with ansible 2.1.1.0:
```
TASK [metron_elasticsearch_templates : Wait for Index to Become Available] *****
ok: [node1]
TASK [metron_elasticsearch_templates : Add Elasticsearch templates for topologies] ***
failed: [node1] (item=/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/yaf_index.template) => {"content": "", "failed": true, "item": "/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/yaf_index.template", "msg": "Status code was not [200]: An unknown error occurred: sendall() argument 1 must be string or buffer, not dict", "redirected": false, "status": -1, "url": "http://node1:9200/_template/yaf_index"}
failed: [node1] (item=/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/snort_index.template) => {"content": "", "failed": true, "item": "/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/snort_index.template", "msg": "Status code was not [200]: An unknown error occurred: sendall() argument 1 must be string or buffer, not dict", "redirected": false, "status": -1, "url": "http://node1:9200/_template/snort_index"}
failed: [node1] (item=/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/bro_index.template) => {"content": "", "failed": true, "item": "/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/bro_index.template", "msg": "Status code was not [200]: An unknown error occurred: sendall() argument 1 must be string or buffer, not dict", "redirected": false, "status": -1, "url": "http://node1:9200/_template/bro_index"}
NO MORE HOSTS LEFT *************************************************************
to retry, use: --limit @playbooks/metron_full_install.retry
PLAY RECAP *********************************************************************
node1 : ok=7 changed=3 unreachable=0 failed=1
```
- Fix closed issue #153.
- Not tested with vagrant! (should work..)
- Ansible documentation of `body_format`: http://docs.ansible.com/ansible/uri_module.html and ansible/ansible-modules-core#1011.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/2xyo/incubator-metron patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/219.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #219
----
commit 6651cf198c8e46375e5108558619ad340fa12bc3
Author: 2*yo <yo...@lepage.info>
Date: 2016-08-21T12:14:16Z
Fix support of ansible 2.1 in metron_elasticsearch_templates - Fix #153
This pull request add the option `body_format: "json"` to support the upload of json.
Fix the following error during the installation with ansible 2.1.1.0:
```
TASK [metron_elasticsearch_templates : Wait for Index to Become Available] *****
ok: [node1]
TASK [metron_elasticsearch_templates : Add Elasticsearch templates for topologies] ***
failed: [node1] (item=/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/yaf_index.template) => {"content": "", "failed": true, "item": "/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/yaf_index.template", "msg": "Status code was not [200]: An unknown error occurred: sendall() argument 1 must be string or buffer, not dict", "redirected": false, "status": -1, "url": "http://node1:9200/_template/yaf_index"}
failed: [node1] (item=/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/snort_index.template) => {"content": "", "failed": true, "item": "/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/snort_index.template", "msg": "Status code was not [200]: An unknown error occurred: sendall() argument 1 must be string or buffer, not dict", "redirected": false, "status": -1, "url": "http://node1:9200/_template/snort_index"}
failed: [node1] (item=/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/bro_index.template) => {"content": "", "failed": true, "item": "/home/yoyo/incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/files/es_templates/bro_index.template", "msg": "Status code was not [200]: An unknown error occurred: sendall() argument 1 must be string or buffer, not dict", "redirected": false, "status": -1, "url": "http://node1:9200/_template/bro_index"}
NO MORE HOSTS LEFT *************************************************************
to retry, use: --limit @playbooks/metron_full_install.retry
PLAY RECAP *********************************************************************
node1 : ok=7 changed=3 unreachable=0 failed=1
```
- Fix closed issue #153.
- Not tested with vagrant! (should work..)
- Ansible documentation of `body_format`: http://docs.ansible.com/ansible/uri_module.html and ansible/ansible-modules-core#1011.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/219
Thanks for the contribution- did you happen to test this on 2.0.0.2?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by 2xyo <gi...@git.apache.org>.
Github user 2xyo commented on the issue:
https://github.com/apache/incubator-metron/pull/219
Disclamer: I'm not a user of Mac/Homebrew...
According to https://github.com/Homebrew/brew > http://braumeister.org/formula/ansible the current version of ansible in homebrew is now 2.1.1.0
Moreover, the preferred way to install ansible on a Mac is [via pip](http://docs.ansible.com/ansible/intro_installation.html#latest-releases-on-mac-osx) :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/219
We version lock Ansible because we have seen some issues with regression. This JSON body issue was one of them. FWIW, I'm happy to support 2.1, but we'll need to version lock on 2.1 There are a few more changes we'd need to make to get there, these are off the top of my head and a quick search, so they may not be complete:
Rev the docker build container:
https://github.com/apache/incubator-metron/tree/master/metron-deployment/packaging/docker/ansible-docker
Rev the build READMEs:
https://github.com/apache/incubator-metron/blob/master/metron-deployment/vagrant/full-dev-platform/README.md
https://github.com/apache/incubator-metron/tree/master/metron-deployment/vagrant/quick-dev-platform
https://github.com/apache/incubator-metron/blob/master/metron-deployment/vagrant/codelab-platform/README.md
https://github.com/apache/incubator-metron/blob/master/metron-deployment/README.md
https://github.com/apache/incubator-metron/blob/master/metron-deployment/amazon-ec2/README.md
If you're willing to take that on, I'd be happy to spend some time running it up in the various environments (full-dev,quick-dev and ec2) to verify.
Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by 2xyo <gi...@git.apache.org>.
Github user 2xyo commented on the issue:
https://github.com/apache/incubator-metron/pull/219
@dlyle65535 : I downgrade to pip 2.0.0.2 and replay the playbook: it fails at "[Add Elasticsearch templates for topologies](https://gist.githubusercontent.com/2xyo/5e99fa83c8419c8298d03bda0ee0eec4/raw/b58f2654ebc758a3846b9751cf93445bebe30218/readme.md)".
Then, upgrade to pip 2.1.1.0 and [it works again](https://gist.githubusercontent.com/2xyo/1da437e176683b60e8bfb1db6ad94042/raw/46d62d8442dfa110e2a31ee1261364670e97f346/readme.md).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/incubator-metron/pull/219
@2xyo Did #499 take care of your concerns here (as @JonZeolla mentioned)? If so, please close this PR when you get a chance. Otherwise, please describe why this PR might still be needed. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by 2xyo <gi...@git.apache.org>.
Github user 2xyo commented on the issue:
https://github.com/apache/incubator-metron/pull/219
Yes, it breaks support of 2.0.0.2 in order to support the [latest stable version of Ansible](https://www.ansible.com/blog/ansible-2.1).
Which one do you prefer? :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/219
Yeah, this was another 2.1 issue we bumped into. Part of the reason we're not running 2.1 right now. :) It was a pile of work to get backward compatibility. At the time, homebrew was installing 2.0.1.0, so we had to look at that as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by ottobackwards <gi...@git.apache.org>.
Github user ottobackwards commented on the issue:
https://github.com/apache/incubator-metron/pull/219
I think this issues should be part of a scheduled release effort. It involves changes to the wiki and other places where we have set out the required versions for working with metron. As I think this is a breaking change for everyone that currently has a working setup, I think there is more to it than just landing this PR
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/219
So, if I apply this patch, it no longer works in 2.0.0.2, is that correct?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by JonZeolla <gi...@git.apache.org>.
Github user JonZeolla commented on the issue:
https://github.com/apache/incubator-metron/pull/219
This is obsolete as of #499.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by 2xyo <gi...@git.apache.org>.
Github user 2xyo commented on the issue:
https://github.com/apache/incubator-metron/pull/219
@dlyle65535 I'd be happy to help on it. However, as I play with Metron in my spare time, I haven't much time to work on it before at least 2 weeks.
Feel free to reference other issues related to ansible here :)
Note: here is another issue with ansible 2.2.0 (dev version):
```
ASK [ambari_config : Deploy cluster with Ambari; http://node1:8080] ***********
fatal: [node1]: FAILED! => {"changed": false, "failed": true, "msg": "value of wait_for_complete must be one of: y,yes,on,1,true,1,True,n,no,off,0,false,0,False, got: True"}
to retry, use: --limit @../../playbooks/metron_full_install.retry
```
Potential fix: use ` | bool` filter
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by JonZeolla <gi...@git.apache.org>.
Github user JonZeolla commented on the issue:
https://github.com/apache/incubator-metron/pull/219
I agree. There are definite benefits to doing an upgrade to Ansible (whether to 2.1 or 2.2). One of many would be the ability to use the [debug feature](https://docs.ansible.com/ansible/playbooks_debugger.html) for troubleshooting. I validated that there is pretty widespread support for both 2.1 and 2.2. Specifically, I quickly looked at:
* [brew](http://braumeister.org/formula/ansible)
* pip (via `pip install ansible==`)
* [apt-get](https://launchpad.net/~ansible/+archive/ubuntu/ansible) for Ubuntu (and probably others)
* yum (via `yum list ansible` with `epel` configured)
That makes me think it may be more worthwhile to jump to 2.2 instead of 2.1, but I'm mostly indifferent.
Is there a large emphasis on having backwards compatible playbooks? Personally I'm not so concerned about that, as long as we do a good job documenting and **announcing** the upgrade, but I'd be interested to see in what situations that's important.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #219: METRON-196 Fix support of ansible 2.1 in metron...
Posted by 2xyo <gi...@git.apache.org>.
Github user 2xyo commented on the issue:
https://github.com/apache/incubator-metron/pull/219
This PR addresses https://issues.apache.org/jira/browse/METRON-196
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---