You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by dlyle65535 <gi...@git.apache.org> on 2017/05/03 16:24:23 UTC
[GitHub] incubator-metron pull request #563: METRON-840: All "ambari_*" hosts need to...
GitHub user dlyle65535 opened a pull request:
https://github.com/apache/incubator-metron/pull/563
METRON-840: All "ambari_*" hosts need to have a /localrepo folder
## Contributor Comments
Fixes broken EC2 deployment. With the default config, each host running needs a yum repo created in /localrepo. It can be empty on all but the Metron host.
To test, run an EC2 deployment.
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.
Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [X] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target branch (typically master)?
### For code changes:
- [X] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
- [X] Have you included steps or a guide to how the change may be verified and tested manually?
- [X] Have you ensured that the full suite of tests and checks have been executed in the root incubating-metron folder via:
```
mvn -q clean integration-test install && build_utils/verify_licenses.sh
```
- [N/A] Have you written or updated unit tests and or integration tests to verify your changes?
- [N/A] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [X] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [N/A] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
```
cd site-book
mvn site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dlyle65535/incubator-metron METRON-840
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/563.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #563
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #563: METRON-840: All "ambari_*" hosts need to have a...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/incubator-metron/pull/563
When spinning this up on EC2, I ran into this problem. Ever seen this before? Maybe unrelated to your PR.
```
TASK [ambari_config : Deploy cluster with Ambari; http://ec2-54-236-37-116.compute-1.amazonaws.com:8080] ***
fatal: [ec2-54-236-37-116.compute-1.amazonaws.com]: FAILED! => {"changed": false, "failed": true,
"msg": "Ambari client exception occurred: Could not create blueprint: request code 400,
request message {\n \"status\" : 400,\n \"message\" : \"Blueprint configuration validation failed:
Missing required properties. Specify a value for these properties in the blueprint configuration.
{metron={metron-env=[metron_jdbc_platform, metron_jdbc_driver, metron_jdbc_username, metron_jdbc_url]}}\"\n}"}
```
Then when re-running, I get a slightly different error.
```
TASK [ambari_master : Setup ambari server] *************************************
fatal: [ec2-54-236-37-116.compute-1.amazonaws.com]: FAILED! => {"changed": false, "cmd":
"ambari-server setup -s && touch /etc/ambari-server/configured", "failed": true,
"failed_when_result": true, "rc": 0, "stderr": false,
"stdout": "skipped, since /etc/ambari-server/configured exists",
"stdout_lines": ["skipped, since /etc/ambari-server/configured exists"]}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #563: METRON-840: All "ambari_*" hosts need to have a...
Posted by dlyle65535 <gi...@git.apache.org>.
Github user dlyle65535 commented on the issue:
https://github.com/apache/incubator-metron/pull/563
small_cluster.yml is used by the ec2 install.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #563: METRON-840: All "ambari_*" hosts need to have a...
Posted by merrimanr <gi...@git.apache.org>.
Github user merrimanr commented on the issue:
https://github.com/apache/incubator-metron/pull/563
The fix should be quick. Does the ec2 playbook use an Ambari blueprint? Which one? The REST jdbc variables need defaults and these are set in the single_node_vm.yml blueprint for full dev:
```
required_configurations:
- metron-env:
storm_rest_addr: "{{ groups.ambari_slave[0] }}:8744"
es_hosts: "{{ groups.search | join(',') }}"
zeppelin_server_url: "{{ groups.zeppelin[0] }}:9995"
metron_jdbc_driver: "org.h2.Driver"
metron_jdbc_url: "jdbc:h2:file:~/metrondb"
metron_jdbc_username: "root"
metron_jdbc_password: "root"
metron_jdbc_platform: "h2"
```
Looks like these variables are not set in small_cluster.yml (are we even using that?). If ec2 was using that than just copying those settings over from single_node_vm.yml would fix it.
This installation step is covered in the REST README and these are required fields with descriptions when installing Metron with the MPack (so it should be obvious) but I can see how it's not obvious if you're automating an install. Any suggestions on how to make it better?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #563: METRON-840: All "ambari_*" hosts need to have a...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/incubator-metron/pull/563
I created METRON-935 to track the issue. Is this a quick fix that we can turn around or should we just revert the original commit?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #563: METRON-840: All "ambari_*" hosts need to...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-metron/pull/563
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #563: METRON-840: All "ambari_*" hosts need to have a...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/incubator-metron/pull/563
I deployed this to EC2 along with the code for #568 and #564. The deployment was successful and telemetry was flowing through end-to-end. Yay!
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #563: METRON-840: All "ambari_*" hosts need to have a...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/incubator-metron/pull/563
@nickwallen @merrimanr Looks like the REST Ambari stuff. I probably should have caught it in review, but it looks like our blueprint needs updating to reflect it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---