You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by sardell <gi...@git.apache.org> on 2018/04/27 14:40:25 UTC

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

GitHub user sardell opened a pull request:

    https://github.com/apache/metron/pull/1004

    METRON-1489: Retrofit UI tests to run reliably during nightly QE runs

    ## Contributor Comments
    This Pull Request includes work started by @iraghumitra on his branch `e2e-using-await-1`. In addition to the asyn/await work he did, I added [protractor-flake](https://github.com/NickTomlin/protractor-flake) in order to re-run failing tests to ensure they are actual failures and not the result of flakey tests. 
    
    
    ## Pull Request Checklist
    
    Thank you for submitting a contribution to Apache Metron.  
    Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.  
    Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.  
    
    
    In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
    
    ### For all changes:
    - [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
    - [ ] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
    - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
    
    
    ### For code changes:
    - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
    - [ ] Have you included steps or a guide to how the change may be verified and tested manually?
    - [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
      ```
      mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh 
      ```
    
    - [ ] Have you written or updated unit tests and or integration tests to verify your changes?
    - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
    - [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
    
    ### For documentation related changes:
    - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
    
      ```
      cd site-book
      mvn site
      ```
    
    #### Note:
    Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
    It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sardell/metron METRON-1489

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/metron/pull/1004.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1004
    
----
commit e17336a7f89756d822f6c7c776d865c0214683da
Author: iraghumitra <ra...@...>
Date:   2018-04-26T09:07:57Z

    Initial commit for e2e tests in new branch

commit 15e379e6ca1b98fe9c5e59ff7d4d777afd072e63
Author: Shane Ardell <sa...@...>
Date:   2018-04-27T10:49:25Z

    METRON-1489: fix test to account for leap year days

commit be90636a5d5ecefdd4469986b1ca1a2878942fd4
Author: Shane Ardell <sa...@...>
Date:   2018-04-27T11:06:16Z

    METRON-1489: fix flakey test results with protractor-flake

----


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r184750091
  
    --- Diff: metron-interface/metron-alerts/e2e/alerts-list/alert-status/alerts-list-status.e2e-spec.ts ---
    @@ -44,84 +45,84 @@ describe('metron-alerts alert status', function() {
         jasmine.addMatchers(customMatchers);
       });
     
    -  it('should change alert status for multiple alerts to OPEN', () => {
    -    page.navigateTo();
    -    page.toggleAlertInList(0);
    -    page.toggleAlertInList(1);
    -    page.toggleAlertInList(2);
    -    page.clickActionDropdownOption('Open');
    -    expect(page.getAlertStatus(0, 'NEW')).toEqual('OPEN');
    -    expect(page.getAlertStatus(1, 'NEW')).toEqual('OPEN');
    -    expect(page.getAlertStatus(2, 'NEW')).toEqual('OPEN');
    +  it('should change alert status for multiple alerts to OPEN', async function() : Promise<any> {
    +    await page.navigateTo();
    +    await page.toggleAlertInList(0);
    +    await page.toggleAlertInList(1);
    +    await page.toggleAlertInList(2);
    +    await page.clickActionDropdownOption('Open');
    +    // expect(await page.getAlertStatus(0, 'NEW')).toEqual('OPEN');
    +    // expect(await page.getAlertStatus(1, 'NEW')).toEqual('OPEN');
    +    // expect(await page.getAlertStatus(2, 'NEW')).toEqual('OPEN');
       });
     
    -  it('should change alert status for multiple alerts to DISMISS', () => {
    -    page.toggleAlertInList(3);
    -    page.toggleAlertInList(4);
    -    page.toggleAlertInList(5);
    -    page.clickActionDropdownOption('Dismiss');
    -    expect(page.getAlertStatus(3, 'NEW')).toEqual('DISMISS');
    -    expect(page.getAlertStatus(4, 'NEW')).toEqual('DISMISS');
    -    expect(page.getAlertStatus(5, 'NEW')).toEqual('DISMISS');
    +  it('should change alert status for multiple alerts to DISMISS', async function() : Promise<any> {
    +    await page.toggleAlertInList(3);
    +    await page.toggleAlertInList(4);
    +    await page.toggleAlertInList(5);
    +    await page.clickActionDropdownOption('Dismiss');
    +    // expect(await page.getAlertStatus(3, 'NEW')).toEqual('DISMISS');
    +    // expect(await page.getAlertStatus(4, 'NEW')).toEqual('DISMISS');
    +    // expect(await page.getAlertStatus(5, 'NEW')).toEqual('DISMISS');
       });
     
    -  it('should change alert status for multiple alerts to ESCALATE', () => {
    -    page.toggleAlertInList(6);
    -    page.toggleAlertInList(7);
    -    page.toggleAlertInList(8);
    -    page.clickActionDropdownOption('Escalate');
    -    expect(page.getAlertStatus(6, 'NEW')).toEqual('ESCALATE');
    -    expect(page.getAlertStatus(7, 'NEW')).toEqual('ESCALATE');
    -    expect(page.getAlertStatus(8, 'NEW')).toEqual('ESCALATE');
    +  it('should change alert status for multiple alerts to ESCALATE', async function() : Promise<any> {
    +    await page.toggleAlertInList(6);
    +    await page.toggleAlertInList(7);
    +    await page.toggleAlertInList(8);
    +    await page.clickActionDropdownOption('Escalate');
    +    // expect(await page.getAlertStatus(6, 'NEW')).toEqual('ESCALATE');
    --- End diff --
    
    Do we need these commented out bits?


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by sardell <gi...@git.apache.org>.
Github user sardell closed the pull request at:

    https://github.com/apache/metron/pull/1004


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably during ni...

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    Maybe I am wrong, but I thought these tests used to run headless.  It looks to me that these are running "headful" now. :)  
    
    Either way, can these tests run headless or is there a major hurdle stopping us from doing that?


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by sardell <gi...@git.apache.org>.
Github user sardell closed the pull request at:

    https://github.com/apache/metron/pull/1004


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by sardell <gi...@git.apache.org>.
GitHub user sardell reopened a pull request:

    https://github.com/apache/metron/pull/1004

    METRON-1489: Retrofit UI tests to run reliably

    ## Contributor Comments
    This Pull Request includes work started by @iraghumitra on his branch `e2e-using-await-1`. In addition to the asyn/await work he did, I added [protractor-flake](https://github.com/NickTomlin/protractor-flake) in order to re-run failing tests to ensure they are actual failures and not the result of flakey tests.
    
    In order to test locally, be sure to install protractor-flake before running the e2e tests. This is done by running `npm install` inside of the metron-alerts directory.
    
    ## Pull Request Checklist
    
    Thank you for submitting a contribution to Apache Metron.  
    Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.  
    Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.  
    
    
    In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
    
    ### For all changes:
    - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
    - [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
    - [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
    
    
    ### For code changes:
    - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
    - [x] Have you included steps or a guide to how the change may be verified and tested manually?
    - [x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
      ```
      mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh 
      ```
    
    - [x] Have you written or updated unit tests and or integration tests to verify your changes?
    - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
    - [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
    
    ### For documentation related changes:
    - [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
    
      ```
      cd site-book
      mvn site
      ```
    
    #### Note:
    Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
    It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sardell/metron METRON-1489

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/metron/pull/1004.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1004
    
----
commit 08ba22a35b4b6a741e504c3c59ccd98889f4ec7e
Author: iraghumitra <ra...@...>
Date:   2018-04-26T09:07:57Z

    Initial commit for e2e tests in new branch

commit 1daefda4b8d39dc24d8cc44bc837efa7bd6734e0
Author: Shane Ardell <sa...@...>
Date:   2018-04-27T10:49:25Z

    METRON-1489: fix test to account for leap year days

commit 92542ad6607e8d84aa312c5c14175adb07573481
Author: Shane Ardell <sa...@...>
Date:   2018-04-27T11:06:16Z

    METRON-1489: fix flakey test results with protractor-flake

commit e7e71875cccb8781595a6ca687174483bca462a0
Author: Shane Ardell <sa...@...>
Date:   2018-05-03T09:16:53Z

    add protractor option to run headless

commit 54f417d06bfd1ed69d2c7a2e29f5cbc765cc480f
Author: Shane Ardell <sa...@...>
Date:   2018-05-03T11:00:55Z

    uncomment tests and assertions that now pass

commit 19876cb48d85920b22052938cd216a59ac55f833
Author: Shane Ardell <sa...@...>
Date:   2018-05-03T11:33:00Z

    remove outdated code

commit 877c05a8b889b105989b0f0eeca08f8935f9c9ef
Author: Shane Ardell <sa...@...>
Date:   2018-05-03T11:34:47Z

    add workaround for ssh2 auth failure

commit 0343ed1c8955062d511b3dcc4cf6a3414deef07b
Author: Shane Ardell <sa...@...>
Date:   2018-05-11T16:37:59Z

    cleanup commented out code

commit 47748bc1c7d4119610d0e519af86e2f4f340255a
Author: Shane Ardell <sa...@...>
Date:   2018-05-14T14:19:55Z

    fix e2e section formatting and typos in alerts documentation

commit a9650ed28b534ab6547e8d99356be81f87730c4c
Author: Shane Ardell <sa...@...>
Date:   2018-05-14T14:40:57Z

    uncomment timestamp test in custom date save test

commit b47737ebf99ea90bdb51bfc45fec9d3372bb4158
Author: Shane Ardell <sa...@...>
Date:   2018-05-14T15:46:39Z

    remove commented out lines from alerts-list test functions

commit 4fc0d99b63f8383d28cadecb7525731015a98304
Author: Shane Ardell <sa...@...>
Date:   2018-05-16T08:45:03Z

    remove unused code from protractor config

commit 5fb88e82e89accc24767b5e014e905b63db73fba
Author: Shane Ardell <sa...@...>
Date:   2018-06-05T12:08:46Z

    add async/await and cleanup alert-details e2e tests

commit d4227187165e2c89e894835843b08bafa345673a
Author: Shane Ardell <sa...@...>
Date:   2018-06-05T12:51:03Z

    uncomment all test files

commit d02d4d6293e6dd25cb585808f158cd670a5f17b9
Author: Shane Ardell <sa...@...>
Date:   2018-06-05T12:52:13Z

    cleanup clickOnRow function

commit 03631aa5149e5532efe082f87d4da9f5472c1003
Author: Shane Ardell <sa...@...>
Date:   2018-06-05T12:54:27Z

    disable only comment tests that rely on getText()

----


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r184753168
  
    --- Diff: metron-interface/metron-alerts/README.md ---
    @@ -126,7 +126,14 @@ The application will be available at http://host:4201 assuming the port is set t
     
     ## E2E Tests
     
    -An expressjs server is available for mocking the elastic search api.
    +#Caveats
    --- End diff --
    
    This markdown does not render properly.  I think you need a space and it probably needs to be 3 hashes as in "### Caveats".


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by simonellistonball <gi...@git.apache.org>.
Github user simonellistonball commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r184743321
  
    --- Diff: metron-interface/metron-alerts/e2e/alert-details/alert-status/alert-details-status.e2e-spec.ts ---
    @@ -101,7 +104,7 @@ describe('metron-alerts alert status', function() {
         page.closeDetailPane();
       });
     
    -  it('should add comments for tree view', () => {
    +  xit('should add comments for tree view', () => {
    --- End diff --
    
    Any particular reason to skip this test?


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by anandsubbu <gi...@git.apache.org>.
Github user anandsubbu commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r184889297
  
    --- Diff: metron-interface/metron-alerts/protractor.conf.js ---
    @@ -32,12 +33,13 @@ exports.config = {
         './e2e/alerts-list/tree-view/tree-view.e2e-spec.ts',
         './e2e/alerts-list/alert-filters/alert-filters.e2e-spec.ts',
         './e2e/alerts-list/alert-status/alerts-list-status.e2e-spec.ts',
    -    './e2e/alert-details/alert-status/alert-details-status.e2e-spec.ts',
    +    // './e2e/alert-details/alert-status/alert-details-status.e2e-spec.ts', //This will not work
         './e2e/alerts-list/meta-alerts/meta-alert.e2e-spec.ts'
       ],
       capabilities: {
         'browserName': 'chrome',
         'chromeOptions': {
    +      args: ["--disable-gpu", "--window-size=1435,850" ],
    --- End diff --
    
    `-headless` can be included here to default to headless execution.


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    @sardell Should the topologies be stopped in the development environment when running the tests?      Does that impact the tests at all?


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by sardell <gi...@git.apache.org>.
Github user sardell commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r187971664
  
    --- Diff: metron-interface/metron-alerts/README.md ---
    @@ -126,7 +126,14 @@ The application will be available at http://host:4201 assuming the port is set t
     
     ## E2E Tests
     
    -An expressjs server is available for mocking the elastic search api.
    +#Caveats
    --- End diff --
    
    Thanks for pointing this out, @nickwallen. Fixed with the latest commit.


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r187350501
  
    --- Diff: metron-interface/metron-alerts/README.md ---
    @@ -126,7 +126,14 @@ The application will be available at http://host:4201 assuming the port is set t
     
     ## E2E Tests
     
    -An expressjs server is available for mocking the elastic search api.
    +#Caveats
    --- End diff --
    
    This is still a problem @sardell 


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by sardell <gi...@git.apache.org>.
Github user sardell commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r185477602
  
    --- Diff: metron-interface/metron-alerts/e2e/alert-details/alert-status/alert-details-status.e2e-spec.ts ---
    @@ -101,7 +104,7 @@ describe('metron-alerts alert status', function() {
         page.closeDetailPane();
       });
     
    -  it('should add comments for tree view', () => {
    +  xit('should add comments for tree view', () => {
    --- End diff --
    
    @iraghumitra Same question as above.


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by simonellistonball <gi...@git.apache.org>.
Github user simonellistonball commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r184743288
  
    --- Diff: metron-interface/metron-alerts/e2e/alert-details/alert-status/alert-details-status.e2e-spec.ts ---
    @@ -23,64 +23,67 @@ import {loadTestData, deleteTestData} from '../../utils/e2e_util';
     import { MetronAlertsPage } from '../../alerts-list/alerts-list.po';
     import {TreeViewPage} from '../../alerts-list/tree-view/tree-view.po';
     
    -describe('metron-alerts alert status', function() {
    +describe('Test spec for metron details page', function() {
       let page: MetronAlertDetailsPage;
       let listPage: MetronAlertsPage;
       let treePage: TreeViewPage;
       let loginPage: LoginPage;
     
    -  beforeAll(() => {
    -    loadTestData();
    +  beforeAll(async function() : Promise<any> {
         loginPage = new LoginPage();
         listPage = new MetronAlertsPage();
         treePage = new TreeViewPage();
    -    loginPage.login();
    +
    +    await loadTestData();
    +    await loginPage.login();
       });
     
    -  afterAll(() => {
    +  afterAll(async function() : Promise<any> {
         new MetronAlertsPage().navigateTo();
    -    loginPage.logout();
    -    deleteTestData();
    +    await loginPage.logout();
    +    await deleteTestData();
       });
     
       beforeEach(() => {
         page = new MetronAlertDetailsPage();
         jasmine.addMatchers(customMatchers);
       });
     
    -  it('should change alert statuses', () => {
    -    let alertId = 'c4c5e418-3938-099e-bb0d-37028a98dca8';
    -
    -    page.navigateTo(alertId);
    -    page.clickNew();
    -    expect(page.getAlertStatus('ANY')).toEqual('NEW');
    -    page.clickOpen();
    -    expect(page.getAlertStatus('NEW')).toEqual('OPEN');
    -    expect(listPage.getAlertStatusById(alertId)).toEqual('OPEN');
    -    page.clickDismiss();
    -    expect(page.getAlertStatus('OPEN')).toEqual('DISMISS');
    -    expect(listPage.getAlertStatusById(alertId)).toEqual('DISMISS');
    -    page.clickEscalate();
    -    expect(page.getAlertStatus('DISMISS')).toEqual('ESCALATE');
    -    expect(listPage.getAlertStatusById(alertId)).toEqual('ESCALATE');
    -    page.clickResolve();
    -    expect(page.getAlertStatus('ESCALATE')).toEqual('RESOLVE');
    -    expect(listPage.getAlertStatusById(alertId)).toEqual('RESOLVE');
    -    page.clickNew();
    +  xit('should change alert statuses', async function() : Promise<any> {
    --- End diff --
    
    Any particular reason to skip this test?


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/metron/pull/1004


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by sardell <gi...@git.apache.org>.
Github user sardell commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r187354305
  
    --- Diff: metron-interface/metron-alerts/e2e/alerts-list/tree-view/tree-view.po.ts ---
    @@ -97,21 +155,24 @@ export class TreeViewPage {
       expandSubGroupByPosition(groupName: string, rowName: string, position: number) {
         let subGroupElement = element.all(by.css('[data-name="' + groupName + '"] tr[data-name="' + rowName + '"]')).get(position);
         return waitForElementVisibility(subGroupElement)
    -    .then(() => browser.actions().mouseMove(subGroupElement).perform())
    -    .then(() => subGroupElement.click());
    +    .then(() => scrollIntoView(subGroupElement.element(by.css('.fa-caret-right')), true))
    +    .then(() => subGroupElement.click())
    +    .then(() => waitForElementVisibility(subGroupElement.element(by.css('.fa-caret-down'))));
       }
     
    -  getDashGroupTableValuesForRow(name: string, rowId: number) {
    -    this.scrollToDashRow(name);
    -    return waitForElementPresence(element(by.css('[data-name="' + name + '"] table tbody tr'))).then(() => {
    -      return element.all(by.css('[data-name="' + name + '"] table tbody tr')).get(rowId).all(by.css('td')).getText();
    -    });
    -  }
    +  // getDashGroupTableValuesForRow(name: string, rowId: number) {
    --- End diff --
    
    @nickwallen No, this looks like something I need to address. Thanks for pointing this out. Previously, I only addressed the specific lines that were called out before in comments. I'll run through all currently commented tests in this PR, update and have @iraghumitra review since this stems from the work he did. I also see some console.log's in there that I'll remove.


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r187948446
  
    --- Diff: metron-interface/metron-alerts/e2e/alerts-list/meta-alerts/meta-alert.e2e-spec.ts ---
    @@ -19,139 +19,142 @@
     import { MetronAlertsPage } from '../alerts-list.po';
     import {customMatchers} from '../../matchers/custom-matchers';
     import {LoginPage} from '../../login/login.po';
    -import {loadTestData, deleteTestData} from '../../utils/e2e_util';
    +import {loadTestData, deleteTestData, createMetaAlertsIndex} from '../../utils/e2e_util';
     import {TreeViewPage} from '../tree-view/tree-view.po';
     import {MetronAlertDetailsPage} from '../../alert-details/alert-details.po';
     import {MetaAlertPage} from './meta-alert.po';
     import {AlertFacetsPage} from '../alert-filters/alert-filters.po';
     
    -describe('meta-alerts workflow', function() {
    +describe('Test spec for meta alerts workflow', function() {
       let detailsPage: MetronAlertDetailsPage;
       let tablePage: MetronAlertsPage;
       let metaAlertPage: MetaAlertPage;
       let treePage: TreeViewPage;
       let loginPage: LoginPage;
       let alertFacetsPage: AlertFacetsPage;
     
    -  beforeAll(() => {
    -    loadTestData();
    -
    +  beforeAll(async function() : Promise<any> {
         loginPage = new LoginPage();
    -    loginPage.login();
         tablePage = new MetronAlertsPage();
         treePage = new TreeViewPage();
         tablePage = new MetronAlertsPage();
         metaAlertPage = new MetaAlertPage();
         detailsPage = new MetronAlertDetailsPage();
         alertFacetsPage = new AlertFacetsPage();
    +
    +    await createMetaAlertsIndex();
    +    await loadTestData();
    +    await loginPage.login();
       });
     
    -  afterAll(() => {
    -    loginPage.logout();
    -    deleteTestData();
    +  afterAll(async function() : Promise<any> {
    +    await loginPage.logout();
    +    await deleteTestData();
       });
     
       beforeEach(() => {
         jasmine.addMatchers(customMatchers);
       });
     
    -  it('should have all the steps for meta alerts workflow', () => {
    +  it('should have all the steps for meta alerts workflow', async function() : Promise<any> {
         let comment1 = 'This is a sample comment';
         let userNameAndTimestamp = '- admin - a few seconds ago';
    -    let confirmText = 'Do you wish to create a meta alert with 113 selected alerts?';
    +    let confirmText = 'Do you wish to create a meta alert with 13 selected alerts?';
         let dashRowValues = {
    -      'firstDashRow': ['0', '192.168.138.158', 'ALERTS', '113'],
    -      'secondDashRow': ['0', '192.168.66.1', 'ALERTS', '56']
    +      'firstDashRow': ['0', 'runlove.us', 'ALERTS', '13']
         };
     
    -    tablePage.navigateTo();
    +    await tablePage.navigateTo();
    +    expect(await tablePage.getChangesAlertTableTitle('Alerts (0)')).toEqual('Alerts (169)');
     
         /* Create Meta Alert */
    -    treePage.selectGroup('ip_src_addr');
    -    expect(treePage.getDashGroupValues('192.168.138.158')).toEqualBcoz(dashRowValues.firstDashRow, 'First Dashrow to be present');
    -    expect(treePage.getDashGroupValues('192.168.66.1')).toEqualBcoz(dashRowValues.secondDashRow, 'Second Dashrow to be present');
    +    await treePage.selectGroup('host');
    +    expect(await treePage.getDashGroupValues('runlove.us')).toEqualBcoz(dashRowValues.firstDashRow, 'First Dashrow to be present');
     
    -    treePage.clickOnMergeAlerts('192.168.138.158');
    -    expect(treePage.getConfirmationText()).toEqualBcoz(confirmText, 'confirmation text to be present');
    -    treePage.clickNoForConfirmation();
    +    await treePage.clickOnMergeAlerts('runlove.us');
    +    expect(await treePage.getConfirmationText()).toEqualBcoz(confirmText, 'confirmation text to be present');
    +    await treePage.clickNoForConfirmation();
     
    -    treePage.clickOnMergeAlerts('192.168.138.158');
    -    treePage.clickYesForConfirmation();
    +    await treePage.clickOnMergeAlerts('runlove.us');
    +    await treePage.clickYesForConfirmation();
     
    -    treePage.waitForElementToDisappear('192.168.138.158');
    +    await treePage.waitForElementToDisappear('runlove.us');
     
    -    treePage.unGroup();
    +    await treePage.unGroup();
     
         /* Table should have all alerts */
    -    tablePage.waitForMetaAlert();
    -    expect(tablePage.getPaginationText()).toEqualBcoz('1 - 25 of 57', 'pagination text to be present');
    -    expect(tablePage.getCellValue(0, 2, '(114)')).toContain('(113)', 'number of alerts in a meta alert should be correct');
    -    expect(tablePage.getNonHiddenRowCount()).toEqualBcoz(25, '25 rows to be visible');
    -    expect(tablePage.getAllRowsCount()).toEqualBcoz(138, '138 rows to be available');
    -    expect(tablePage.getHiddenRowCount()).toEqualBcoz(113, '113 rows to be hidden');
    -    tablePage.expandMetaAlert(0);
    -    expect(tablePage.getNonHiddenRowCount()).toEqualBcoz(138, '138 rows to be visible after group expand');
    -    expect(tablePage.getAllRowsCount()).toEqualBcoz(138, '138 rows to be available after group expand');
    -    expect(tablePage.getHiddenRowCount()).toEqualBcoz(0, '0 rows to be hidden after group expand');
    -
    -    /* Meta Alert Status Change */
    -    tablePage.toggleAlertInList(0);
    -    tablePage.clickActionDropdownOption('Open');
    -    expect(tablePage.getAlertStatus(0, 'NEW', 2)).toEqual('OPEN');
    -    expect(tablePage.getAlertStatus(1, 'NEW')).toEqual('OPEN');
    -    expect(tablePage.getAlertStatus(2, 'NEW')).toEqual('OPEN');
    +    await tablePage.waitForMetaAlert(157);
    +    expect(await tablePage.getChangedPaginationText('1 - 25 of 169')).toEqualBcoz('1 - 25 of 157', 'pagination text to be present');
    +    expect(await tablePage.getCellValue(0, 2, '(14)')).toContain('(13)', 'number of alerts in a meta alert should be correct');
    +    expect(await tablePage.getNonHiddenRowCount()).toEqualBcoz(25, '25 rows to be visible');
    +    expect(await tablePage.getAllRowsCount()).toEqualBcoz(38, '38 rows to be available');
    +    expect(await tablePage.getHiddenRowCount()).toEqualBcoz(13, '13 rows to be hidden');
    +    await tablePage.expandMetaAlert(0);
    +    expect(await tablePage.getNonHiddenRowCount()).toEqualBcoz(38, '38 rows to be visible after group expand');
    +    expect(await tablePage.getAllRowsCount()).toEqualBcoz(38, '38 rows to be available after group expand');
    +    expect(await tablePage.getHiddenRowCount()).toEqualBcoz(0, '0 rows to be hidden after group expand');
    +
    +    /* Meta Alert Status Change // These do not work till patch works */
    +    // await tablePage.toggleAlertInList(0);
    --- End diff --
    
    What do these commented out sections mean?  What patch?  When will the patch work?



---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by sardell <gi...@git.apache.org>.
Github user sardell commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r185477633
  
    --- Diff: metron-interface/metron-alerts/e2e/alerts-list/alert-status/alerts-list-status.e2e-spec.ts ---
    @@ -44,84 +45,84 @@ describe('metron-alerts alert status', function() {
         jasmine.addMatchers(customMatchers);
       });
     
    -  it('should change alert status for multiple alerts to OPEN', () => {
    -    page.navigateTo();
    -    page.toggleAlertInList(0);
    -    page.toggleAlertInList(1);
    -    page.toggleAlertInList(2);
    -    page.clickActionDropdownOption('Open');
    -    expect(page.getAlertStatus(0, 'NEW')).toEqual('OPEN');
    -    expect(page.getAlertStatus(1, 'NEW')).toEqual('OPEN');
    -    expect(page.getAlertStatus(2, 'NEW')).toEqual('OPEN');
    +  it('should change alert status for multiple alerts to OPEN', async function() : Promise<any> {
    +    await page.navigateTo();
    +    await page.toggleAlertInList(0);
    +    await page.toggleAlertInList(1);
    +    await page.toggleAlertInList(2);
    +    await page.clickActionDropdownOption('Open');
    +    // expect(await page.getAlertStatus(0, 'NEW')).toEqual('OPEN');
    +    // expect(await page.getAlertStatus(1, 'NEW')).toEqual('OPEN');
    +    // expect(await page.getAlertStatus(2, 'NEW')).toEqual('OPEN');
       });
     
    -  it('should change alert status for multiple alerts to DISMISS', () => {
    -    page.toggleAlertInList(3);
    -    page.toggleAlertInList(4);
    -    page.toggleAlertInList(5);
    -    page.clickActionDropdownOption('Dismiss');
    -    expect(page.getAlertStatus(3, 'NEW')).toEqual('DISMISS');
    -    expect(page.getAlertStatus(4, 'NEW')).toEqual('DISMISS');
    -    expect(page.getAlertStatus(5, 'NEW')).toEqual('DISMISS');
    +  it('should change alert status for multiple alerts to DISMISS', async function() : Promise<any> {
    +    await page.toggleAlertInList(3);
    +    await page.toggleAlertInList(4);
    +    await page.toggleAlertInList(5);
    +    await page.clickActionDropdownOption('Dismiss');
    +    // expect(await page.getAlertStatus(3, 'NEW')).toEqual('DISMISS');
    +    // expect(await page.getAlertStatus(4, 'NEW')).toEqual('DISMISS');
    +    // expect(await page.getAlertStatus(5, 'NEW')).toEqual('DISMISS');
       });
     
    -  it('should change alert status for multiple alerts to ESCALATE', () => {
    -    page.toggleAlertInList(6);
    -    page.toggleAlertInList(7);
    -    page.toggleAlertInList(8);
    -    page.clickActionDropdownOption('Escalate');
    -    expect(page.getAlertStatus(6, 'NEW')).toEqual('ESCALATE');
    -    expect(page.getAlertStatus(7, 'NEW')).toEqual('ESCALATE');
    -    expect(page.getAlertStatus(8, 'NEW')).toEqual('ESCALATE');
    +  it('should change alert status for multiple alerts to ESCALATE', async function() : Promise<any> {
    +    await page.toggleAlertInList(6);
    +    await page.toggleAlertInList(7);
    +    await page.toggleAlertInList(8);
    +    await page.clickActionDropdownOption('Escalate');
    +    // expect(await page.getAlertStatus(6, 'NEW')).toEqual('ESCALATE');
    --- End diff --
    
    @iraghumitra what was the reason for commenting out this test?


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by sardell <gi...@git.apache.org>.
Github user sardell commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r185477581
  
    --- Diff: metron-interface/metron-alerts/e2e/alert-details/alert-status/alert-details-status.e2e-spec.ts ---
    @@ -23,64 +23,67 @@ import {loadTestData, deleteTestData} from '../../utils/e2e_util';
     import { MetronAlertsPage } from '../../alerts-list/alerts-list.po';
     import {TreeViewPage} from '../../alerts-list/tree-view/tree-view.po';
     
    -describe('metron-alerts alert status', function() {
    +describe('Test spec for metron details page', function() {
       let page: MetronAlertDetailsPage;
       let listPage: MetronAlertsPage;
       let treePage: TreeViewPage;
       let loginPage: LoginPage;
     
    -  beforeAll(() => {
    -    loadTestData();
    +  beforeAll(async function() : Promise<any> {
         loginPage = new LoginPage();
         listPage = new MetronAlertsPage();
         treePage = new TreeViewPage();
    -    loginPage.login();
    +
    +    await loadTestData();
    +    await loginPage.login();
       });
     
    -  afterAll(() => {
    +  afterAll(async function() : Promise<any> {
         new MetronAlertsPage().navigateTo();
    -    loginPage.logout();
    -    deleteTestData();
    +    await loginPage.logout();
    +    await deleteTestData();
       });
     
       beforeEach(() => {
         page = new MetronAlertDetailsPage();
         jasmine.addMatchers(customMatchers);
       });
     
    -  it('should change alert statuses', () => {
    -    let alertId = 'c4c5e418-3938-099e-bb0d-37028a98dca8';
    -
    -    page.navigateTo(alertId);
    -    page.clickNew();
    -    expect(page.getAlertStatus('ANY')).toEqual('NEW');
    -    page.clickOpen();
    -    expect(page.getAlertStatus('NEW')).toEqual('OPEN');
    -    expect(listPage.getAlertStatusById(alertId)).toEqual('OPEN');
    -    page.clickDismiss();
    -    expect(page.getAlertStatus('OPEN')).toEqual('DISMISS');
    -    expect(listPage.getAlertStatusById(alertId)).toEqual('DISMISS');
    -    page.clickEscalate();
    -    expect(page.getAlertStatus('DISMISS')).toEqual('ESCALATE');
    -    expect(listPage.getAlertStatusById(alertId)).toEqual('ESCALATE');
    -    page.clickResolve();
    -    expect(page.getAlertStatus('ESCALATE')).toEqual('RESOLVE');
    -    expect(listPage.getAlertStatusById(alertId)).toEqual('RESOLVE');
    -    page.clickNew();
    +  xit('should change alert statuses', async function() : Promise<any> {
    --- End diff --
    
    This change is from @iraghumitra's original work, so it's probably best he answers this question.


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably during ni...

Posted by mmiklavc <gi...@git.apache.org>.
Github user mmiklavc commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    @sardell This is great to see - we may get our e2e tests running reliably as part of our build yet! In your intro notes, you mention doing an npm install to get the protractor-flake package installed. I believe we have Maven commands that handle installing these packages during the e2e test build. I think it's fine to sort out how we'll run the e2e tests in context of the full Metron build in a follow-on PR, but for this PR I think we should get that manual step to work consistently with how we install the other packages.
    
    Small tip on Jira and PR titles - the community is likely not going to care or know about private QE environments (regardless of them being appreciative of those external efforts :) ). It's probably better to describe this Jira/PR as getting it stable to run as part of a daily Travis build, or something along those lines.


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by sardell <gi...@git.apache.org>.
GitHub user sardell reopened a pull request:

    https://github.com/apache/metron/pull/1004

    METRON-1489: Retrofit UI tests to run reliably

    ## Contributor Comments
    This Pull Request includes work started by @iraghumitra on his branch `e2e-using-await-1`. In addition to the asyn/await work he did, I added [protractor-flake](https://github.com/NickTomlin/protractor-flake) in order to re-run failing tests to ensure they are actual failures and not the result of flakey tests.
    
    In order to test locally, be sure to install protractor-flake before running the e2e tests. This is done by running `npm install` inside of the metron-alerts directory.
    
    ## Pull Request Checklist
    
    Thank you for submitting a contribution to Apache Metron.  
    Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.  
    Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.  
    
    
    In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
    
    ### For all changes:
    - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
    - [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
    - [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
    
    
    ### For code changes:
    - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
    - [x] Have you included steps or a guide to how the change may be verified and tested manually?
    - [x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
      ```
      mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh 
      ```
    
    - [x] Have you written or updated unit tests and or integration tests to verify your changes?
    - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
    - [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
    
    ### For documentation related changes:
    - [x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
    
      ```
      cd site-book
      mvn site
      ```
    
    #### Note:
    Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
    It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sardell/metron METRON-1489

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/metron/pull/1004.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1004
    
----
commit 4073976e0808f7b496826deb487699e1d6925e21
Author: iraghumitra <ra...@...>
Date:   2018-04-26T09:07:57Z

    Initial commit for e2e tests in new branch

commit a4378c39c568a2c4c63720d9babbecdafe8ef2e5
Author: Shane Ardell <sa...@...>
Date:   2018-04-27T10:49:25Z

    METRON-1489: fix test to account for leap year days

commit c9a04371046383a67ee41947f4f406556b3b3146
Author: Shane Ardell <sa...@...>
Date:   2018-04-27T11:06:16Z

    METRON-1489: fix flakey test results with protractor-flake

commit 6e2dfccadc423bdf2906d3614a2d49803894139f
Author: Shane Ardell <sa...@...>
Date:   2018-05-03T09:16:53Z

    add protractor option to run headless

commit 7c6e98e0d2cf2a338afabf5988d5974ddaebad5a
Author: Shane Ardell <sa...@...>
Date:   2018-05-03T11:00:55Z

    uncomment tests and assertions that now pass

commit b8ad518cf2092b220dcd05c9b6ee4b170e142782
Author: Shane Ardell <sa...@...>
Date:   2018-05-03T11:33:00Z

    remove outdated code

commit 4c8f83e6bdbf691bdd61efd9ec89db914059f105
Author: Shane Ardell <sa...@...>
Date:   2018-05-03T11:34:47Z

    add workaround for ssh2 auth failure

commit 67580a6d400cfcdf2b7583ba3a1695fcfb30e051
Author: Shane Ardell <sa...@...>
Date:   2018-05-11T16:37:59Z

    cleanup commented out code

commit fd1fb6e948b60ad4d64bc9ebea1bd2ccece995db
Author: Shane Ardell <sa...@...>
Date:   2018-05-14T14:19:55Z

    fix e2e section formatting and typos in alerts documentation

commit 9d95f6a8867fc4e7b536ab539d3287a4a426ee26
Author: Shane Ardell <sa...@...>
Date:   2018-05-14T14:40:57Z

    uncomment timestamp test in custom date save test

commit 8a37ee22a950dc288bfb822814d13e3be3ee1e3f
Author: Shane Ardell <sa...@...>
Date:   2018-05-14T15:46:39Z

    remove commented out lines from alerts-list test functions

commit 4fd175b9694f74b0e2bde263eb4fff7035147c0e
Author: Shane Ardell <sa...@...>
Date:   2018-05-16T08:45:03Z

    remove unused code from protractor config

commit 51d5875c9b24d9fb280aee9dd05e6d270193ba33
Author: Shane Ardell <sa...@...>
Date:   2018-06-05T12:08:46Z

    add async/await and cleanup alert-details e2e tests

commit 812c4c631f3d407ea578f73b59d50e7a4634fc0c
Author: Shane Ardell <sa...@...>
Date:   2018-06-05T12:51:03Z

    uncomment all test files

commit 452826c93b0c45f1b7b22e64b28c315b0962487b
Author: Shane Ardell <sa...@...>
Date:   2018-06-05T12:52:13Z

    cleanup clickOnRow function

commit d2599169a930ac4d62d7cccfe715735c19a8ee1e
Author: Shane Ardell <sa...@...>
Date:   2018-06-05T12:54:27Z

    disable only comment tests that rely on getText()

----


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by sardell <gi...@git.apache.org>.
Github user sardell commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    @nickwallen I wasn't able to reproduce the failure you experienced, but I added a note to the README stating the parsers will sometimes cause failures for users.


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    +1 Thanks for all the hard work on this one.  I know this was a battle to get it over the finish line.
    
    I will get this merged once the CI build completes.


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    The first time I ran against a development environment where the topologies were running and so new data (outside the e2e tests) was hitting the search indices.  I ran the e2e tests and had quite a few failures.
    
    Then I stopped the topologies and since then either all tests pass on the first try or a couple fail and the automated retry kicks in which leads to eventual success.
    
    I am just not sure if that is coincidence or if we need to update the README so that everyone knows to stop the topologies when running these tests.


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    @sardell What is the status of this?  Are you happy with what you have?
    
    I was able to run the e2e tests successfully without any interventions this time.  It took a couple of automated retries, but it did finish successfully.


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    Please fix the formatting issue with the README.  I have mentioned in another thread, but you might have missed it.


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably during ni...

Posted by anandsubbu <gi...@git.apache.org>.
Github user anandsubbu commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    @nickwallen , in order to get past the following error you are seeing..
    ```
    <snip>
    [13:44:09] E/launcher - All configured authentication methods failed
    [13:44:09] E/launcher - Error: All configured authentication methods failed
        at tryNextAuth (/Users/nallen/tmp/metron-pr1004/metron-interface/metron-alerts/node_modules/ssh2/lib/client.js:380:17)
        at SSH2Stream.onUSERAUTH_FAILURE (/Users/nallen/tmp/metron-pr1004/metron-interface/metron-alerts/node_modules/ssh2/lib/client.js:591:5)
        at SSH2Stream.emit (events.js:160:13)
        at parsePacket (/Users/nallen/tmp/metron-pr1004/metron-interface/metron-alerts/node_modules/ssh2-streams/lib/ssh.js:3929:10)
        at SSH2Stream._transform (/Users/nallen/tmp/metron-pr1004/metron-interface/metron-alerts/node_modules/ssh2-streams/lib/ssh.js:669:13)
        at SSH2Stream.Transform._read (_stream_transform.js:185:10)
        at SSH2Stream._read (/Users/nallen/tmp/metron-pr1004/metron-interface/metron-alerts/node_modules/ssh2-streams/lib/ssh.js:251:15)
        at SSH2Stream.Transform._write (_stream_transform.js:173:12)
        at doWrite (_stream_writable.js:407:12)
        at writeOrBuffer (_stream_writable.js:393:5)
    [13:44:09] E/launcher - Process exited with error code 199
    
    ```
    The error you are seeing is because of the ssh2 module which authenticates to the vagrant node to do cleanup tasks. This needs to be fixed.
    
    As a workaround, you can do the following in (metron-interface/metron-alerts/e2e/utils/clean_metron_update_table.ts) i.e. https://github.com/apache/metron/pull/1004/files#diff-2cc98b255ddf3a954ad22dfa2b5545fb
    
    ```
    +        resolve();
    +        //cleanupTable(resolve, reject);
    ```
    i.e. you need to uncomment the `resolve()` function and comment out ` cleanupTable()`. 


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r184750333
  
    --- Diff: metron-interface/metron-alerts/e2e/alerts-list/alerts-list.po.ts ---
    @@ -272,63 +308,149 @@ export class MetronAlertsPage {
       toggleAlertInList(index: number) {
         let selector = by.css('app-alerts-list tbody tr label');
         let checkbox = element.all(selector).get(index);
    -    this.waitForElementPresence(checkbox).then(() => {
    -      browser.actions().mouseMove(checkbox).perform().then(() => {
    -        checkbox.click();
    -      });
    -    });
    +    return this.waitForElementPresence(checkbox)
    +    .then(() => scrollIntoView(checkbox, true))
    +    .then(() => checkbox.click());
       }
     
       getAlertStatus(rowIndex: number, previousText: string, colIndex = 8) {
         let row = element.all(by.css('app-alerts-list tbody tr')).get(rowIndex);
         let column = row.all(by.css('td a')).get(colIndex);
    -    return this.waitForTextChange(column, previousText).then(() => {
    -      return column.getText();
    -    });
    +    return this.waitForTextChange(column, previousText).then(() => column.getText());
       }
     
    -  waitForMetaAlert() {
    -    browser.sleep(2000);
    -    return element(by.css('button[data-name="search"]')).click()
    -    .then(() => waitForElementPresence(element(by.css('.icon-cell.dropdown-cell'))));
    +  waitForMetaAlert(expectedCount) {
    +    let title = element(by.css('.col-form-label-lg'));
    +    function waitForMetaAlert$()
    +    {
    +      return function () {
    +        return browser.sleep(2000)
    +        .then(() => element(by.css('button[data-name="search"]')).click())
    +        .then(() => waitForTextChange(title, `Alerts (169)`))
    +        .then(() => title.getText())
    +        .then((text) => text === `Alerts (${expectedCount})`)
    +        .catch(catchNoSuchElementError())
    +      }
    +    }
    +    return browser.wait(waitForMetaAlert$()).catch(catchNoSuchElementError());
       }
     
       isDateSeettingDisabled() {
         return element.all(by.css('app-time-range button.btn.btn-search[disabled=""]')).count().then((count) => { return (count === 1); });
       }
     
       clickDateSettings() {
    -    element(by.css('app-time-range button.btn-search')).click();
    -    browser.sleep(2000);
    +    return scrollIntoView(element(by.css('app-time-range button.btn-search')), true)
    +    .then(() => element(by.css('app-time-range button.btn-search')).click())
    +    .then(() => waitForCssClass(element(by.css('app-time-range #time-range')), 'show'));
    +  }
    +
    +  hideDateSettings() {
    +    return element(by.css('app-time-range button.btn-search')).click()
    +    .then(() => waitForCssClassNotToBePresent(element(by.css('app-time-range #time-range')), 'show'))
    +    .then(() => waitForElementInVisibility(element(by.css('app-time-range #time-range'))));
       }
     
       getTimeRangeTitles() {
    -    return element.all(by.css('app-time-range .title')).getText();
    +    return element.all(by.css('app-time-range .title')).reduce(reduce_for_get_all(), []);
       }
     
       getQuickTimeRanges() {
    -    return element.all(by.css('app-time-range .quick-ranges span')).getText();
    +    return element.all(by.css('app-time-range .quick-ranges span')).reduce(reduce_for_get_all(), []);
       }
     
       getValueForManualTimeRange() {
    -    return element.all(by.css('app-time-range input.form-control')). getAttribute('value');
    +    return element.all(by.css('app-time-range input.form-control')).reduce((acc, ele) => {
    +      return ele.getAttribute('value').then(value => {
    +        acc.push(value);
    +        return acc;
    +      });
    +    }, []);
       }
     
       isManulaTimeRangeApplyButtonPresent() {
         return element.all(by.css('app-time-range')).all(by.buttonText('APPLY')).count().then(count => count === 1);
       }
     
    +  waitForTextAndSubTextInTimeRange(currentTimeRangeVal) {
    +    return waitForTextChange(element(by.css('app-time-range .time-range-value')), currentTimeRangeVal[1])
    +    .then(() => waitForTextChange(element(by.css('app-time-range .time-range-text')), currentTimeRangeVal[0]))
    +  }
    +
    +  async selectQuickTimeRangeAndGetTimeRangeAndTimeText(quickRange: string) {
    +    let currentTimeRangeVal: any = [];
    +    // return element.all(by.css('app-time-range button span')).getText()
    --- End diff --
    
    Same here...


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by anandsubbu <gi...@git.apache.org>.
Github user anandsubbu commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r184888777
  
    --- Diff: metron-interface/metron-alerts/e2e/utils/clean_metron_update_table.ts ---
    @@ -0,0 +1,52 @@
    +declare var Promise: any;
    +var chalk = require('chalk');
    +var Client = require('ssh2').Client;
    +var errorMsg = '';
    +
    +export function cleanMetronUpdateTable() {
    +  return  new Promise(
    +      function (resolve, reject) {
    +        // resolve();
    +        cleanupTable(resolve, reject);
    --- End diff --
    
    IMHO, it would be better to default to `resolve()` only owing to the issues with Vagrant `ssh2` client. 


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by sardell <gi...@git.apache.org>.
Github user sardell commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    Re-running Travis.


---

[GitHub] metron issue #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by sardell <gi...@git.apache.org>.
Github user sardell commented on the issue:

    https://github.com/apache/metron/pull/1004
  
    @nickwallen This work is ready on my end. I'm happy with where I'm at since addressing the feedback given by others here. The automated retries are to be expected and means protractor-flake is doing its job.


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably du...

Posted by sardell <gi...@git.apache.org>.
Github user sardell commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r185477658
  
    --- Diff: metron-interface/metron-alerts/e2e/alerts-list/alerts-list.po.ts ---
    @@ -272,63 +308,149 @@ export class MetronAlertsPage {
       toggleAlertInList(index: number) {
         let selector = by.css('app-alerts-list tbody tr label');
         let checkbox = element.all(selector).get(index);
    -    this.waitForElementPresence(checkbox).then(() => {
    -      browser.actions().mouseMove(checkbox).perform().then(() => {
    -        checkbox.click();
    -      });
    -    });
    +    return this.waitForElementPresence(checkbox)
    +    .then(() => scrollIntoView(checkbox, true))
    +    .then(() => checkbox.click());
       }
     
       getAlertStatus(rowIndex: number, previousText: string, colIndex = 8) {
         let row = element.all(by.css('app-alerts-list tbody tr')).get(rowIndex);
         let column = row.all(by.css('td a')).get(colIndex);
    -    return this.waitForTextChange(column, previousText).then(() => {
    -      return column.getText();
    -    });
    +    return this.waitForTextChange(column, previousText).then(() => column.getText());
       }
     
    -  waitForMetaAlert() {
    -    browser.sleep(2000);
    -    return element(by.css('button[data-name="search"]')).click()
    -    .then(() => waitForElementPresence(element(by.css('.icon-cell.dropdown-cell'))));
    +  waitForMetaAlert(expectedCount) {
    +    let title = element(by.css('.col-form-label-lg'));
    +    function waitForMetaAlert$()
    +    {
    +      return function () {
    +        return browser.sleep(2000)
    +        .then(() => element(by.css('button[data-name="search"]')).click())
    +        .then(() => waitForTextChange(title, `Alerts (169)`))
    +        .then(() => title.getText())
    +        .then((text) => text === `Alerts (${expectedCount})`)
    +        .catch(catchNoSuchElementError())
    +      }
    +    }
    +    return browser.wait(waitForMetaAlert$()).catch(catchNoSuchElementError());
       }
     
       isDateSeettingDisabled() {
         return element.all(by.css('app-time-range button.btn.btn-search[disabled=""]')).count().then((count) => { return (count === 1); });
       }
     
       clickDateSettings() {
    -    element(by.css('app-time-range button.btn-search')).click();
    -    browser.sleep(2000);
    +    return scrollIntoView(element(by.css('app-time-range button.btn-search')), true)
    +    .then(() => element(by.css('app-time-range button.btn-search')).click())
    +    .then(() => waitForCssClass(element(by.css('app-time-range #time-range')), 'show'));
    +  }
    +
    +  hideDateSettings() {
    +    return element(by.css('app-time-range button.btn-search')).click()
    +    .then(() => waitForCssClassNotToBePresent(element(by.css('app-time-range #time-range')), 'show'))
    +    .then(() => waitForElementInVisibility(element(by.css('app-time-range #time-range'))));
       }
     
       getTimeRangeTitles() {
    -    return element.all(by.css('app-time-range .title')).getText();
    +    return element.all(by.css('app-time-range .title')).reduce(reduce_for_get_all(), []);
       }
     
       getQuickTimeRanges() {
    -    return element.all(by.css('app-time-range .quick-ranges span')).getText();
    +    return element.all(by.css('app-time-range .quick-ranges span')).reduce(reduce_for_get_all(), []);
       }
     
       getValueForManualTimeRange() {
    -    return element.all(by.css('app-time-range input.form-control')). getAttribute('value');
    +    return element.all(by.css('app-time-range input.form-control')).reduce((acc, ele) => {
    +      return ele.getAttribute('value').then(value => {
    +        acc.push(value);
    +        return acc;
    +      });
    +    }, []);
       }
     
       isManulaTimeRangeApplyButtonPresent() {
         return element.all(by.css('app-time-range')).all(by.buttonText('APPLY')).count().then(count => count === 1);
       }
     
    +  waitForTextAndSubTextInTimeRange(currentTimeRangeVal) {
    +    return waitForTextChange(element(by.css('app-time-range .time-range-value')), currentTimeRangeVal[1])
    +    .then(() => waitForTextChange(element(by.css('app-time-range .time-range-text')), currentTimeRangeVal[0]))
    +  }
    +
    +  async selectQuickTimeRangeAndGetTimeRangeAndTimeText(quickRange: string) {
    +    let currentTimeRangeVal: any = [];
    +    // return element.all(by.css('app-time-range button span')).getText()
    --- End diff --
    
    @iraghumitra what was the reason for commenting out this test?


---

[GitHub] metron pull request #1004: METRON-1489: Retrofit UI tests to run reliably

Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1004#discussion_r187350891
  
    --- Diff: metron-interface/metron-alerts/e2e/alerts-list/tree-view/tree-view.po.ts ---
    @@ -97,21 +155,24 @@ export class TreeViewPage {
       expandSubGroupByPosition(groupName: string, rowName: string, position: number) {
         let subGroupElement = element.all(by.css('[data-name="' + groupName + '"] tr[data-name="' + rowName + '"]')).get(position);
         return waitForElementVisibility(subGroupElement)
    -    .then(() => browser.actions().mouseMove(subGroupElement).perform())
    -    .then(() => subGroupElement.click());
    +    .then(() => scrollIntoView(subGroupElement.element(by.css('.fa-caret-right')), true))
    +    .then(() => subGroupElement.click())
    +    .then(() => waitForElementVisibility(subGroupElement.element(by.css('.fa-caret-down'))));
       }
     
    -  getDashGroupTableValuesForRow(name: string, rowId: number) {
    -    this.scrollToDashRow(name);
    -    return waitForElementPresence(element(by.css('[data-name="' + name + '"] table tbody tr'))).then(() => {
    -      return element.all(by.css('[data-name="' + name + '"] table tbody tr')).get(rowId).all(by.css('td')).getText();
    -    });
    -  }
    +  // getDashGroupTableValuesForRow(name: string, rowId: number) {
    --- End diff --
    
    There are still blocks of code commented out that need cleaned up.  Maybe you didn't push some additional commits out?


---