[jira] [Updated] (FLINK-12513) Improve end-to-end (bash based) tests

["Alex (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alex updated FLINK-12513:
-------------------------
    Description: 
Tests in {{flink-end-to-end-tests/test-scripts}} directory are re-using the same Flink directory to configure and launch a Flink cluster for a test.

As part of their setup steps, they may modify Flink config files (application config, logs config), the directory itself (e.g. by copying some jars into {{lib}} folder). Also, some tests involve using additional services (like spinning up Zookeper, Kafka and so on clusters).

The corresponding clean up code (to stop services, to revert Flink directory to original state) is spread out and a little bit not well-structured. In particular
 * the test runner itself reverts the Flink config (but doesn't revert other changes in Flink dir); 
 * some tests use shell's {{trap}} on exit hook for clean up callback. Adding multiple of such callbacks in one test would result in non-proper test tear down.

As the result, some tests may have left overs that may affect execution of next steps.

The proposal is to introduce a helper method for using one global (per test run) {{trap}} hook that would enable adding multiple clean up callbacks. This should enable registering "resource" clean up callbacks in the same place where resource is used/launched.

Optional improvement: -make the test runner create a temporal copy of Flink directory and launch test using that temporal directory. After the test is done, the temporal directory would be removed.-

Update: the test runner now rollbacks some Flink distribution folders (conf, lib and plugins in other PR). The above idea with temporal folder currently doesn't work for tests that involve dockerized Flink, because the {{build.sh}} script uses the relative path to the Flink dir under test.

  was:
Tests in {{flink-end-to-end-tests/test-scripts}} directory are re-using the same Flink directory to configure and launch a Flink cluster for a test.

As part of their setup steps, they may modify Flink config files (application config, logs config), the directory itself (e.g. by copying some jars into {{lib}} folder). Also, some tests involve using additional services (like spinning up Zookeper, Kafka and so on clusters).

The corresponding clean up code (to stop services, to revert Flink directory to original state) is spread out and a little bit not well-structured. In particular
 * the test runner itself reverts the Flink config (but doesn't revert other changes in Flink dir); 
 * some tests use shell's {{trap}} on exit hook for clean up callback. Adding multiple of such callbacks in one test would result in non-proper test tear down.

As the result, some tests may have left overs that may affect execution of next steps.

The proposal is to introduce a helper method for using one global (per test run) {{trap}} hook that would enable adding multiple clean up callbacks. This should enable registering "resource" clean up callbacks in the same place where resource is used/launched.

Optional improvement: make the test runner create a temporal copy of Flink directory and launch test using that temporal directory. After the test is done, the temporal directory would be removed.


> Improve end-to-end (bash based) tests
> -------------------------------------
>
>                 Key: FLINK-12513
>                 URL: https://issues.apache.org/jira/browse/FLINK-12513
>             Project: Flink
>          Issue Type: Improvement
>          Components: Tests
>            Reporter: Alex
>            Assignee: Alex
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 1.9.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Tests in {{flink-end-to-end-tests/test-scripts}} directory are re-using the same Flink directory to configure and launch a Flink cluster for a test.
> As part of their setup steps, they may modify Flink config files (application config, logs config), the directory itself (e.g. by copying some jars into {{lib}} folder). Also, some tests involve using additional services (like spinning up Zookeper, Kafka and so on clusters).
> The corresponding clean up code (to stop services, to revert Flink directory to original state) is spread out and a little bit not well-structured. In particular
>  * the test runner itself reverts the Flink config (but doesn't revert other changes in Flink dir); 
>  * some tests use shell's {{trap}} on exit hook for clean up callback. Adding multiple of such callbacks in one test would result in non-proper test tear down.
> As the result, some tests may have left overs that may affect execution of next steps.
> The proposal is to introduce a helper method for using one global (per test run) {{trap}} hook that would enable adding multiple clean up callbacks. This should enable registering "resource" clean up callbacks in the same place where resource is used/launched.
> Optional improvement: -make the test runner create a temporal copy of Flink directory and launch test using that temporal directory. After the test is done, the temporal directory would be removed.-
> Update: the test runner now rollbacks some Flink distribution folders (conf, lib and plugins in other PR). The above idea with temporal folder currently doesn't work for tests that involve dockerized Flink, because the {{build.sh}} script uses the relative path to the Flink dir under test.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)