You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/02/21 11:18:44 UTC
[GitHub] [flink] zentol opened a new pull request #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
zentol opened a new pull request #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173
Based on #11143.
With this we ship 2 versions of Zookeeper (3.4.10, 3.5.6, as defined by flink-shaded) in the distribution.
ZK 3.4 is in /lib, while ZK 3.5 is in /opt.
Users can control which ZK version is used by switching the jars.
The added `zk-3.5` profile is used for CI purposes. It controls which ZK version is used during tests and bundled in lib/opt respectively. It also bumps `curator.version`, which by now is only used for curator-test, to version 4. Version 2 is still required for testing against ZK 3.4 .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-592467064
ping @StephanEwen
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
StephanEwen commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-592587987
> Simply put we propose to the drop the ZK 3.5 profile, only run unit/it tests against ZK 3.4, and possible adjust an E2E test to run against 3.5, with the added benefit of actually testing how users would switch between versions.
I think that proposal sounds good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
StephanEwen commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589661376
> @StephanEwen you still want to use zk 3.5 during IT cases / tests, and having the logic in the build system is more maintainable than custom code in the e2e scripts.
Is the plan to execute several tests with both ZK versions? Could we then have the profile in the project that has this dependency, rather than in the root pom?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
Hash:80755e8c69c712ebc1a2216f4b38bf23939deeea Status:UNKNOWN URL:TBD TriggerType:PUSH TriggerID:80755e8c69c712ebc1a2216f4b38bf23939deeea
-->
## CI report:
* 80755e8c69c712ebc1a2216f4b38bf23939deeea UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
Hash:80755e8c69c712ebc1a2216f4b38bf23939deeea Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/150002330 TriggerType:PUSH TriggerID:80755e8c69c712ebc1a2216f4b38bf23939deeea
Hash:e4bef951ad8869425ce4afcfa8bfb19980de030d Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/150026334 TriggerType:PUSH TriggerID:e4bef951ad8869425ce4afcfa8bfb19980de030d
-->
## CI report:
* 80755e8c69c712ebc1a2216f4b38bf23939deeea Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150002330)
* e4bef951ad8869425ce4afcfa8bfb19980de030d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150026334)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-591402360
One example for `curator-test` being used outside of `flink-runtime` is `flink-tests`, which runs some Zookeeper IT cases. For these the curator-test version has to correspond to the version in flink-runtime. So this isn't just a problem in the build system but also how we write tests.
AFAIK some other modules also use curator-test but the flink-runtime ZK may not necessarily be involved here.
Ultimately I don't know what most of these modules do with Zookeeper, but I understood the JIRA to add ZK 3.5 support for _all modules_, not just flink-runtime.
I had an offline chat with Till about this PR.
He raised a good argument in that the "global ZK version" approach is flawed as any module bundling ZK (apart from ZK) is hard-wired at compile-time to that version anyway, with no way for users to switch versions aside from recompiling Flink.
Additionally, it shouldn't be required to actually test against both ZK 3.4 and 3.5 . So long as we use 3.4 as the default curator should ensure that we can talk to both versions.
Simply put we propose to the drop the ZK 3.5 profile, only run unit/it tests against ZK 3.4, and possible adjust an E2E test to run against 3.5, with the added benefit of actually testing how users would switch between versions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] TisonKun commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
TisonKun commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589671893
@zentol Thanks for your explanation. I get it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
Hash:80755e8c69c712ebc1a2216f4b38bf23939deeea Status:PENDING URL:https://travis-ci.com/flink-ci/flink/builds/150002330 TriggerType:PUSH TriggerID:80755e8c69c712ebc1a2216f4b38bf23939deeea
-->
## CI report:
* 80755e8c69c712ebc1a2216f4b38bf23939deeea Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/150002330)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150002330",
"triggerID" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"triggerType" : "PUSH"
}, {
"hash" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150026334",
"triggerID" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"triggerType" : "PUSH"
}, {
"hash" : "5886b5fb58462c1cb7f2da19f684c3075e7c09c1",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151058259",
"triggerID" : "5886b5fb58462c1cb7f2da19f684c3075e7c09c1",
"triggerType" : "PUSH"
}, {
"hash" : "e5737cada14cd00a6642c07998590231c38172e5",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151167390",
"triggerID" : "e5737cada14cd00a6642c07998590231c38172e5",
"triggerType" : "PUSH"
}, {
"hash" : "e5737cada14cd00a6642c07998590231c38172e5",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5776",
"triggerID" : "e5737cada14cd00a6642c07998590231c38172e5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5886b5fb58462c1cb7f2da19f684c3075e7c09c1 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/151058259)
* e5737cada14cd00a6642c07998590231c38172e5 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/151167390) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5776)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150002330",
"triggerID" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"triggerType" : "PUSH"
}, {
"hash" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150026334",
"triggerID" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"triggerType" : "PUSH"
}, {
"hash" : "5886b5fb58462c1cb7f2da19f684c3075e7c09c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5886b5fb58462c1cb7f2da19f684c3075e7c09c1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e4bef951ad8869425ce4afcfa8bfb19980de030d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150026334)
* 5886b5fb58462c1cb7f2da19f684c3075e7c09c1 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] TisonKun commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
TisonKun commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589643407
LGTM.
A possible follow-up is separating `curator.version` & `curator.test.version`. zk 3.4 works well with curator 4.x except when testing it requires curator-test 2.x, which is another version than curator-framework and so on.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
StephanEwen commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-590084670
I see. I guess under the current setup, that approach is fair enough.
It would be nice to keep working on a bit better encapsulation, because this contributes to the increasing "unwieldyness" of the build system.
Concretely, for the ZK / Curator case: Why is that needed in 6 modules? Should it not be only needed in `flink-runtime` for our HA Services? If the same think is used in `flink-mesos`, don't we implicitly require that the Mesos ZK and the Flink HA ZK are the same version?
In a clean setup, I would imagine that we have
- a `flink-ha-services` module with the HA Services abstraction
- two different sub-modules for ZK 3.4 (`flink-ha-services-zk-3_4`) and ZK 3.5 (`flink-ha-services-zk-3_5`), with the code and the tests.
- Maybe most of the code/tests is taken from a shared `flink-ha-services-zk-base` module.
- Each module does its tests for the ZK HA integration. No dependency change needed in any other module to test cross versions.
- Possibly one end-2-end tests per version, but those should work "like users" meaning taking the relevant jar from the `/opt` folder.
- In that approach, we also have the flexibility to implement the operations differently in different versions, like using "container" nodes in ZK 3.5, transactions, etc.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
StephanEwen commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-591330906
To avoid confusion, I am +1 to merge this, but would suggest to think in the way of the above comments for the future.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589651425
@StephanEwen you still want to use zk 3.5 during IT cases / tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
StephanEwen commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629392
Looks good in general.
I am wondering if we could avoid the extra maven profile. With each additional profile, it becomes a tad bit harder to understand/maintain.
Could the tests (at least the end to end tests) not simply copy the jar from opt to lib, rather than relying on a build profile?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] StephanEwen edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
StephanEwen edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-590084670
I see. I guess under the current setup, that approach is fair enough.
It would be nice to keep working on a bit better encapsulation, because the missing encapsulation contributes to the increasing "unwieldyness" of the build system.
Concretely, for the ZK / Curator case: Why is that needed in 6 modules? Should it not be only needed in `flink-runtime` for our HA Services? If the same think is used in `flink-mesos`, don't we implicitly require that the Mesos ZK and the Flink HA ZK are the same version?
In a clean setup, I would imagine that we have
- a `flink-ha-services` module with the HA Services abstraction
- two different sub-modules for ZK 3.4 (`flink-ha-services-zk-3_4`) and ZK 3.5 (`flink-ha-services-zk-3_5`), with the code and the tests.
- Maybe most of the code/tests is taken from a shared `flink-ha-services-zk-base` module.
- Each module does its tests for the ZK HA integration. No dependency change needed in any other module to test cross versions.
- Possibly one end-2-end tests per version, but those should work "like users" meaning taking the relevant jar from the `/opt` folder.
- In that approach, we also have the flexibility to implement the operations differently in different versions, like using "container" nodes in ZK 3.5, transactions, etc.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589671556
The profile modifies `curator.version` which is used in 6 different modules, along with `zookeeper.version` which is used in the dependencyManagement entry for `org.apache.zookeeper` (also in the root pom) which probably also appears in various placed in the dependency graph.
In other words the root pom is pretty much the only place to put it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-591402360
One example for `curator-test` being used outside of `flink-runtime` is `flink-tests`, which runs some Zookeeper IT cases. For these the curator-test version has to correspond to the version in flink-runtime.
AFAIK some other modules also use curator-test but the flink-runtime ZK may not necessarily be involved here.
Ultimately I don't know what most of these modules do with Zookeeper, but I understood the JIRA to add ZK 3.5 support for _all modules_, not just flink-runtime.
I had an offline chat with Till about this PR.
He raised a good argument in that the "global ZK version" approach is flawed as any module bundling ZK (apart from ZK) is hard-wired at compile-time to that version.
Additionally, it shouldn't be required to actually test against both ZK 3.4 and 3.5 . So long as we use 3.4 as the default curator should ensure that we can talk to both versions.
Simply put we propose to the drop the ZK 3.5 profile, only run unit/it tests against ZK 3.4, and possible adjust an E2E test to run against 3.5, with the added benefit of actually testing how users would switch between versions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589612980
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 80755e8c69c712ebc1a2216f4b38bf23939deeea (Fri Feb 21 11:21:46 UTC 2020)
**Warnings:**
* **4 pom.xml files were touched**: Check for build and licensing issues.
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150002330",
"triggerID" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"triggerType" : "PUSH"
}, {
"hash" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150026334",
"triggerID" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 80755e8c69c712ebc1a2216f4b38bf23939deeea Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150002330)
* e4bef951ad8869425ce4afcfa8bfb19980de030d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150026334)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-591402360
One example for `curator-test` being used outside of `flink-runtime` is `flink-tests`, which runs some Zookeeper IT cases. For these the curator-test version has to correspond to the version in flink-runtime.
AFAIK some other modules also use curator-test but the flink-runtime ZK may not necessarily be involved here.
Ultimately I don't know what most of these modules do with Zookeeper, but I understood the JIRA to add ZK 3.5 support for _all modules_, not just flink-runtime.
I had an offline chat with Till about this PR.
He raised a good argument in that the "global ZK version" approach is flawed as any module bundling ZK (apart from ZK) is hard-wired at compile-time to that version.
Additionally, it shouldn't be required to actually test against both ZK 3.4 and 3.5 . So long as use 3.4 as the default curator should ensure that we can talk to both versions.
Simply put we propose to the drop the ZK 3.5 profile, only run tests against ZK 3.4, and possible adjust an E2E test to run against 3.5, with the added benefit of actually testing how users would switch between versions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
Hash:80755e8c69c712ebc1a2216f4b38bf23939deeea Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/150002330 TriggerType:PUSH TriggerID:80755e8c69c712ebc1a2216f4b38bf23939deeea
Hash:e4bef951ad8869425ce4afcfa8bfb19980de030d Status:PENDING URL:https://travis-ci.com/flink-ci/flink/builds/150026334 TriggerType:PUSH TriggerID:e4bef951ad8869425ce4afcfa8bfb19980de030d
-->
## CI report:
* 80755e8c69c712ebc1a2216f4b38bf23939deeea Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150002330)
* e4bef951ad8869425ce4afcfa8bfb19980de030d Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/150026334)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589671556
The profile modifies `curator.version` which is used in 6 different modules, along with `zookeeper.version` which is used in the dependencyManagement entry for `org.apache.zookeeper` which probably also appears in various placed in the dependency graph.
In other words the root pom is pretty much the only place to put it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] StephanEwen edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
StephanEwen edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-590084670
I see. I guess under the current setup, that approach is fair enough.
It would be nice to keep working on a bit better encapsulation, because the missing encapsulation contributes to the increasing "unwieldyness" of the build system.
Concretely, for the ZK / Curator case: Why is that needed in 6 modules? Should it not be only needed in `flink-runtime` for our HA Services? If the same version/dependency is used in `flink-mesos`, don't we implicitly require that the Mesos ZK and the Flink HA ZK are the same version?
In a clean setup, I would imagine that we have
- a `flink-ha-services` module with the HA Services abstraction
- two different sub-modules for ZK 3.4 (`flink-ha-services-zk-3_4`) and ZK 3.5 (`flink-ha-services-zk-3_5`), with the code and the tests.
- Maybe most of the code/tests is taken from a shared `flink-ha-services-zk-base` module.
- Each module does its tests for the ZK HA integration. No dependency change needed in any other module to test cross versions.
- Possibly one end-2-end tests per version, but those should work "like users" meaning taking the relevant jar from the `/opt` folder.
- In that approach, we also have the flexibility to implement the operations differently in different versions, like using "container" nodes in ZK 3.5, transactions, etc.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150002330",
"triggerID" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"triggerType" : "PUSH"
}, {
"hash" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150026334",
"triggerID" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e4bef951ad8869425ce4afcfa8bfb19980de030d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150026334)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589651735
@TisonKun As noted in the PR description, `curator.version` is only used for curator-test, nothing else. It does not affect the runtime curator version, which is also 4.X.Y.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589651425
@StephanEwen you still want to use zk 3.5 during IT cases / tests, and having the logic in the build system is more maintainable than custom code in the e2e scripts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-591402360
One example for `curator-test` being used outside of `flink-runtime` is `flink-tests`, which runs some Zookeeper IT cases. For these the curator-test version has to correspond to the version in flink-runtime. So this isn't just a problem in the build system but also how we write tests.
AFAIK some other modules also use curator-test but the flink-runtime ZK may not necessarily be involved here.
Ultimately I don't know what most of these modules do with Zookeeper, but I understood the JIRA to add ZK 3.5 support for _all modules_, not just flink-runtime.
I had an offline chat with Till about this PR.
He raised a good argument in that the "global ZK version" approach is flawed as any module bundling ZK (apart from ZK) is hard-wired at compile-time to that version.
Additionally, it shouldn't be required to actually test against both ZK 3.4 and 3.5 . So long as we use 3.4 as the default curator should ensure that we can talk to both versions.
Simply put we propose to the drop the ZK 3.5 profile, only run unit/it tests against ZK 3.4, and possible adjust an E2E test to run against 3.5, with the added benefit of actually testing how users would switch between versions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150002330",
"triggerID" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"triggerType" : "PUSH"
}, {
"hash" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150026334",
"triggerID" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"triggerType" : "PUSH"
}, {
"hash" : "5886b5fb58462c1cb7f2da19f684c3075e7c09c1",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151058259",
"triggerID" : "5886b5fb58462c1cb7f2da19f684c3075e7c09c1",
"triggerType" : "PUSH"
}, {
"hash" : "e5737cada14cd00a6642c07998590231c38172e5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5737cada14cd00a6642c07998590231c38172e5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5886b5fb58462c1cb7f2da19f684c3075e7c09c1 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/151058259)
* e5737cada14cd00a6642c07998590231c38172e5 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150002330",
"triggerID" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"triggerType" : "PUSH"
}, {
"hash" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150026334",
"triggerID" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"triggerType" : "PUSH"
}, {
"hash" : "5886b5fb58462c1cb7f2da19f684c3075e7c09c1",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151058259",
"triggerID" : "5886b5fb58462c1cb7f2da19f684c3075e7c09c1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e4bef951ad8869425ce4afcfa8bfb19980de030d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150026334)
* 5886b5fb58462c1cb7f2da19f684c3075e7c09c1 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/151058259)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150002330",
"triggerID" : "80755e8c69c712ebc1a2216f4b38bf23939deeea",
"triggerType" : "PUSH"
}, {
"hash" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150026334",
"triggerID" : "e4bef951ad8869425ce4afcfa8bfb19980de030d",
"triggerType" : "PUSH"
}, {
"hash" : "5886b5fb58462c1cb7f2da19f684c3075e7c09c1",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/151058259",
"triggerID" : "5886b5fb58462c1cb7f2da19f684c3075e7c09c1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5886b5fb58462c1cb7f2da19f684c3075e7c09c1 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/151058259)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-591402360
One example for `curator-test` being used outside of `flink-runtime` is `flink-tests`, which runs some Zookeeper IT cases. For these the curator-test version has to correspond to the version in flink-runtime.
AFAIK some other modules also use curator-test but the flink-runtime ZK may not necessarily be involved here.
Ultimately I don't know what most of these modules do with Zookeeper, but I understood the JIRA to add ZK 3.5 support for _all modules_, not just flink-runtime.
I had an offline chat with Till about this PR.
He raised a good argument in that the "global ZK version" approach is flawed as any module bundling ZK (apart from ZK) is hard-wired at compile-time to that version.
Additionally, it shouldn't be required to actually test against both ZK 3.4 and 3.5 . So long as we use 3.4 as the default curator should ensure that we can talk to both versions.
Simply put we propose to the drop the ZK 3.5 profile, only run tests against ZK 3.4, and possible adjust an E2E test to run against 3.5, with the added benefit of actually testing how users would switch between versions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol merged pull request #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol merged pull request #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
Hash:80755e8c69c712ebc1a2216f4b38bf23939deeea Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/150002330 TriggerType:PUSH TriggerID:80755e8c69c712ebc1a2216f4b38bf23939deeea
-->
## CI report:
* 80755e8c69c712ebc1a2216f4b38bf23939deeea Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150002330)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol commented on issue #11173: [FLINK-13417][build]
Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol commented on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-591403293
I agree that in the long term we should just have different HA server implementations that are loaded as plugins (at which point we can drop the shading again).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589629527
<!--
Meta data
Hash:80755e8c69c712ebc1a2216f4b38bf23939deeea Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/150002330 TriggerType:PUSH TriggerID:80755e8c69c712ebc1a2216f4b38bf23939deeea
Hash:e4bef951ad8869425ce4afcfa8bfb19980de030d Status:UNKNOWN URL:TBD TriggerType:PUSH TriggerID:e4bef951ad8869425ce4afcfa8bfb19980de030d
-->
## CI report:
* 80755e8c69c712ebc1a2216f4b38bf23939deeea Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150002330)
* e4bef951ad8869425ce4afcfa8bfb19980de030d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] zentol edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
zentol edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-591402360
One example for `curator-test` being used outside of `flink-runtime` is `flink-tests`, which runs some Zookeeper IT cases. For these the curator-test version has to correspond to the version in flink-runtime. So this isn't just a problem in the build system but also how we write tests.
AFAIK some other modules also use curator-test but the flink-runtime ZK may not necessarily be involved here.
Ultimately I don't know what most of these modules do with Zookeeper, but I understood the JIRA to add ZK 3.5 support for _all modules_, not just flink-runtime.
I had an offline chat with Till about this PR.
He raised a good argument in that the "global ZK version" approach is flawed as any module bundling ZK (apart from flink-runtime) is hard-wired at compile-time to that version anyway, with no way for users to switch versions aside from recompiling Flink.
Additionally, it shouldn't be required to actually test against both ZK 3.4 and 3.5 . So long as we use 3.4 as the default curator should ensure that we can talk to both versions.
Simply put we propose to the drop the ZK 3.5 profile, only run unit/it tests against ZK 3.4, and possible adjust an E2E test to run against 3.5, with the added benefit of actually testing how users would switch between versions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] TisonKun edited a comment on issue #11173:
[FLINK-13417][build] Bundle Zookeeper 3.5 in opt
Posted by GitBox <gi...@apache.org>.
TisonKun edited a comment on issue #11173: [FLINK-13417][build] Bundle Zookeeper 3.5 in opt
URL: https://github.com/apache/flink/pull/11173#issuecomment-589643407
LGTM.
A possible follow-up is separating `curator.version` & `curator.test.version`. zk 3.4 works well with curator 4.x except when testing it requires curator-test 2.x, which is another artifact than curator-framework and so on.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services