You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gearpump.apache.org by manuzhang <gi...@git.apache.org> on 2017/10/20 00:02:30 UTC
[GitHub] incubator-gearpump pull request #233: [WIP] Fix integration test
GitHub user manuzhang opened a pull request:
https://github.com/apache/incubator-gearpump/pull/233
[WIP] Fix integration test
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the commit message is formatted like:
`[GEARPUMP-<Jira issue #>] Meaningful description of pull request`
- [ ] Make sure tests pass via `sbt clean test`.
- [ ] Make sure old documentation affected by the pull request has been updated and new documentation added for new functionality.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/manuzhang/incubator-gearpump fix-it
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-gearpump/pull/233.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #233
----
commit e888924ef8ce3f73efdc20a6207fee0e3b38e91b
Author: manuzhang <ow...@gmail.com>
Date: 2017-10-20T00:01:52Z
[WIP] Fix integration test
----
---
[GitHub] incubator-gearpump pull request #233: [GEARPUMP-361] Build gearpump docker i...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/233#discussion_r147378048
--- Diff: integrationtest/docker/gearpump/Dockerfile ---
@@ -18,7 +18,8 @@
FROM openjdk:8u131-jre-alpine
# Install python
-RUN apk add --no-cache python && \
+RUN apk add --update curl && \
--- End diff --
please, use `apk add --update curl python` (one line)
---
[GitHub] incubator-gearpump issue #233: [GEARPUMP-361] Build gearpump docker image lo...
Posted by huafengw <gi...@git.apache.org>.
Github user huafengw commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
+1
---
[GitHub] incubator-gearpump issue #233: [WIP] Fix integration test
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
I've just confirmed. The problem persists, even when networking/proxy problems are overcomed:
```
$ docker build --no-cache -t gearpump/gearpump-launcher integrationtest/docker/gearpump
Sending build context to Docker daemon 8.192kB
Step 1/7 : FROM grubykarol/jre:8u121b13_server
---> f7e482e81614
Step 2/7 : RUN opkg-install python
---> Running in 0939e0c36efc
Downloading http://downloads.openwrt.org/snapshots/trunk/x86/64/packages/base/Packages.gz.
Inflating http://downloads.openwrt.org/snapshots/trunk/x86/64/packages/base/Packages.gz.
Updated list of available packages in /var/opkg-lists/base.
Downloading http://downloads.openwrt.org/snapshots/trunk/x86/64/packages/packages/Packages.gz.
Inflating http://downloads.openwrt.org/snapshots/trunk/x86/64/packages/packages/Packages.gz.
Updated list of available packages in /var/opkg-lists/packages.
Unknown package 'python'.
Collected errors:
* opkg_install_cmd: Cannot install package python.
```
`Unknown package 'python'.`
---
[GitHub] incubator-gearpump issue #233: [GEARPUMP-361] Build gearpump docker image lo...
Posted by huafengw <gi...@git.apache.org>.
Github user huafengw commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
The 3rd problem occurs on Mac and centos works fine.
---
[GitHub] incubator-gearpump pull request #233: [GEARPUMP-361] Build gearpump docker i...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-gearpump/pull/233
---
[GitHub] incubator-gearpump issue #233: [WIP] Fix integration test
Posted by manuzhang <gi...@git.apache.org>.
Github user manuzhang commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
It seems python package is not supported by busybox. Can we switch to the official `openjdk/8-jre-alpine` image ? It's 120MB after installing python.
---
[GitHub] incubator-gearpump issue #233: [GEARPUMP-361] Build gearpump docker image lo...
Posted by huafengw <gi...@git.apache.org>.
Github user huafengw commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
1. I have to build with explicit proxies like
`docker build --build-arg HTTP_PROXY=... --build-arg HTTPS_PROXY=...`.
If not, building step
`fetch http://dl-cdn.alpinelinux.org/alpine/v3.6/main/x86_64/APKINDEX.tar.gz` will always fail.
2. Please add following change in `integrationtest/docker/gearpump/Dockerfile`
```
-RUN apk add --no-cache python && \
+RUN apk add --update curl && \
+ apk add --no-cache python && \
```
3. After resolved these issues, the integration test still can't proceed. Here is the error log:
```
[DEBUG] [15:21:49.279] [Docker$] Container master0====>> curl http://master0:8090/api/v1.0/master
[DEBUG] [15:21:49.279] [Docker$] INPUT==>>
[DEBUG] [15:21:49.279] [ShellExec$] EXEC master0 => `docker exec master0 curl -s --cookie cookie.txt http://master0:8090/api/v1.0/master`
[DEBUG] [15:21:49.636] [ShellExec$] EXEC master0 <= `<HTML>
<HEAD><TITLE>Redirection</TITLE></HEAD>
<BODY><H1>Redirect</H1></BODY>
</HTML>` exit 0
[DEBUG] [15:21:49.636] [Docker$] <<==OUTPUT
[DEBUG] [15:21:49.636] [Docker$] <HTML>
<HEAD><TITLE>Redirection</TITLE></HEAD>
<BODY><H1>Redirect</H1></BODY>
</HTML>
[DEBUG] [15:21:49.637] [Docker$] <<====Command END. Container master0, curl http://master0:8090/api/v1.0/master
[ERROR] [15:21:49.637] [RestClient] Failed to decode Rest response to MasterData
[ERROR] [15:21:49.637] [Util$] Failed due to (false == cluster running), retrying for the 15 times...
```
---
[GitHub] incubator-gearpump issue #233: [WIP] Fix integration test
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
Even when providing proxy args for docker build:
```
docker build --no-cache --build-arg HTTP_PROXY=$http_proxy --build-arg HTTPS_PROXY=$https_proxy -t gearpump/gearpump-launcher integrationtest/docker/gearpump
```
It still fails to install python. I'll try to find a workaround for this.
---
[GitHub] incubator-gearpump issue #233: [GEARPUMP-361] Build gearpump docker image lo...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
The third problem - works for me on Ubuntu 16.04.
```
[DEBUG] [05:25:55.233] [Docker$] Container master0====>> curl http://master0:8090/api/v1.0/master
[DEBUG] [05:25:55.233] [Docker$] INPUT==>>
[DEBUG] [05:25:55.233] [ShellExec$] EXEC master0 => `docker exec master0 curl -s --cookie cookie.txt http://master0:8090/api/v1.0/master`
[DEBUG] [05:25:55.379] [ShellExec$] EXEC master0 <= `{"masterDescription":{"leader":{"host":"master@master0","port":3000},"cluster":[{"host":"master0","port":3000}],"aliveFor":"8068","logFile":"/opt/gearpump/var/log/gearpump/daemon","jarStore":"jarstore...` exit 0
[DEBUG] [05:25:55.379] [Docker$] <<==OUTPUT
```
Checking on Mac...
---
[GitHub] incubator-gearpump issue #233: [GEARPUMP-361] Build gearpump docker image lo...
Posted by manuzhang <gi...@git.apache.org>.
Github user manuzhang commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
@huafengw can't reproduce 1 and 3 on Mac
---
[GitHub] incubator-gearpump issue #233: [GEARPUMP-361] Build gearpump docker image lo...
Posted by manuzhang <gi...@git.apache.org>.
Github user manuzhang commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
@karol-brejna-i @huafengw rebased. Please help to verify again
---
[GitHub] incubator-gearpump issue #233: [WIP] Fix integration test
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
Docker image problem solved with https://github.com/apache/incubator-gearpump/pull/235.
---
[GitHub] incubator-gearpump issue #233: [WIP] Fix integration test
Posted by codecov-io <gi...@git.apache.org>.
Github user codecov-io commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
# [Codecov](https://codecov.io/gh/apache/incubator-gearpump/pull/233?src=pr&el=h1) Report
> Merging [#233](https://codecov.io/gh/apache/incubator-gearpump/pull/233?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gearpump/commit/0ac68b87f45b31a082cac74e616c09c14d6c69b0?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #233 +/- ##
=======================================
Coverage 69.05% 69.05%
=======================================
Files 191 191
Lines 6123 6123
Branches 357 357
=======================================
Hits 4228 4228
Misses 1895 1895
```
---
[GitHub] incubator-gearpump issue #233: [GEARPUMP-361] Build gearpump docker image lo...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
```[DEBUG] [13:07:44.774] [Docker$] Container master0====>> curl http://master0:8090/api/v1.0/master
[DEBUG] [13:07:44.774] [Docker$] INPUT==>>
[DEBUG] [13:07:44.775] [ShellExec$] EXEC master0 => `docker exec master0 curl -s --cookie cookie.txt http://master0:8090/api/v1.0/master`
[DEBUG] [13:07:45.490] [ShellExec$] EXEC master0 <= `{"masterDescription":{"leader":{"host":"master@master0","port":3000},"cluster":[{"host":"master0","port":3000}],"aliveFor":"18124","logFile":"/opt/gearpump/logs","jarStore":"jarstore/","masterStatus":...` exit 0
[DEBUG] [13:07:45.490] [Docker$] <<==OUTPUT
[DEBUG] [13:07:45.490] [Docker$] {"masterDescription":{"leader":{"host":"master@master0","port":3000},"cluster":[{"host":"master0","port":3000}],"aliveFor":"18124","logFile":"/opt/gearpump/logs","jarStore":"jarstore/","masterStatus":"synced","homeDirectory":"/opt/gearpump","activities":[],"jvmName":"7@master0","historyMetricsConfig":{"retainHistoryDataHours":72,"retainHistoryDataIntervalMs":3600000,"retainRecentDataSeconds":300,"retainRecentDataIntervalMs":15000}}}
[DEBUG] [13:07:45.491] [Docker$] <<====Command END. Container master0, curl http://master0:8090/api/v1.0/master
```
Number 3 looks good on mac, too. For me it seems like some proxy-caused redirect...
---
[GitHub] incubator-gearpump issue #233: [WIP] Fix integration test
Posted by manuzhang <gi...@git.apache.org>.
Github user manuzhang commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
R: @karol-brejna-i @huafengw Build Gearpump Image locally to make it easier to update and fix bugs
---
[GitHub] incubator-gearpump issue #233: [WIP] Fix integration test
Posted by huafengw <gi...@git.apache.org>.
Github user huafengw commented on the issue:
https://github.com/apache/incubator-gearpump/pull/233
Verified on centos, the updated image can be launched successfully.
---