You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gearpump.apache.org by karol-brejna-i <gi...@git.apache.org> on 2017/02/22 09:00:53 UTC
[GitHub] incubator-gearpump pull request #158: [GEARPUMP-267] Changing docker image f...
GitHub user karol-brejna-i opened a pull request:
https://github.com/apache/incubator-gearpump/pull/158
[GEARPUMP-267] Changing docker image for Kafka (spotify/kafka doesn't have tag for 0.8.x)
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the commit message is formatted like:
`[GEARPUMP-<Jira issue #>] Meaningful description of pull request`
- [ ] Make sure tests pass via `sbt clean test`.
- [ ] Make sure old documentation affected by the pull request has been updated and new documentation added for new functionality.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/karol-brejna-i/incubator-gearpump GEARPUMP-267
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-gearpump/pull/158.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #158
----
commit 7da53b7201fd4ca95e66c9920191a2970b6ca95e
Author: karol brejna <ka...@intel.com>
Date: 2017-02-21T13:54:10Z
[GEARPUMP-236] Using new sbt-assembly version to fix java.lang.VerifyError
commit 12e8a0380754a6dcb253e75b310dcf2a82244aa5
Author: karol brejna <ka...@intel.com>
Date: 2017-02-22T08:39:21Z
[GEARPUMP-267] Changing docker image for Kafka (spotify/kafka doesn't have tag for 0.8.x)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
Probably related to
https://docs.oracle.com/javase/8/docs/api/java/time/Instant.html#toEpochMilli--
The logs show:
`[INFO] [02/22/2017 10:25:34.395] [KafkaSource] KafkaSource opened at start time -292275055-05-16T16:47:04.192Z`
Then it looks like converting it in AbstractKafkaSource.open() fails:
`maybeRecover(startTime.toEpochMilli)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by huafengw <gi...@git.apache.org>.
Github user huafengw commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
Hi Karol, this pull request includes the former GEARPUMP-236 and it's also covered by your another pr, so maybe we can close this one?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by manuzhang <gi...@git.apache.org>.
Github user manuzhang commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
What's your jdk version ? Looks like hitting https://bugs.openjdk.java.net/browse/JDK-8074032
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
I used Java 8 from Oracle, more precisely:
``` java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
```
Let me upgrade to newer version and send an update.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
On the other hand converting fron tihs date works on my dev machine:
```
import java.time.Instant
type TimeStamp = Long
val MIN_TIME_MILLIS: Long = Long.MinValue
var _minClock: TimeStamp = MIN_TIME_MILLIS
val startTime = Instant.ofEpochMilli(_minClock)
val result = startTime.toEpochMilli
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
gearump-launcher that we use in integration tests uses errordeveloper/oracle-jre image that has JDK 8u45-b14. I updated JRE image and will use it in gearpump-launcher to see if new version of Java fixes the tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
I've collected docker definitions in one place (integrationtest/docker), upgraded java version for gearpump-launcher, switched to "our" kafka image (0.8.2.1): https://github.com/apache/incubator-gearpump/pull/162
It looks like all the integration tests pass now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
The good news is, Kafka "cluster" is up again (during integration tests).
The bad news, there is some error running Kafka example.
I found this in application logs:
```
INFO] [02/22/2017 08:20:08.282] [Executor@app2exec1] Executor received restart tasks
[WARN] [02/22/2017 08:20:08.282] [ProducerConfig] The configuration value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer was supplied but isn't a known config.
[WARN] [02/22/2017 08:20:08.283] [ProducerConfig] The configuration key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer was supplied but isn't a known config.
[INFO] [02/22/2017 08:20:08.283] [KafkaSource] created checkpoint store for [topic3,0]
[ERROR] [02/22/2017 08:20:08.283] [Executor@app2exec1] We got java.lang.ArithmeticException from Some(TaskId(0,0)), we will treat it as MessageLoss, so that the system will replay all lost message
java.lang.ArithmeticException: long overflow
at java.lang.Math.multiplyExact(Math.java:892)
at java.time.Instant.toEpochMilli(Instant.java:1232)
at org.apache.gearpump.streaming.kafka.lib.source.AbstractKafkaSource.open(AbstractKafkaSource.scala:94)
at org.apache.gearpump.streaming.source.DataSourceTask.onStart(DataSourceTask.scala:71)
at org.apache.gearpump.streaming.task.TaskWrapper.onStart(TaskWrapper.scala:96)
at org.apache.gearpump.streaming.task.TaskActor.onStart(TaskActor.scala:107)
at org.apache.gearpump.streaming.task.TaskActor.org$apache$gearpump$streaming$task$TaskActor$$onStartClock(TaskActor.scala:195)
at org.apache.gearpump.streaming.task.TaskActor$$anonfun$waitForStartClock$1.applyOrElse(TaskActor.scala:212)
at akka.actor.Actor$class.aroundReceive(Actor.scala:496)
at org.apache.gearpump.streaming.task.TaskActor.aroundReceive(TaskActor.scala:42)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[WARN] [02/22/2017 08:20:08.284] [OneForOneStrategy] long overflow
[INFO] [02/22/2017 08:20:08.285] [DataSourceTask@app2exec1TaskId(0,0)] closing data source...
[INFO] [02/22/2017 08:20:08.283] [DataSinkTask@app2exec1TaskId(1,0)] closing data sink...
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by huafengw <gi...@git.apache.org>.
Github user huafengw commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
@manuzhang Looks like it's related to the timestamp
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
Now it's obsolete (implemented by two other PRs). Closing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #158: [GEARPUMP-267] Changing docker image for Kafk...
Posted by codecov-io <gi...@git.apache.org>.
Github user codecov-io commented on the issue:
https://github.com/apache/incubator-gearpump/pull/158
# [Codecov](https://codecov.io/gh/apache/incubator-gearpump/pull/158?src=pr&el=h1) Report
> Merging [#158](https://codecov.io/gh/apache/incubator-gearpump/pull/158?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gearpump/commit/c9b2cea813e5e8262fbbcf0e9e72c13cd463a424?src=pr&el=desc) will **decrease** coverage by `-0.37%`.
> The diff coverage is `100%`.
```diff
@@ Coverage Diff @@
## master #158 +/- ##
==========================================
- Coverage 71.89% 71.53% -0.37%
==========================================
Files 190 190
Lines 6056 6053 -3
Branches 534 534
==========================================
- Hits 4354 4330 -24
- Misses 1702 1723 +21
```
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gearpump/pull/158?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `\u0394 = absolute <relative> (impact)`, `� = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-gearpump/pull/158?src=pr&el=footer). Last update [c9b2cea...12e8a03](https://codecov.io/gh/apache/incubator-gearpump/compare/c9b2cea813e5e8262fbbcf0e9e72c13cd463a424...12e8a0380754a6dcb253e75b310dcf2a82244aa5?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #158: [GEARPUMP-267] Changing docker image f...
Posted by karol-brejna-i <gi...@git.apache.org>.
Github user karol-brejna-i closed the pull request at:
https://github.com/apache/incubator-gearpump/pull/158
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---