You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@heron.apache.org by GitBox <gi...@apache.org> on 2020/07/29 07:29:57 UTC
[GitHub] [incubator-heron] nicknezis opened a new pull request #3593: Updating Kryo
nicknezis opened a new pull request #3593:
URL: https://github.com/apache/incubator-heron/pull/3593
Updating Kryo to a newer version. Currently 5.0.0-RC7 is the most recent release.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis commented on pull request #3593: Updating Kryo
Posted by GitBox <gi...@apache.org>.
nicknezis commented on pull request #3593:
URL: https://github.com/apache/incubator-heron/pull/3593#issuecomment-668383154
Waiting on this pull request before this is ready to merge. We will be able to remove the HashSetSerializer once this is incorporated and released as a new Kryo version. https://github.com/EsotericSoftware/kryo/pull/753
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] joshfischer1108 edited a comment on pull request #3593: Updating Kryo
Posted by GitBox <gi...@apache.org>.
joshfischer1108 edited a comment on pull request #3593:
URL: https://github.com/apache/incubator-heron/pull/3593#issuecomment-674588198
I built from source and ran an ECO topology locally. No issues I could see. Steps to reproduce are below:
```
$ bazel build --config=darwin scripts/packages:binpkg
$ ./bazel-bin/scripts/packages/heron-install.sh --user
$ heron version
heron.build.git.revision : bb1509bc56a873ffc91c914a8f5f973ab2981718
heron.build.git.status : Clean
heron.build.host : Joshs-MacBook-Pro-2.local
heron.build.time : Sun Aug 16 16:27:24 CDT 2020
heron.build.timestamp : 1597613244000
heron.build.user : joshfischer
heron.build.version : nicknezis/kryo-upgrade
$ heron submit local ~/.heron/examples/heron-eco-examples.jar org.apache.heron.eco.Eco --eco-config-file ~/.heron/examples/heron_wordcount.yaml
$ tail -f ~/.herondata/topologies/local/joshfischer/heron-simple-wordcount-topology/log-files/container_1_bolt-2_3.log.0
[2020-08-16 17:50:31 -0500] [INFO] org.apache.heron.examples.eco.LogInfoBolt: { }source: bolt-1, stream: default, [bart, 118]
[2020-08-16 17:50:31 -0500] [INFO] org.apache.heron.examples.eco.LogInfoBolt: { }source: bolt-1, stream: default, [bart, 119]
[2020-08-16 17:50:31 -0500] [INFO] org.apache.heron.examples.eco.LogInfoBolt: { }source: bolt-1, stream: default, [bart, 120]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] joshfischer1108 commented on pull request #3593: Updating Kryo
Posted by GitBox <gi...@apache.org>.
joshfischer1108 commented on pull request #3593:
URL: https://github.com/apache/incubator-heron/pull/3593#issuecomment-674588198
I built from source and ran an ECO topology locally. No issues I could see. Steps to reproduce are below:
```
$ bazel build --config=darwin scripts/packages:binpkg
$ ./bazel-bin/scripts/packages/heron-install.sh --user
$ heron version
heron.build.git.revision : bb1509bc56a873ffc91c914a8f5f973ab2981718
heron.build.git.status : Clean
heron.build.host : Joshs-MacBook-Pro-2.local
heron.build.time : Sun Aug 16 16:27:24 CDT 2020
heron.build.timestamp : 1597613244000
heron.build.user : joshfischer
heron.build.version : nicknezis/kryo-upgrade
$ tail -f ~/.herondata/topologies/local/joshfischer/heron-simple-wordcount-topology/log-files/container_1_bolt-2_3.log.0
[2020-08-16 17:50:31 -0500] [INFO] org.apache.heron.examples.eco.LogInfoBolt: { }source: bolt-1, stream: default, [bart, 118]
[2020-08-16 17:50:31 -0500] [INFO] org.apache.heron.examples.eco.LogInfoBolt: { }source: bolt-1, stream: default, [bart, 119]
[2020-08-16 17:50:31 -0500] [INFO] org.apache.heron.examples.eco.LogInfoBolt: { }source: bolt-1, stream: default, [bart, 120]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis merged pull request #3593: Updating Kryo
Posted by GitBox <gi...@apache.org>.
nicknezis merged pull request #3593:
URL: https://github.com/apache/incubator-heron/pull/3593
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis commented on pull request #3593: Updating Kryo
Posted by GitBox <gi...@apache.org>.
nicknezis commented on pull request #3593:
URL: https://github.com/apache/incubator-heron/pull/3593#issuecomment-667715923
I'm waiting for some feedback on a feature request in Kryo. Also might wait until the official 5.0.0 release because the remaining issue before release is a performance related issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] joshfischer1108 commented on pull request #3593: Updating Kryo
Posted by GitBox <gi...@apache.org>.
joshfischer1108 commented on pull request #3593:
URL: https://github.com/apache/incubator-heron/pull/3593#issuecomment-678459311
@nwangtw @huijunw could you check this PR before me merge?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis commented on a change in pull request #3593: Updating Kryo
Posted by GitBox <gi...@apache.org>.
nicknezis commented on a change in pull request #3593:
URL: https://github.com/apache/incubator-heron/pull/3593#discussion_r471203981
##########
File path: maven_install.json
##########
@@ -1,6 +1,6 @@
{
"dependency_tree": {
- "__AUTOGENERATED_FILE_DO_NOT_MODIFY_THIS_FILE_MANUALLY": -1332813254,
+ "__AUTOGENERATED_FILE_DO_NOT_MODIFY_THIS_FILE_MANUALLY": -790274667,
Review comment:
Where would be a good spot to add it? Currently I only added it here in the `WORKSPACE` file. https://github.com/apache/incubator-heron/blob/ee73293b2ce5f1c0aa0506a4398bb2efe696944f/WORKSPACE#L152-L155
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] joshfischer1108 commented on a change in pull request #3593: Updating Kryo
Posted by GitBox <gi...@apache.org>.
joshfischer1108 commented on a change in pull request #3593:
URL: https://github.com/apache/incubator-heron/pull/3593#discussion_r471170359
##########
File path: release/maven/heron-with-kryo.template.pom
##########
@@ -36,7 +36,7 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/ma
<dependency>
<groupId>com.esotericsoftware</groupId>
<artifactId>kryo</artifactId>
- <version>3.0.3</version>
+ <version>5.0.0-RC9</version>
Review comment:
It's awesome you worked with them to get this release candidate out for us. 💯
##########
File path: examples/src/java/org/apache/heron/examples/api/AckingTopology.java
##########
@@ -60,6 +60,8 @@ public static void main(String[] args) throws Exception {
Config conf = new Config();
conf.setDebug(true);
+ conf.setSerializationClassName(Config.HERON_KRYO_SERIALIZER_CLASS_NAME);
+
Review comment:
It would be good to create an issue for us to try and explain in more detail how Kryo fits into and works with the Heron system. I think it's a bit of a black box at the moment. Doesn't need to be done with this PR.
##########
File path: maven_install.json
##########
@@ -1,6 +1,6 @@
{
"dependency_tree": {
- "__AUTOGENERATED_FILE_DO_NOT_MODIFY_THIS_FILE_MANUALLY": -1332813254,
+ "__AUTOGENERATED_FILE_DO_NOT_MODIFY_THIS_FILE_MANUALLY": -790274667,
Review comment:
Do we have anything in the documentation that shows how to regenerate this `maven_install` file? Again, doesn't need to be done in this PR. But would be helpful to know more about it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org