You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by GitBox <gi...@apache.org> on 2020/03/26 23:36:20 UTC
[GitHub] [samza-hello-samza] kw2542 opened a new pull request #79: Update
doc and javadoc from config factory to config loader
kw2542 opened a new pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79
Update doc and javadoc from config factory to config loader
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] cameronlee314 commented on a change in pull
request #79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
cameronlee314 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r404301906
##########
File path: src/main/config/azure-blob-application.properties
##########
@@ -30,10 +30,14 @@ yarn.package.path=file://${basedir}/target/${project.artifactId}-${pom.version}-
# StreamApplication class
app.class=samza.examples.azure.AzureBlobApplication
-#Azure blob essential configs
+# Azure blob essential configs
systems.azure-blob-container.samza.factory=org.apache.samza.system.azureblob.AzureBlobSystemFactory
sensitive.systems.azure-blob-container.azureblob.account.name=your-azure-storage-account-name
sensitive.systems.azure-blob-container.azureblob.account.key=your-azure-storage-account-key
+# Config Loader
+job.config.loader.factory=org.apache.samza.config.loaders.PropertiesConfigLoaderFactory
+job.config.loader.properties.path=./__package/config/azure-blob-application.properties
Review comment:
Do you think it might be confusing to users what `__package` refers to? That is kind of a YARN implementation detail. Maybe documentation can help clarify, but if you can find a programatic way to hide that implementation detail, that would be nice too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] cameronlee314 commented on a change in pull
request #79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
cameronlee314 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r400365471
##########
File path: README.md
##########
@@ -61,13 +61,19 @@ Package [samza.examples.cookbook](https://github.com/apache/samza-hello-samza/tr
Package [samza.examples.wikipedia.application](https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application) contains a small Samza application which consumes the real-time feeds from Wikipedia, extracts the metadata of the events, and calculates statistics of all edits in a 10-second window. You can start the app on the grid using the run-app.sh script:
```
-./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application.properties
+./deploy/samza/bin/run-app.sh \
+ --config app.class=samza.examples.wikipedia.application.WikipediaApplication \
+ --config yarn.package.path=file:///Users/kwu/workspace/hello-samza/target/hello-samza-1.5.0-SNAPSHOT-dist.tar.gz \
+ --config job.name=wikipedia-application \
+ --config job.factory.class=org.apache.samza.job.yarn.YarnJobFactory \
Review comment:
Do you have a clean way to describe how to specify submission configs in general? Based on your comments, it seems like standalone (`LocalApplicationRunner`) specifies a different set of submission configs than YARN (`RemoteApplicationRunner`), even though some of those configs are general to Samza (e.g. `app.class`).
It would be good to have as few runner-specific steps as possible. `ApplicationRunner` is the interface, so it would be nice to not have to worry about the specific `ApplicationRunner` being used when trying to start the app. I admit that Samza does already do some environment-specific configs (e.g. YARN-specific configs are needed when using `YarnJobFactory`), but we should generally minimize that.
I'm not sure if this works, but could we recommend standalone to pass the larger set of submission configs (similar to YARN) also? Then there would be more consistency. It would be easier to describe what submission configs are and how to specify them in general.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] kw2542 commented on a change in pull request
#79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
kw2542 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r404500446
##########
File path: src/main/config/azure-blob-application.properties
##########
@@ -30,10 +30,14 @@ yarn.package.path=file://${basedir}/target/${project.artifactId}-${pom.version}-
# StreamApplication class
app.class=samza.examples.azure.AzureBlobApplication
-#Azure blob essential configs
+# Azure blob essential configs
systems.azure-blob-container.samza.factory=org.apache.samza.system.azureblob.AzureBlobSystemFactory
sensitive.systems.azure-blob-container.azureblob.account.name=your-azure-storage-account-name
sensitive.systems.azure-blob-container.azureblob.account.key=your-azure-storage-account-key
+# Config Loader
+job.config.loader.factory=org.apache.samza.config.loaders.PropertiesConfigLoaderFactory
+job.config.loader.properties.path=./__package/config/azure-blob-application.properties
Review comment:
Yes, `samza-hello-samza` build tarball already include config directory.
Agree, the migration guide for 1.5 need detailed migration instructions on this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] cameronlee314 commented on a change in pull
request #79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
cameronlee314 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r399574730
##########
File path: README.md
##########
@@ -61,13 +61,19 @@ Package [samza.examples.cookbook](https://github.com/apache/samza-hello-samza/tr
Package [samza.examples.wikipedia.application](https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application) contains a small Samza application which consumes the real-time feeds from Wikipedia, extracts the metadata of the events, and calculates statistics of all edits in a 10-second window. You can start the app on the grid using the run-app.sh script:
```
-./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application.properties
+./deploy/samza/bin/run-app.sh \
+ --config app.class=samza.examples.wikipedia.application.WikipediaApplication \
+ --config yarn.package.path=file:///Users/kwu/workspace/hello-samza/target/hello-samza-1.5.0-SNAPSHOT-dist.tar.gz \
+ --config job.name=wikipedia-application \
+ --config job.factory.class=org.apache.samza.job.yarn.YarnJobFactory \
Review comment:
These all seem to be copied from the properties file. It seems like it would be non-trivial to keep the properties file and this list of configs consistent, since they are in different places.
https://github.com/apache/samza/pull/1256 indicates that you only need the config loader and properties path, but here you are adding several other configs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] kw2542 commented on a change in pull request
#79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
kw2542 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r404501360
##########
File path: src/main/config/azure-blob-application.properties
##########
@@ -30,10 +30,14 @@ yarn.package.path=file://${basedir}/target/${project.artifactId}-${pom.version}-
# StreamApplication class
app.class=samza.examples.azure.AzureBlobApplication
-#Azure blob essential configs
+# Azure blob essential configs
systems.azure-blob-container.samza.factory=org.apache.samza.system.azureblob.AzureBlobSystemFactory
sensitive.systems.azure-blob-container.azureblob.account.name=your-azure-storage-account-name
sensitive.systems.azure-blob-container.azureblob.account.key=your-azure-storage-account-key
+# Config Loader
+job.config.loader.factory=org.apache.samza.config.loaders.PropertiesConfigLoaderFactory
+job.config.loader.properties.path=./__package/config/azure-blob-application.properties
Review comment:
Yes, "__package" is Samza's internal implementation detail where localized job to be unzipped at.
It is challenging to do it programmatically as clients may or may not include __package in the path, and different programs may have different locations to put their configs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] cameronlee314 commented on a change in pull
request #79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
cameronlee314 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r404985923
##########
File path: gradle/wrapper/gradle-wrapper.properties
##########
@@ -1,6 +1,6 @@
-#Mon Mar 23 14:55:28 PDT 2015
+#Fri Mar 27 16:28:33 PDT 2020
+distributionUrl=https\://services.gradle.org/distributions/gradle-2.3-all.zip
Review comment:
It's pretty minor. It's ok to leave it in this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] cameronlee314 commented on a change in pull
request #79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
cameronlee314 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r404293260
##########
File path: README.md
##########
@@ -61,7 +61,7 @@ Package [samza.examples.cookbook](https://github.com/apache/samza-hello-samza/tr
Package [samza.examples.wikipedia.application](https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application) contains a small Samza application which consumes the real-time feeds from Wikipedia, extracts the metadata of the events, and calculates statistics of all edits in a 10-second window. You can start the app on the grid using the run-app.sh script:
```
-./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application.properties
+./deploy/samza/bin/run-app.sh --config-path=$PWD/deploy/samza/config/wikipedia-application.properties
Review comment:
This `--config-path` here is for submission configs only, right? Can you think of a way to help clarify that? It looks like the example is suggesting to pass full job configs as submission configs. However, that could end up being a problem if the full job configs are too large (IIRC, there is a limit for the size of the submission configs env variable when passing to YARN).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] kw2542 commented on a change in pull request
#79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
kw2542 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r399576572
##########
File path: README.md
##########
@@ -61,13 +61,19 @@ Package [samza.examples.cookbook](https://github.com/apache/samza-hello-samza/tr
Package [samza.examples.wikipedia.application](https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application) contains a small Samza application which consumes the real-time feeds from Wikipedia, extracts the metadata of the events, and calculates statistics of all edits in a 10-second window. You can start the app on the grid using the run-app.sh script:
```
-./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application.properties
+./deploy/samza/bin/run-app.sh \
+ --config app.class=samza.examples.wikipedia.application.WikipediaApplication \
+ --config yarn.package.path=file:///Users/kwu/workspace/hello-samza/target/hello-samza-1.5.0-SNAPSHOT-dist.tar.gz \
+ --config job.name=wikipedia-application \
+ --config job.factory.class=org.apache.samza.job.yarn.YarnJobFactory \
Review comment:
submission related configs, such as job.name, app.class, job.factory.class and yarn.package.path needs to be explicitly passed in during submission since we are not reading config files anymore during submission.
I am cleaning up properties file ATM as well.
I believe the previous PR missed the instruction that all submission related configs needs to be provided explicitly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] cameronlee314 commented on a change in pull
request #79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
cameronlee314 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r404294708
##########
File path: gradle/wrapper/gradle-wrapper.properties
##########
@@ -1,6 +1,6 @@
-#Mon Mar 23 14:55:28 PDT 2015
+#Fri Mar 27 16:28:33 PDT 2020
+distributionUrl=https\://services.gradle.org/distributions/gradle-2.3-all.zip
Review comment:
This change is intentional for this PR, right? If so, it's ok to keep it; just double checking since it isn't quite related.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] kw2542 commented on a change in pull request
#79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
kw2542 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r404501952
##########
File path: gradle/wrapper/gradle-wrapper.properties
##########
@@ -1,6 +1,6 @@
-#Mon Mar 23 14:55:28 PDT 2015
+#Fri Mar 27 16:28:33 PDT 2020
+distributionUrl=https\://services.gradle.org/distributions/gradle-2.3-all.zip
Review comment:
It is auto updated by the build tool. I think I can revert this to keep it separated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] kw2542 commented on a change in pull request
#79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
kw2542 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r404502243
##########
File path: README.md
##########
@@ -61,7 +61,7 @@ Package [samza.examples.cookbook](https://github.com/apache/samza-hello-samza/tr
Package [samza.examples.wikipedia.application](https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application) contains a small Samza application which consumes the real-time feeds from Wikipedia, extracts the metadata of the events, and calculates statistics of all edits in a 10-second window. You can start the app on the grid using the run-app.sh script:
```
-./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application.properties
+./deploy/samza/bin/run-app.sh --config-path=$PWD/deploy/samza/config/wikipedia-application.properties
Review comment:
`LocalApplicationRunner` with `--config-path` is still submission config, no full config is needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] cameronlee314 commented on a change in pull
request #79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
cameronlee314 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r404307632
##########
File path: README.md
##########
@@ -61,7 +61,7 @@ Package [samza.examples.cookbook](https://github.com/apache/samza-hello-samza/tr
Package [samza.examples.wikipedia.application](https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application) contains a small Samza application which consumes the real-time feeds from Wikipedia, extracts the metadata of the events, and calculates statistics of all edits in a 10-second window. You can start the app on the grid using the run-app.sh script:
```
-./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application.properties
+./deploy/samza/bin/run-app.sh --config-path=$PWD/deploy/samza/config/wikipedia-application.properties
Review comment:
On the other hand, if someone switched this to use `LocalApplicationRunner`, then would the `--config-path` need to be the full configs? Would this overloading of the `--config-path` argument be confusing?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] cameronlee314 commented on a change in pull
request #79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
cameronlee314 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r399573818
##########
File path: README.md
##########
@@ -61,13 +61,19 @@ Package [samza.examples.cookbook](https://github.com/apache/samza-hello-samza/tr
Package [samza.examples.wikipedia.application](https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application) contains a small Samza application which consumes the real-time feeds from Wikipedia, extracts the metadata of the events, and calculates statistics of all edits in a 10-second window. You can start the app on the grid using the run-app.sh script:
```
-./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application.properties
+./deploy/samza/bin/run-app.sh \
+ --config app.class=samza.examples.wikipedia.application.WikipediaApplication \
+ --config yarn.package.path=file:///Users/kwu/workspace/hello-samza/target/hello-samza-1.5.0-SNAPSHOT-dist.tar.gz \
+ --config job.name=wikipedia-application \
+ --config job.factory.class=org.apache.samza.job.yarn.YarnJobFactory \
+ --config job.config.loader.factory=org.apache.samza.config.loaders.PropertiesConfigLoaderFactory \
+ --config job.config.loader.properties.path=$PWD/deploy/samza/config/wikipedia-application.properties
Review comment:
There are several other places which did not have these additional configs (e.g. `run-event-hubs-zk-application.sh`, `CouchbaseTableExample.java`). Do those places need to be updated also?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] kw2542 commented on a change in pull request
#79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
kw2542 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r399599309
##########
File path: README.md
##########
@@ -61,13 +61,19 @@ Package [samza.examples.cookbook](https://github.com/apache/samza-hello-samza/tr
Package [samza.examples.wikipedia.application](https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application) contains a small Samza application which consumes the real-time feeds from Wikipedia, extracts the metadata of the events, and calculates statistics of all edits in a 10-second window. You can start the app on the grid using the run-app.sh script:
```
-./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application.properties
+./deploy/samza/bin/run-app.sh \
+ --config app.class=samza.examples.wikipedia.application.WikipediaApplication \
+ --config yarn.package.path=file:///Users/kwu/workspace/hello-samza/target/hello-samza-1.5.0-SNAPSHOT-dist.tar.gz \
+ --config job.name=wikipedia-application \
+ --config job.factory.class=org.apache.samza.job.yarn.YarnJobFactory \
Review comment:
properties files cannot be simplified as they may also be used for standalone deployment where we do not need to feed the submission config.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] cameronlee314 commented on a change in pull
request #79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
cameronlee314 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r404303771
##########
File path: src/main/config/azure-blob-application.properties
##########
@@ -30,10 +30,14 @@ yarn.package.path=file://${basedir}/target/${project.artifactId}-${pom.version}-
# StreamApplication class
app.class=samza.examples.azure.AzureBlobApplication
-#Azure blob essential configs
+# Azure blob essential configs
systems.azure-blob-container.samza.factory=org.apache.samza.system.azureblob.AzureBlobSystemFactory
sensitive.systems.azure-blob-container.azureblob.account.name=your-azure-storage-account-name
sensitive.systems.azure-blob-container.azureblob.account.key=your-azure-storage-account-key
+# Config Loader
+job.config.loader.factory=org.apache.samza.config.loaders.PropertiesConfigLoaderFactory
+job.config.loader.properties.path=./__package/config/azure-blob-application.properties
Review comment:
If I understand correctly, in the past, it was not necessary to have this config file in the application package on the YARN containers. Was the `samza-hello-samza` build already set up to include the properties file into the application package?
This may also be something you would need to call out in the migration documentation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] kw2542 commented on a change in pull request
#79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
kw2542 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r399599247
##########
File path: README.md
##########
@@ -61,13 +61,19 @@ Package [samza.examples.cookbook](https://github.com/apache/samza-hello-samza/tr
Package [samza.examples.wikipedia.application](https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application) contains a small Samza application which consumes the real-time feeds from Wikipedia, extracts the metadata of the events, and calculates statistics of all edits in a 10-second window. You can start the app on the grid using the run-app.sh script:
```
-./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application.properties
+./deploy/samza/bin/run-app.sh \
+ --config app.class=samza.examples.wikipedia.application.WikipediaApplication \
+ --config yarn.package.path=file:///Users/kwu/workspace/hello-samza/target/hello-samza-1.5.0-SNAPSHOT-dist.tar.gz \
+ --config job.name=wikipedia-application \
+ --config job.factory.class=org.apache.samza.job.yarn.YarnJobFactory \
+ --config job.config.loader.factory=org.apache.samza.config.loaders.PropertiesConfigLoaderFactory \
+ --config job.config.loader.properties.path=$PWD/deploy/samza/config/wikipedia-application.properties
Review comment:
Yes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [samza-hello-samza] kw2542 commented on a change in pull request
#79: Update doc and javadoc from config factory to config loader
Posted by GitBox <gi...@apache.org>.
kw2542 commented on a change in pull request #79: Update doc and javadoc from config factory to config loader
URL: https://github.com/apache/samza-hello-samza/pull/79#discussion_r399599247
##########
File path: README.md
##########
@@ -61,13 +61,19 @@ Package [samza.examples.cookbook](https://github.com/apache/samza-hello-samza/tr
Package [samza.examples.wikipedia.application](https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application) contains a small Samza application which consumes the real-time feeds from Wikipedia, extracts the metadata of the events, and calculates statistics of all edits in a 10-second window. You can start the app on the grid using the run-app.sh script:
```
-./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application.properties
+./deploy/samza/bin/run-app.sh \
+ --config app.class=samza.examples.wikipedia.application.WikipediaApplication \
+ --config yarn.package.path=file:///Users/kwu/workspace/hello-samza/target/hello-samza-1.5.0-SNAPSHOT-dist.tar.gz \
+ --config job.name=wikipedia-application \
+ --config job.factory.class=org.apache.samza.job.yarn.YarnJobFactory \
+ --config job.config.loader.factory=org.apache.samza.config.loaders.PropertiesConfigLoaderFactory \
+ --config job.config.loader.properties.path=$PWD/deploy/samza/config/wikipedia-application.properties
Review comment:
For standalone applications, we do not need to but for Yarn ones, Yes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services