You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@wayang.apache.org by "JorgeQuiane (via GitHub)" <gi...@apache.org> on 2023/04/11 21:40:56 UTC
[GitHub] [incubator-wayang] JorgeQuiane opened a new issue, #282: Smooth-Installation
JorgeQuiane opened a new issue, #282:
URL: https://github.com/apache/incubator-wayang/issues/282
We need to improve the installation of wayang so that most people (including non-experts) can install and get jobs running on Wayang. For this we need to be able to:
- make sure the readme instructions are correct and correct them if necessary (https://github.com/apache/incubator-wayang)
- give a pointer to these instructions in the Wayang website (https://wayang.apache.org/documentation/)
- improve the installation process so that it gets super simple to do by non-experts
- test all the benchmarks (https://github.com/apache/incubator-wayang/tree/main/wayang-benchmark) and write the instructions to run such benchmarks (this will be like the test of the Wayang installation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] zkaoudi commented on issue #282: Smooth-Installation
Posted by "zkaoudi (via GitHub)" <gi...@apache.org>.
zkaoudi commented on issue #282:
URL: https://github.com/apache/incubator-wayang/issues/282#issuecomment-1517754784
I believe this issue has now been addressed. Thanks @bgeng777!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] bgeng777 commented on issue #282: Smooth-Installation
Posted by "bgeng777 (via GitHub)" <gi...@apache.org>.
bgeng777 commented on issue #282:
URL: https://github.com/apache/incubator-wayang/issues/282#issuecomment-1507855760
@zkaoudi You are right that the "meaningless" info(maybe jars under some classpath) is printed after I type command like `wayang-submit org.apache.wayang.apps.wordcount.Main java file://$(pwd)/README.md`. Also, I can see some output that alerts me that there are no jars/lib directories under WAYANG_HOME. I have not gave a deep look at the source code and am not so sure if they are relevant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] zkaoudi closed issue #282: Smooth-Installation
Posted by "zkaoudi (via GitHub)" <gi...@apache.org>.
zkaoudi closed issue #282: Smooth-Installation
URL: https://github.com/apache/incubator-wayang/issues/282
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] zkaoudi commented on issue #282: Smooth-Installation
Posted by "zkaoudi (via GitHub)" <gi...@apache.org>.
zkaoudi commented on issue #282:
URL: https://github.com/apache/incubator-wayang/issues/282#issuecomment-1506798119
Just to make things more concrete: We need to separate between two things:
1. Running Wayang after building it with maven.
- Improve instructions for building, if necessary
- Include scripts or instructions on how to execute an example and the benchmark tasks
2. Running Wayang after downloading the file from the downloads page. Current problems with this are:
- there is no wayang-submit script, in contrast to what is written in the README
- The file from the download page is a zip and the command indicated in the README assumes a .gz file
- The file linked in the download page is a source release. Is there somewhere a file with binaries only? If yes, we should point to that one. If not, we should create one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] bgeng777 commented on issue #282: Smooth-Installation
Posted by "bgeng777 (via GitHub)" <gi...@apache.org>.
bgeng777 commented on issue #282:
URL: https://github.com/apache/incubator-wayang/issues/282#issuecomment-1507208045
Hi there, thanks for creating this issue. I am new to wayang and when I tried to follow the current README to run the wordcount example, I met the problem that there is no wayang-submit script.
I finally managed to run the example with wayang-submit and the process is as follows:
#### 1. Get the wayang-submit binary
As @zkaoudi mentioned, there is no wayang-submit binary file in the downloaded zip file. To get the binary, I build the main branch of the repo with `./mvnw clean install -DskipTests ` and finally get the jar in the repo's `bin` folder:
#### 2. Copy the binary to extracted apache-wayang-0.6.0-incubating folder and relevant jars for the WordCount example
After copying wayang-submit to extracted apache-wayang-0.6.0-incubating folder, I tried to run command like `wayang-submit org.apache.wayang.apps.wordcount.Main java file://$(pwd)/README.md`, but I met many errors about class missing.
After some debugging, I make dir named 'jars' under WAYANG_HOME (it looks like Wayang will add jars under WAYANG_HOME/jars into the classpath) and copied following jars from the project repo into it(these jars should be built or downloaded by maven after building the project).
#### 3. Run `wayang-submit` with WordCount example
After step 1 and 2, I can run:
```
wayang-submit org.apache.wayang.apps.wordcount.Main java file://$(pwd)/README.md
```
and get the output:
So according to my experience, I believe there are at least 2 more things we may need to take care:
1. Add necessary jars into the released package
2. Avoid printing some meaningless info in the terminal like:
3. We may need to get rid of `aws-java-sdk-bundle-1.11.271.jar` dependency when running locally.
I hope my experience can help others who just want to give a try on the quickstart and I am willing to help to make this process more smooth and user-friendly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] zkaoudi commented on issue #282: Smooth-Installation
Posted by "zkaoudi (via GitHub)" <gi...@apache.org>.
zkaoudi commented on issue #282:
URL: https://github.com/apache/incubator-wayang/issues/282#issuecomment-1507511424
That is great. Just a question:
At which point you get all this meaningless info in the terminal? When running in the WordCount?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org