You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mrql.apache.org by Apache Wiki <wi...@apache.org> on 2016/03/03 18:05:44 UTC

[Mrql Wiki] Update of "GettingStarted" by LeonidasFegaras

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Mrql Wiki" for change notification.

The "GettingStarted" page has been changed by LeonidasFegaras:
https://wiki.apache.org/mrql/GettingStarted?action=diff&rev1=15&rev2=16

  
  Apache MRQL can run in 4 modes: in Map-Reduce mode using [[http://hadoop.apache.org/|Apache Hadoop]],
  in BSP mode (Bulk Synchronous Parallel mode) using [[http://hama.apache.org/|Apache Hama]],
- in Spark mode using [[http://spark.apache.org/|Apache Spark]], and in Flink mode using [[http://flink.apache.org|Apache Flink]] (only in MRQL-0.9.4-incubating).
+ in Spark mode using [[http://spark.apache.org/|Apache Spark]], and in Flink mode using [[http://flink.apache.org|Apache Flink]].
  
- The latest stable MRQL version, MRQL-0.9.4-incubating, is compatible with the following Apache releases: 
+ The latest stable MRQL version, MRQL-0.9.6-incubating, is compatible with the following Apache releases: 
  The MRQL !MapReduce mode is compatible with Apache Hadoop releases 1.x and 2.x (Yarn). 
  You can download the Hadoop tarball from [[http://hadoop.apache.org/releases.html|Apache Hadoop]].
- The BSP and Spark modes are optional. The BSP mode is compatible with Apache Hama 0.6.2, 0.6.3, and 0.6.4.
+ The BSP and Spark modes are optional. The BSP mode is compatible with Apache Hama 0.6.2, 0.6.3, 0.6.4, and 0.7.0.
  You can download the latest tarball from [[http://hama.apache.org/|Apache Hama]].
- The Spark mode is compatible with Apache Spark 1.0.0 through 1.2.0.
+ The Spark mode is compatible with Apache Spark 1.0.0 through 1.6.0.
  You can download the latest tarball prebuilt for Hadoop1 or Hadoop2 from [[http://spark.apache.org/|Apache Spark]].
- The Flink mode is compatible with Apache Flink 0.6-incubating through 0.8.0 in local and Yarn modes.
+ The Flink mode is compatible with Apache Flink 0.10.1 and 0.10.2 in local and Yarn modes.
  You can download the latest tarball prebuilt for Hadoop2 from [[http://flink.apache.org/downloads.html|Apache Flink]].
  
  The following instructions assume that you have already installed Hadoop !MapReduce
@@ -24, +24 @@

  === How to install MRQL ===
  
  Download the latest stable MRQL binary release from [[http://www.apache.org/dyn/closer.cgi/incubator/mrql]] and extract the files.
+ The MRQL 0.9.6 binary release uses Hadoop 2.7.1 (Yarn), Hama 0.7.0, Spark 1.6.0, and Flink 0.10.2
  The scripts {{{bin/mrql}}}, {{{bin/mrql.bsp}}}, {{{bin/mrql.spark}}}, and {{{bin/mrql.flink}}} evaluate MRQL queries in Hadoop, Hama, Spark, and Flink modes, respectively.
  
  === How to run MRQL on a Hadoop MapReduce cluster: ===
@@ -53, +54 @@

  
  === How to run MRQL in Flink mode on a Yarn cluster: ===
  
+ Change the configuration file {{{conf/mrql-env.sh}}} to match your Flink installation. Then run the [[Pagerank|PageRank example]] or the [[Kmeans|k-means clustering example]] using the {{{bin/mrql.flink}}} script.
- First, start the Flink application manager on Yarn using {{{${FLINK_HOME}/bin/yarn-session.sh -n #_of_nodes}}} (see [[http://flink.apache.org/docs/0.8/yarn_setup.html|Yarn Setup]]).
- The run the [[Pagerank|PageRank example]] or the [[Kmeans|k-means clustering example]] using the {{{bin/mrql.flink}}} script.
  
  === How to Recompile MRQL ===
  
@@ -79, +79 @@

  
  which runs the queries in {{{tests/queries}}} in memory, local Hadoop mode, local Hama mode, local Spark mode, and local Flink mode.
  
- Currently, the "mvn install" builds MRQL with Hadoop 1.2.1, Hama 0.6.4, Spark 1.2.0, and Flink 0.8.0. 
+ Currently, the "mvn install" in MRQL 0.9.6 builds MRQL using Hadoop 2.7.1 (Yarn), Hama 0.7.0, Spark 1.6.0, and Flink 0.10.2. 
- To build MRQL on some other Hadoop 1.x, such as 1.0.3, use: 
+ To build MRQL on Hadoop 1.x, such as 1.0.3, use: 
  
  {{{
- mvn -Dhadoop.version=1.0.3 clean install 
+ mvn -Dhadoop1 -Dhadoop.version=1.0.3 clean install 
  }}}
  
- To build MRQL on Hadoop 2.x (yarn) use: 
+ To build MRQL on another Hadoop 2.x (yarn) use: 
  
  {{{
- mvn -Dyarn -Dyarn.version=2.2.0 -Dhadoop.version=1.2.1 clean install 
+ mvn -Dyarn.version=2.2.0 clean install 
  }}}