You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pig.apache.org by Apache Wiki <wi...@apache.org> on 2008/05/31 00:03:20 UTC
[Pig Wiki] Trivial Update of "FrontPage" by CorinneC
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.
The following page has been changed by CorinneC:
http://wiki.apache.org/pig/FrontPage
------------------------------------------------------------------------------
= Pig Wiki =
- [http://incubator.apache.org/pig/ Pig] is a dataflow programming environment for processing very large files. Pig's language is called Pig Latin. A Pig Latin program consists of a directed acyclic graph where each node represents an operation that transforms data. Operations are of two flavors: (1) relational-algebra style operations such as join, filter, project; (2) functional-programming style operators such as map, reduce.
+ Pig is a dataflow programming environment for processing very large files. Pig's language is called Pig Latin. A Pig Latin program consists of a directed acyclic graph where each node represents an operation that transforms data. Operations are of two flavors: (1) relational-algebra style operations such as join, filter, project; (2) functional-programming style operators such as map, reduce.
Pig compiles these dataflow programs into (sequences of) map-reduce jobs and executes them using Hadoop. It is also possible to execute Pig Latin programs in a "local" mode (without Hadoop cluster), in which case all processing takes place in a single local JVM.
- == NEWS ==
+ == News! ==
- We are undertaking a complete redesign of the execution and optimization framework, including:
+ We are completely redesigning the execution and optimization framework. This work includes:
- 1. new (hopefully cleaner) operator representations at the various layers (logical, physical, map-reduce) to facilitate optimization along the lines of PigOptimizationWishList, and
+
+ 1. Creating new operator representations at the various layers (logical, physical, map-reduce) to facilitate optimization.
- 2. streamlining the execution pipeline.
+ 2. Streamlining the execution pipeline.
- See PigExecutionModel for the design details; implementation of this design is underway.
+ See PigOptimizationWishList and PigExecutionModel for the design details. Implementation is already underway ...
== General Information ==
- * [http://incubator.apache.org/pig/ official Pig project website]
+ * [http://incubator.apache.org/pig/ Official Apache Pig Website]
* PigOverview - an overview of Pig's capabilities
* Pig talks:
* 60-minute talk given at universities in early 2008: [http://coursecast.panopto.com/CourseCast/Viewer/Default.aspx?id=238 video]; [http://www.cs.cmu.edu/~olston/pig.ppt slides]
@@ -33, +34 @@
== User Documentation ==
* [http://incubator.apache.org/pig/javadoc/doc/ Javadoc]
- * IsPigRightForYou ?
+ * IsPigRightForYou? - Find out!
- * GettingStarted
- * PigTutorial
+ * PigTutorial - Get a quick start here ...
+ * GettingStarted - Build your own Pig JAR files
- * PigLatin - the language manual
+ * PigLatin - The language manual
- * PigFunctions - Guide for writing your own Pig functions
+ * PigFunctions - A guide for writing your own Pig functions
- * [http://wiki.apache.org/pig/Grunt Grunt] - shell manual
+ * [http://wiki.apache.org/pig/Grunt Grunt] - The shell manual
- * [http://wiki.apache.org/pig/ExampleGenerator ExampleGenerator] - Guide for using the ILLUSTRATE command to help debug scripts
+ * [http://wiki.apache.org/pig/ExampleGenerator ExampleGenerator] - A guide for using the ILLUSTRATE command to help debug scripts
== Developer Documentation ==
* How tos