You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pig.apache.org by ol...@apache.org on 2009/03/17 02:20:31 UTC
svn commit: r755066 - in /hadoop/pig/trunk: CHANGES.txt build.xml docs/
src/overview.html
Author: olga
Date: Tue Mar 17 01:20:31 2009
New Revision: 755066
URL: http://svn.apache.org/viewvc?rev=755066&view=rev
Log:
PIG-720: further doc cleanup (gkesavan via olgan)
Added:
hadoop/pig/trunk/src/overview.html
Removed:
hadoop/pig/trunk/docs/
Modified:
hadoop/pig/trunk/CHANGES.txt
hadoop/pig/trunk/build.xml
Modified: hadoop/pig/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/pig/trunk/CHANGES.txt?rev=755066&r1=755065&r2=755066&view=diff
==============================================================================
--- hadoop/pig/trunk/CHANGES.txt (original)
+++ hadoop/pig/trunk/CHANGES.txt Tue Mar 17 01:20:31 2009
@@ -390,6 +390,8 @@
PIG-718: To add standard ant targets to build.xml file (gkesavan via olgan)
+ PIG-720: further doc cleanup (gkesavan via olgan)
+
Release 0.1.1 - 2008-12-04
INCOMPATIBLE CHANGES
Modified: hadoop/pig/trunk/build.xml
URL: http://svn.apache.org/viewvc/hadoop/pig/trunk/build.xml?rev=755066&r1=755065&r2=755066&view=diff
==============================================================================
--- hadoop/pig/trunk/build.xml (original)
+++ hadoop/pig/trunk/build.xml Tue Mar 17 01:20:31 2009
@@ -36,7 +36,6 @@
<property name="src.dir" value="${basedir}/src/" />
<property name="src.lib.dir" value="${basedir}/lib-src/" />
<property name="src.gen.dir" value="${basedir}/src-gen/" />
- <property name="src.docs.dir" value="${basedir}/docs/" />
<property name="docs.dir" value="${basedir}/src/docs/" />
@@ -263,7 +262,7 @@
<!-- ================================================================== -->
<target name="javadoc" depends="jar" description="Create documentation">
<mkdir dir="${build.javadoc}" />
- <javadoc overview="${src.docs.dir}/overview.html" packagenames="org.apache.pig.*" destdir="${build.javadoc}" author="true" version="true" use="true" windowtitle="${Name} ${version} API" doctitle="${Name} ${version} API" bottom="Copyright &copy; ${year} The Apache Software Foundation">
+ <javadoc overview="${src.dir}/overview.html" packagenames="org.apache.pig.*" destdir="${build.javadoc}" author="true" version="true" use="true" windowtitle="${Name} ${version} API" doctitle="${Name} ${version} API" bottom="Copyright &copy; ${year} The Apache Software Foundation">
<packageset dir="${src.dir}" />
<link href="${javadoc.link.java}" />
<classpath>
@@ -434,7 +433,7 @@
<!-- ================================================================== -->
<!-- D I S T R I B U T I O N -->
<!-- ================================================================== -->
- <target name="package" depends="jar, javadoc" description="Create a Pig release">
+ <target name="package" depends="docs" description="Create a Pig release">
<mkdir dir="${dist.dir}" />
<mkdir dir="${dist.dir}/lib" />
<mkdir dir="${dist.dir}/scripts" />
Added: hadoop/pig/trunk/src/overview.html
URL: http://svn.apache.org/viewvc/hadoop/pig/trunk/src/overview.html?rev=755066&view=auto
==============================================================================
--- hadoop/pig/trunk/src/overview.html (added)
+++ hadoop/pig/trunk/src/overview.html Tue Mar 17 01:20:31 2009
@@ -0,0 +1,45 @@
+<HTML>
+<BODY>
+Pig is a platform for a data flow programming on large data sets in a parallel
+environment. It consists of a language to specify these programs,
+<a href="http://wiki.apache.org/pig/PigLatin">Pig Latin</a>,
+a compiler for this language, and an execution engine to execute the programs.
+<p>
+Pig currently runs on the <a href="http://hadoop.apache.org/core/">hadoop</a>
+platform, reading data from and writing data to hdfs, and doing processing via
+one or more map-reduce jobs.
+
+<h2> Design </h2>
+This section gives a very high overview of the design of the Pig system.
+Throughout the documents you can see design for that package or class by
+looking for the Design heading in the documentation.
+
+<h3> Overview </h3>
+<p>
+Pig's design is guided by our <a href="http://incubator.apache.org/pig/philosophy.html">
+pig philosophy</a> and by our experience with similar data processing
+systems.
+<p>
+Pig shares many similarities with a traditional RDBMS design. It has a parser,
+type checker, optimizer, and operators that perform the data processing. However,
+there are some
+significant differences. Pig does not have a data catalog, there are no
+transactions, pig does not directly manage data storage, nor does it implement the
+execution framework.
+<p>
+<h3> High Level Architecture </h3>
+Pig is split between the front and back ends of the engine. The front end handles
+parsing, checking, and doing initial optimization on a Pig Latin script. The
+result is a {@link org.apache.pig.impl.logicalLayer.LogicalPlan} that defines how
+the script will be executed.
+<p>
+Once a LogicalPlan has been generated, the backend of Pig handles executing the
+script. Pig supports multiple different
+backend implementations, in order to allow Pig to run on different systems.
+Currently pig comes with two backends, Map-Reduce and local. For a given run,
+pig selects the backend to use via configuration.
+
+
+</BODY>
+</HTML>
+