You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@crunch.apache.org by bu...@apache.org on 2014/01/14 22:40:53 UTC
svn commit: r894173 - in /websites/staging/crunch/trunk/content: ./
user-guide.html
Author: buildbot
Date: Tue Jan 14 21:40:52 2014
New Revision: 894173
Log:
Staging update by buildbot for crunch
Modified:
websites/staging/crunch/trunk/content/ (props changed)
websites/staging/crunch/trunk/content/user-guide.html
Propchange: websites/staging/crunch/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Tue Jan 14 21:40:52 2014
@@ -1 +1 @@
-1558156
+1558208
Modified: websites/staging/crunch/trunk/content/user-guide.html
==============================================================================
--- websites/staging/crunch/trunk/content/user-guide.html (original)
+++ websites/staging/crunch/trunk/content/user-guide.html Tue Jan 14 21:40:52 2014
@@ -582,42 +582,52 @@ can be used to kick off a shuffle on the
<p>Let's look at some more example PTypes created using the common primitive and collection types. For most of your pipelines,
you will use one type family exclusively, and so you can cut down on some of the boilerplate in your classes by importing
all of the methods from the <code>Writables</code> or <code>Avros</code> classes into your class:</p>
-<p><pre
+<pre>
// Import all of the PType factory methods from Avros
-import static org.apache.crunch.types.avro.Avros.*;</p>
-<p>import org.apache.crunch.Pair;
+import static org.apache.crunch.types.avro.Avros.*;
+
+import org.apache.crunch.Pair;
import org.apache.crunch.Tuple3;
-import org.apache.crunch.TupleN;</p>
-<p>import java.nio.ByteBuffer;
+import org.apache.crunch.TupleN;
+
+import java.nio.ByteBuffer;
import java.util.Collection;
-import java.util.Map;</p>
-<p>public class MyPipeline {</p>
-<p>// Common primitive types
+import java.util.Map;
+
+public class MyPipeline {
+
+ // Common primitive types
PType<Integer> intType = ints();
PType<Long> longType = longs();
PType<Double> doubleType = doubles();
// Bytes are represented by java.nio.ByteBuffer
- PType<ByteBuffer> bytesType = bytes();</p>
-<p>// A PTableType: using tableOf will return a PTable instead of a
+ PType<ByteBuffer> bytesType = bytes();
+
+ // A PTableType: using tableOf will return a PTable instead of a
// PCollection from a parallelDo call.
- PTableType<String, Boolean> tableType = tableOf(strings(), booleans());</p>
-<p>// Pair types:
- PType<Pair<String, Boolean>> pairType = pairs(strings(), booleans());
- PType<Pair<String, Pair<Long, Long>> nestedPairType = pairs(strings(), pairs(longs(), longs()));</p>
-<p>// A triple
- PType<Tuple3<Long, Float, Float>> tripType = trips(longs(), floats(), floats());
+ PTableType<String, Boolean> tableType = tableOf(strings(), booleans());
+
+ // Pair types:
+ PType<Pair<String, Boolean>> pairType = pairs(strings(), booleans());
+ PType<Pair<String, Pair<Long, Long>> nestedPairType = pairs(strings(), pairs(longs(), longs()));
+
+ // A triple
+ PType<Tuple3<Long, Float, Float>> tripType = trips(longs(), floats(), floats());
// An arbitrary length tuple-- note that we lose the generic type information
- PType<TupleN> tupleType = tupleN(ints(), ints(), floats(), strings(), strings(), ints());</p>
-<p>// A Collection type
- PType<Collection<Long>> longsType = collections(longs());
+ PType<TupleN> tupleType = tupleN(ints(), ints(), floats(), strings(), strings(), ints());
+
+ // A Collection type
+ PType<Collection<Long>> longsType = collections(longs());
// A Map Type-- note that the keys are always strings, we only specify the value.
- PType<Map<String, Boolean>> mapType = maps(booleans());</p>
-<p>// A Pair of collections
- PType<Pair<Collection<String>, Collection<Long>>> pairColType = pairs(
+ PType<Map<String, Boolean>> mapType = maps(booleans());
+
+ // A Pair of collections
+ PType<Pair<Collection<String>, Collection<Long>>> pairColType = pairs(
collections(strings()),
collections(longs()));
}
-</pre></p>
+</pre>
+
<p>Both type families also have a method named <code>PType<T> records(Class<T> clazz)</code> that can be used to create PTypes that support the common
record format for each type family. For the WritableTypeFamily, the records method supports PTypes for implementations of the <code>Writable</code>
interface, and for the AvroTypeFamily, the records method supports PTypes for implementations of Avro's <code>IndexedRecord</code> interface, which