You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by cd...@apache.org on 2008/10/20 21:00:41 UTC
svn commit: r706368 [9/12] - in /hadoop/core/branches/branch-0.19: ./ docs/
src/docs/src/documentation/content/xdocs/
Modified: hadoop/core/branches/branch-0.19/docs/index.html
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.19/docs/index.html?rev=706368&r1=706367&r2=706368&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.19/docs/index.html (original)
+++ hadoop/core/branches/branch-0.19/docs/index.html Mon Oct 20 12:00:39 2008
@@ -5,7 +5,7 @@
<meta content="Apache Forrest" name="Generator">
<meta name="Forrest-version" content="0.8">
<meta name="Forrest-skin-name" content="pelt">
-<title>Hadoop Documentation</title>
+<title>Overview</title>
<link type="text/css" href="skin/basic.css" rel="stylesheet">
<link media="screen" type="text/css" href="skin/screen.css" rel="stylesheet">
<link media="print" type="text/css" href="skin/print.css" rel="stylesheet">
@@ -108,49 +108,55 @@
<div class="menupagetitle">Overview</div>
</div>
<div class="menuitem">
-<a href="quickstart.html">Quickstart</a>
+<a href="quickstart.html">Hadoop Quick Start</a>
</div>
<div class="menuitem">
-<a href="cluster_setup.html">Cluster Setup</a>
+<a href="cluster_setup.html">Hadoop Cluster Setup</a>
</div>
<div class="menuitem">
-<a href="hdfs_design.html">HDFS Architecture</a>
+<a href="mapred_tutorial.html">Hadoop Map/Reduce Tutorial</a>
</div>
<div class="menuitem">
-<a href="hdfs_user_guide.html">HDFS User Guide</a>
+<a href="commands_manual.html">Hadoop Command Guide</a>
+</div>
+<div class="menuitem">
+<a href="hdfs_shell.html">Hadoop FS Shell Guide</a>
</div>
<div class="menuitem">
-<a href="hdfs_permissions_guide.html">HDFS Permissions Guide</a>
+<a href="distcp.html">Hadoop DistCp Guide</a>
</div>
<div class="menuitem">
-<a href="hdfs_quota_admin_guide.html">HDFS Quotas Administrator Guide</a>
+<a href="native_libraries.html">Hadoop Native Libraries</a>
</div>
<div class="menuitem">
-<a href="commands_manual.html">Commands Manual</a>
+<a href="streaming.html">Hadoop Streaming</a>
</div>
<div class="menuitem">
-<a href="hdfs_shell.html">FS Shell Guide</a>
+<a href="hadoop_archives.html">Hadoop Archives</a>
</div>
<div class="menuitem">
-<a href="SLG_user_guide.html">Synthetic Load Generator User Guide</a>
+<a href="hdfs_user_guide.html">HDFS User Guide</a>
</div>
<div class="menuitem">
-<a href="distcp.html">DistCp Guide</a>
+<a href="hdfs_design.html">HDFS Architecture</a>
</div>
<div class="menuitem">
-<a href="mapred_tutorial.html">Map-Reduce Tutorial</a>
+<a href="hdfs_permissions_guide.html">HDFS Admin Guide: Permissions</a>
</div>
<div class="menuitem">
-<a href="native_libraries.html">Native Hadoop Libraries</a>
+<a href="hdfs_quota_admin_guide.html">HDFS Admin Guide: Quotas</a>
</div>
<div class="menuitem">
-<a href="streaming.html">Streaming</a>
+<a href="SLG_user_guide.html">HDFS Utilities</a>
</div>
<div class="menuitem">
-<a href="hadoop_archives.html">Hadoop Archives</a>
+<a href="hod_user_guide.html">HOD User Guide</a>
</div>
<div class="menuitem">
-<a href="hod.html">Hadoop On Demand</a>
+<a href="hod_admin_guide.html">HOD Admin Guide</a>
+</div>
+<div class="menuitem">
+<a href="hod_config_guide.html">HOD Config Guide</a>
</div>
<div class="menuitem">
<a href="capacity_scheduler.html">Capacity Scheduler</a>
@@ -168,13 +174,10 @@
<a href="http://wiki.apache.org/hadoop/FAQ">FAQ</a>
</div>
<div class="menuitem">
-<a href="http://hadoop.apache.org/core/mailing_lists.html">Mailing Lists</a>
-</div>
-<div class="menuitem">
<a href="releasenotes.html">Release Notes</a>
</div>
<div class="menuitem">
-<a href="changes.html">All Changes</a>
+<a href="changes.html">Change Log</a>
</div>
</div>
<div id="credit">
@@ -199,49 +202,17 @@
<a class="dida" href="index.pdf"><img alt="PDF -icon" src="skin/images/pdfdoc.gif" class="skin"><br>
PDF</a>
</div>
-<h1>Hadoop Documentation</h1>
-
+<h1>Overview</h1>
+
+<p>
+ The Hadoop Documentation provides the information you need to get started using Hadoop, the Hadoop Distributed File System (HDFS), and Hadoop on Demand (HOD).
+ </p>
<p>
- The following documents provide concepts and procedures that will help you
- get started using Hadoop. If you have more questions, you can ask the
- <a href="http://hadoop.apache.org/core/mailing_lists.html">mailing list</a> or browse the archives.
+Begin with the <a href="quickstart.html">Hadoop Quick Start</a> which shows you how to set up a single-node Hadoop installation. Then move on to the <a href="cluster_setup.html">Hadoop Cluster Setup</a> to learn how to set up a multi-node Hadoop installation. Once your Hadoop installation is in place, try out the <a href="mapred_tutorial.html">Hadoop Map/Reduce Tutorial</a>.
+ </p>
+<p>
+If you have more questions, you can ask on the <a href="http://hadoop.apache.org/core/mailing_lists.html">Hadoop Core Mailing Lists</a> or browse the <a href="http://mail-archives.apache.org/mod_mbox/hadoop-core-commits/">Mailing List Archives</a>.
</p>
-
-<ul>
-
-<li>
-<a href="quickstart.html">Hadoop Quickstart</a>
-</li>
-
-<li>
-<a href="cluster_setup.html">Hadoop Cluster Setup</a>
-</li>
-
-<li>
-<a href="hdfs_design.html">Hadoop Distributed File System</a>
-</li>
-
-<li>
-<a href="mapred_tutorial.html">Hadoop Map-Reduce Tutorial</a>
-</li>
-
-<li>
-<a href="native_libraries.html">Native Hadoop Libraries</a>
-</li>
-
-<li>
-<a href="api/index.html">API Docs</a>
-</li>
-
-<li>
-<a href="http://wiki.apache.org/hadoop/">Wiki</a>
-</li>
-
-<li>
-<a href="http://wiki.apache.org/hadoop/FAQ">FAQ</a>
-</li>
-
-</ul>
</div>
<!--+
Modified: hadoop/core/branches/branch-0.19/docs/index.pdf
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.19/docs/index.pdf?rev=706368&r1=706367&r2=706368&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.19/docs/index.pdf (original)
+++ hadoop/core/branches/branch-0.19/docs/index.pdf Mon Oct 20 12:00:39 2008
@@ -5,10 +5,10 @@
/Producer (FOP 0.20.5) >>
endobj
5 0 obj
-<< /Length 1271 /Filter [ /ASCII85Decode /FlateDecode ]
+<< /Length 1307 /Filter [ /ASCII85Decode /FlateDecode ]
>>
stream
-Gat=,9lo&I&A@[:i98BA#RCJSG]]B88Z1soZ"A;qHCu]Xig:>nIRrLK++R^#Os3>1I'@dJIJ^6p==j8fja;=B&7q6%e]&cp?fVd*F^.?,T8Nd2'9$<p8@7*Gs6dUSlSLc=6gq!-olkVPd\ts%6P1)RH?b5E8#nj3-q>nl6$l/XpTYjB7U#2DJ`5aL]1n_>krCqDX_Z`8kLOe"dSmd_@8V?,e.F;>&*r*K8?L'/JHa"F/?EuSPJgWdPoQq!R[2OW.M.j]@90=LZebTr"iIFt@[Zt6<7_Uaa4JR([U"+Z]A0;%Y^Vlf2oX'lRg>%k"/n`h!r&i9c@;pcT2&*"IK,6GcPJS!6IYZ_eb&Tl\K2McG+JgK/@ghekfJ*l5-EW'Cl>k3=;4SbI2F>3)(m6WVD3qHI,Zl6(d`I"p@P7SLmG7S"b`(!0,h:g/23,BTO;W*g$1b6krDh3Wng_^G:QQkI$d@7pnVrc*?B@`k]e02ZAJED.A*Q:94YH!@Teu5AK1pUm6%.A^r(KW-Z/BqO(^P_/`:[K5sE1Oc8-;)a<<sE4aQ^`&09LX'_$N%j63=#Z58B\l?8]_DR[:*DGYI4;!>o?A"e$gq6iTl#'-<OA>(T.e>H/;,E1mrl2I9=H(hMm^3s[Y9?cN:",K,@eZC&%QEY3M+k\5CPfH4=+?jt8l[akl3<23!Sf@>f^l>lu%tK5$B/$<?%Y><S%l"\"jSMM-dO]i"m8Z0j35d`nYEWl41#+_<"-!lY3b]N?83b&7e0R*X2Q?WM<+?2`#bg]8e#KRWm6Uj.VcS<-p.KMO!jCm_bB.Q$e"2S(cYY^T!Oss&knl1;o*hL<Ca#`ME8]&-%!C@cqfd:4s'<s5dA/H#n4dhi9/q[qL(KkaiJHljD(+>T.Z6-A_dnld,f0J#C$=*CraEI4/lq#C;&U(k6iZB5Lso=kL\ZlckWFL&2!>K;Ikp1r'#I89%0_.BV*':2n(66'QYt&3KP:O#%)'r
q$O(qXMP%4/Mp2@O!aQ*3%5T@FU)hho7?;V`)g*="S`0"OI@Wto87lC(k0oB67>;ADe\Cbk6q=Wm"_@9^nZr#V/B^OX`k?P9ngL_\XKKk)%7C080C^7'b^IA[`#*29Od^6!hagl4[[6Xb@mBY;W<g*3DY1KrS1t;H*+]Iog+63QhnJB;e7TcfcfP#W&KL%IX/XY@CjLKRipEnQD``DEmTPM],-BD.c_uj#@i.N/.GY^$0E[o%ZokFj[jDrFeSc[IUWT,0>NI=rWAV+grrlha_$^~>
+Gatm;8T3?C'Y`m7n<a2),>k8&A(/rp%&O;6qprtWQ"c_d(712#jPLKUnqgQ0_]X`%1r&.oh]o+uSb9L4&Y[iH]7X\Hs'j/@8'=3,qK#J8c["$[Xr!a:MLOEq>e%EPI_A8OU>=]6c]CLtr!:5g++#n_6+[fF7ir#;^UX.ZmIU7Wa3T$9V*$9';WO7;50NjJp:Sa.-"IV@+d]0ZH-7^$a>i&ZE3;DIH)V$;)9EMoX9t(19tLpA^#L"oMJ6F-\QYaV8h#3h,GC)f!+bt-PW[LVf=S64S*fP?K:[t+rNZQR%aR6XX]8UZc`d(f[Ku/-dMMGlNrnfU7<pY[Im`^NBDo)=%eSchZ5h30Sk99`.LM%`7DsB!bV5/%)[FSW=1#BL>>MjP/<=I`UIg!0^<0S]6_mbZX/])"C(Rm-%YJ/WTf5%UJ_0@ULjq'\22H#sDd@*MiL(B2`#`-^GAoW/J_b/`r6?q]H/asH'eI(j6k#edSFUI#d!+cUq6he]gJoB-&QkfS0B""YHp?(S1Z<KDVXha%aAaf?8[BdPNE2MZLif(mpjCOUX34?fXNheSN!)+RItaQm'tTf;Z9GKXs6EfH<0V8keo`>S"s?>enZbaN1<&1\LI/bJP"KRoEl#!%Xje&0<Gb'2!#,V.S$m[imsp(?\%Jff1P'a5]-2lD@2KUuam=N"G?Q;hki+`=B?+fQ)T>"7GdHpha^RI!7(I\75RK2L^_Pc1:84,.j/8,[TQ@.(Q?JtOa.Z:.AZWEPUkf3`LNW7$R9SB6ZckP(=l&dkg8$a$;rjm+F3-,@R%"TJlDr+;kkt4.n#L^'ll/mT>'*b[8jEgeeSJYZhfh.)J_sa5k#+Uh9[5LK=?h-[TE3FVS%fG+)gqjudPR:uoV^\;S[.6`!?=TPBi4DI@8C2<QKnZ'7`q^Z*)cI\T[#MdL7)7d23<?/=(>g!e?\&Em$&\N.0=Y=#0RF-gAqWIIVKr$]\,#X/`a#@9aSHe!EH<
16m_&FX$%(Tr>(6c\QR6s'F&\3k-'\;.=@=GV8sl?psi2Kp`FWm04t':1:L`sMfZ,irfZPi`,A4egML9D?0E9VId1Q`a`'`^lFYdHK/PTOWp?ae:+-O$U7gdpid8i__]CX"J`6Gk0=`rrlg9t=925.^@T%mE#RYKi4$\AOO"rnU$_L(H-3if+\?ocCUH>oFIF]ZC@521]Ecc!V+#cT9hnPB\`2=$$Yke?2.!2`bghd,U2^scbk",2uk*8_J1%g@TG+_@]lZbMdG`H5mhWBUa)G)WhafB6]Y8nW78bA\'0#3hj>[I1HPl;aLJs3/~>
endstream
endobj
6 0 obj
@@ -28,18 +28,15 @@
11 0 R
12 0 R
13 0 R
-14 0 R
-15 0 R
-16 0 R
]
endobj
8 0 obj
<< /Type /Annot
/Subtype /Link
-/Rect [ 376.956 572.6 431.304 560.6 ]
+/Rect [ 163.668 551.4 259.656 539.4 ]
/C [ 0 0 0 ]
/Border [ 0 0 0 ]
-/A << /URI (http://hadoop.apache.org/core/mailing_lists.html)
+/A << /URI (quickstart.html)
/S /URI >>
/H /I
>>
@@ -47,10 +44,10 @@
9 0 obj
<< /Type /Annot
/Subtype /Link
-/Rect [ 108.0 542.2 198.984 530.2 ]
+/Rect [ 251.664 538.2 357.66 526.2 ]
/C [ 0 0 0 ]
/Border [ 0 0 0 ]
-/A << /URI (quickstart.html)
+/A << /URI (cluster_setup.html)
/S /URI >>
/H /I
>>
@@ -58,10 +55,10 @@
10 0 obj
<< /Type /Annot
/Subtype /Link
-/Rect [ 108.0 529.0 213.996 517.0 ]
+/Rect [ 447.972 525.0 485.964 513.0 ]
/C [ 0 0 0 ]
/Border [ 0 0 0 ]
-/A << /URI (cluster_setup.html)
+/A << /URI (mapred_tutorial.html)
/S /URI >>
/H /I
>>
@@ -69,10 +66,10 @@
11 0 obj
<< /Type /Annot
/Subtype /Link
-/Rect [ 108.0 515.8 263.004 503.8 ]
+/Rect [ 90.0 511.8 192.984 499.8 ]
/C [ 0 0 0 ]
/Border [ 0 0 0 ]
-/A << /URI (hdfs_design.html)
+/A << /URI (mapred_tutorial.html)
/S /URI >>
/H /I
>>
@@ -80,10 +77,10 @@
12 0 obj
<< /Type /Annot
/Subtype /Link
-/Rect [ 108.0 502.6 252.636 490.6 ]
+/Rect [ 318.96 490.6 450.624 478.6 ]
/C [ 0 0 0 ]
/Border [ 0 0 0 ]
-/A << /URI (mapred_tutorial.html)
+/A << /URI (http://hadoop.apache.org/core/mailing_lists.html)
/S /URI >>
/H /I
>>
@@ -91,69 +88,36 @@
13 0 obj
<< /Type /Annot
/Subtype /Link
-/Rect [ 108.0 489.4 227.304 477.4 ]
+/Rect [ 90.0 477.4 195.996 465.4 ]
/C [ 0 0 0 ]
/Border [ 0 0 0 ]
-/A << /URI (native_libraries.html)
+/A << /URI (http://mail-archives.apache.org/mod_mbox/hadoop-core-commits/)
/S /URI >>
/H /I
>>
endobj
14 0 obj
-<< /Type /Annot
-/Subtype /Link
-/Rect [ 108.0 476.2 154.992 464.2 ]
-/C [ 0 0 0 ]
-/Border [ 0 0 0 ]
-/A << /URI (api/index.html)
-/S /URI >>
-/H /I
->>
-endobj
-15 0 obj
-<< /Type /Annot
-/Subtype /Link
-/Rect [ 108.0 463.0 132.0 451.0 ]
-/C [ 0 0 0 ]
-/Border [ 0 0 0 ]
-/A << /URI (http://wiki.apache.org/hadoop/)
-/S /URI >>
-/H /I
->>
-endobj
-16 0 obj
-<< /Type /Annot
-/Subtype /Link
-/Rect [ 108.0 449.8 132.0 437.8 ]
-/C [ 0 0 0 ]
-/Border [ 0 0 0 ]
-/A << /URI (http://wiki.apache.org/hadoop/FAQ)
-/S /URI >>
-/H /I
->>
-endobj
-17 0 obj
<< /Type /Font
/Subtype /Type1
/Name /F3
/BaseFont /Helvetica-Bold
/Encoding /WinAnsiEncoding >>
endobj
-18 0 obj
+15 0 obj
<< /Type /Font
/Subtype /Type1
/Name /F5
/BaseFont /Times-Roman
/Encoding /WinAnsiEncoding >>
endobj
-19 0 obj
+16 0 obj
<< /Type /Font
/Subtype /Type1
/Name /F1
/BaseFont /Helvetica
/Encoding /WinAnsiEncoding >>
endobj
-20 0 obj
+17 0 obj
<< /Type /Font
/Subtype /Type1
/Name /F2
@@ -172,38 +136,35 @@
endobj
3 0 obj
<<
-/Font << /F3 17 0 R /F5 18 0 R /F1 19 0 R /F2 20 0 R >>
+/Font << /F3 14 0 R /F5 15 0 R /F1 16 0 R /F2 17 0 R >>
/ProcSet [ /PDF /ImageC /Text ] >>
endobj
xref
-0 21
+0 18
0000000000 65535 f
-0000003624 00000 n
-0000003682 00000 n
-0000003732 00000 n
+0000003168 00000 n
+0000003226 00000 n
+0000003276 00000 n
0000000015 00000 n
0000000071 00000 n
-0000001434 00000 n
-0000001554 00000 n
-0000001634 00000 n
-0000001831 00000 n
-0000001993 00000 n
-0000002159 00000 n
-0000002323 00000 n
-0000002491 00000 n
-0000002660 00000 n
-0000002822 00000 n
-0000002998 00000 n
-0000003177 00000 n
-0000003290 00000 n
-0000003400 00000 n
-0000003508 00000 n
+0000001470 00000 n
+0000001590 00000 n
+0000001649 00000 n
+0000001813 00000 n
+0000001979 00000 n
+0000002149 00000 n
+0000002316 00000 n
+0000002513 00000 n
+0000002721 00000 n
+0000002834 00000 n
+0000002944 00000 n
+0000003052 00000 n
trailer
<<
-/Size 21
+/Size 18
/Root 2 0 R
/Info 4 0 R
>>
startxref
-3844
+3388
%%EOF
Modified: hadoop/core/branches/branch-0.19/docs/linkmap.html
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.19/docs/linkmap.html?rev=706368&r1=706367&r2=706368&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.19/docs/linkmap.html (original)
+++ hadoop/core/branches/branch-0.19/docs/linkmap.html Mon Oct 20 12:00:39 2008
@@ -108,49 +108,55 @@
<a href="index.html">Overview</a>
</div>
<div class="menuitem">
-<a href="quickstart.html">Quickstart</a>
+<a href="quickstart.html">Hadoop Quick Start</a>
</div>
<div class="menuitem">
-<a href="cluster_setup.html">Cluster Setup</a>
+<a href="cluster_setup.html">Hadoop Cluster Setup</a>
</div>
<div class="menuitem">
-<a href="hdfs_design.html">HDFS Architecture</a>
+<a href="mapred_tutorial.html">Hadoop Map/Reduce Tutorial</a>
</div>
<div class="menuitem">
-<a href="hdfs_user_guide.html">HDFS User Guide</a>
+<a href="commands_manual.html">Hadoop Command Guide</a>
</div>
<div class="menuitem">
-<a href="hdfs_permissions_guide.html">HDFS Permissions Guide</a>
+<a href="hdfs_shell.html">Hadoop FS Shell Guide</a>
</div>
<div class="menuitem">
-<a href="hdfs_quota_admin_guide.html">HDFS Quotas Administrator Guide</a>
+<a href="distcp.html">Hadoop DistCp Guide</a>
</div>
<div class="menuitem">
-<a href="commands_manual.html">Commands Manual</a>
+<a href="native_libraries.html">Hadoop Native Libraries</a>
</div>
<div class="menuitem">
-<a href="hdfs_shell.html">FS Shell Guide</a>
+<a href="streaming.html">Hadoop Streaming</a>
</div>
<div class="menuitem">
-<a href="SLG_user_guide.html">Synthetic Load Generator User Guide</a>
+<a href="hadoop_archives.html">Hadoop Archives</a>
</div>
<div class="menuitem">
-<a href="distcp.html">DistCp Guide</a>
+<a href="hdfs_user_guide.html">HDFS User Guide</a>
</div>
<div class="menuitem">
-<a href="mapred_tutorial.html">Map-Reduce Tutorial</a>
+<a href="hdfs_design.html">HDFS Architecture</a>
</div>
<div class="menuitem">
-<a href="native_libraries.html">Native Hadoop Libraries</a>
+<a href="hdfs_permissions_guide.html">HDFS Admin Guide: Permissions</a>
</div>
<div class="menuitem">
-<a href="streaming.html">Streaming</a>
+<a href="hdfs_quota_admin_guide.html">HDFS Admin Guide: Quotas</a>
</div>
<div class="menuitem">
-<a href="hadoop_archives.html">Hadoop Archives</a>
+<a href="SLG_user_guide.html">HDFS Utilities</a>
+</div>
+<div class="menuitem">
+<a href="hod_user_guide.html">HOD User Guide</a>
</div>
<div class="menuitem">
-<a href="hod.html">Hadoop On Demand</a>
+<a href="hod_admin_guide.html">HOD Admin Guide</a>
+</div>
+<div class="menuitem">
+<a href="hod_config_guide.html">HOD Config Guide</a>
</div>
<div class="menuitem">
<a href="capacity_scheduler.html">Capacity Scheduler</a>
@@ -168,13 +174,10 @@
<a href="http://wiki.apache.org/hadoop/FAQ">FAQ</a>
</div>
<div class="menuitem">
-<a href="http://hadoop.apache.org/core/mailing_lists.html">Mailing Lists</a>
-</div>
-<div class="menuitem">
<a href="releasenotes.html">Release Notes</a>
</div>
<div class="menuitem">
-<a href="changes.html">All Changes</a>
+<a href="changes.html">Change Log</a>
</div>
</div>
<div id="credit"></div>
@@ -221,98 +224,104 @@
<ul>
<li>
-<a href="quickstart.html">Quickstart</a> ___________________ <em>quickstart</em>
+<a href="quickstart.html">Hadoop Quick Start</a> ___________________ <em>quickstart</em>
</li>
</ul>
<ul>
<li>
-<a href="cluster_setup.html">Cluster Setup</a> ___________________ <em>setup</em>
+<a href="cluster_setup.html">Hadoop Cluster Setup</a> ___________________ <em>setup</em>
</li>
</ul>
<ul>
<li>
-<a href="hdfs_design.html">HDFS Architecture</a> ___________________ <em>hdfs</em>
+<a href="mapred_tutorial.html">Hadoop Map/Reduce Tutorial</a> ___________________ <em>mapred</em>
</li>
</ul>
<ul>
<li>
-<a href="hdfs_user_guide.html">HDFS User Guide</a> ___________________ <em>hdfs</em>
+<a href="commands_manual.html">Hadoop Command Guide</a> ___________________ <em>commands</em>
</li>
</ul>
<ul>
<li>
-<a href="hdfs_permissions_guide.html">HDFS Permissions Guide</a> ___________________ <em>hdfs</em>
+<a href="hdfs_shell.html">Hadoop FS Shell Guide</a> ___________________ <em>fs</em>
</li>
</ul>
<ul>
<li>
-<a href="hdfs_quota_admin_guide.html">HDFS Quotas Administrator Guide</a> ___________________ <em>hdfs</em>
+<a href="distcp.html">Hadoop DistCp Guide</a> ___________________ <em>distcp</em>
</li>
</ul>
<ul>
<li>
-<a href="commands_manual.html">Commands Manual</a> ___________________ <em>commands</em>
+<a href="native_libraries.html">Hadoop Native Libraries</a> ___________________ <em>mapred</em>
</li>
</ul>
<ul>
<li>
-<a href="hdfs_shell.html">FS Shell Guide</a> ___________________ <em>fs</em>
+<a href="streaming.html">Hadoop Streaming</a> ___________________ <em>streaming</em>
</li>
</ul>
<ul>
<li>
-<a href="SLG_user_guide.html">Synthetic Load Generator User Guide</a> ___________________ <em>fs</em>
+<a href="hadoop_archives.html">Hadoop Archives</a> ___________________ <em>archives</em>
</li>
</ul>
<ul>
<li>
-<a href="distcp.html">DistCp Guide</a> ___________________ <em>distcp</em>
+<a href="hdfs_user_guide.html">HDFS User Guide</a> ___________________ <em>hdfs</em>
</li>
</ul>
<ul>
<li>
-<a href="mapred_tutorial.html">Map-Reduce Tutorial</a> ___________________ <em>mapred</em>
+<a href="hdfs_design.html">HDFS Architecture</a> ___________________ <em>hdfs</em>
</li>
</ul>
<ul>
<li>
-<a href="native_libraries.html">Native Hadoop Libraries</a> ___________________ <em>mapred</em>
+<a href="hdfs_permissions_guide.html">HDFS Admin Guide: Permissions</a> ___________________ <em>hdfs</em>
</li>
</ul>
<ul>
<li>
-<a href="streaming.html">Streaming</a> ___________________ <em>streaming</em>
+<a href="hdfs_quota_admin_guide.html">HDFS Admin Guide: Quotas</a> ___________________ <em>hdfs</em>
</li>
</ul>
<ul>
<li>
-<a href="hadoop_archives.html">Hadoop Archives</a> ___________________ <em>archives</em>
+<a href="SLG_user_guide.html">HDFS Utilities</a> ___________________ <em>fs</em>
</li>
</ul>
<ul>
<li>
-<a href="hod.html">Hadoop On Demand</a> ___________________ <em>hod</em>
+<a href="hod_user_guide.html">HOD User Guide</a> ___________________ <em>hod-user-guide</em>
</li>
-<ul>
-
-
-
+</ul>
+<ul>
+<li>
+<a href="hod_admin_guide.html">HOD Admin Guide</a> ___________________ <em>hod-admin-guide</em>
+</li>
</ul>
+
+<ul>
+<li>
+<a href="hod_config_guide.html">HOD Config Guide</a> ___________________ <em>hod-config-guide</em>
+</li>
</ul>
<ul>
@@ -347,25 +356,19 @@
<ul>
<li>
-<a href="http://hadoop.apache.org/core/mailing_lists.html">Mailing Lists</a> ___________________ <em>lists</em>
-</li>
-</ul>
-
-<ul>
-<li>
<a href="releasenotes.html">Release Notes</a> ___________________ <em>relnotes</em>
</li>
</ul>
<ul>
<li>
-<a href="changes.html">All Changes</a> ___________________ <em>changes</em>
+<a href="changes.html">Change Log</a> ___________________ <em>changes</em>
</li>
</ul>
</ul>
</ul>
-
+
Modified: hadoop/core/branches/branch-0.19/docs/linkmap.pdf
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.19/docs/linkmap.pdf?rev=706368&r1=706367&r2=706368&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.19/docs/linkmap.pdf (original)
+++ hadoop/core/branches/branch-0.19/docs/linkmap.pdf Mon Oct 20 12:00:39 2008
@@ -5,10 +5,10 @@
/Producer (FOP 0.20.5) >>
endobj
5 0 obj
-<< /Length 1195 /Filter [ /ASCII85Decode /FlateDecode ]
+<< /Length 1190 /Filter [ /ASCII85Decode /FlateDecode ]
>>
stream
-Gatn'gN&cE'Sc)T'O7LO;:,ucFIesDVPhr3oO<#?7g1nd`ukV?1O8j)E/iq=M?c528Iu<rs0TFfBj%W?Z^8-\5QlbVC-;sZJ*AKmmUn3ZbqBKgp)YG%^aosr7I]cr;JP_G\40>3Sm8sqj[6kn_i5E2\Zqa,ol.T@V&#o1?NgB4X*7%$"q^IMNZ=$&^d?JmX`@RR+_^c6b.d#Ujdr+eoTK0\cG_J.<O(?cD;D1f4lU$b.;;GTel@q!V@qh&%R'/8!D0q:s,IZhb@G&6VM2;eOPdBm__\.C\^7Porpf0t+X$Y,+Gq:4''+f6kNI[dXG%c.T[:NhL9R<JNCo\WHu<1^<DH!#"SZT<:o;[U'a&"6%>rA:&)*qPW-2'$9r=X%@)uR^FU1#t(*X<:aR0t751H$&H9*#41M3[M>%'qb8;mci*RQGs9s$iA'_kSiAocR5;g*$O*6#LM3!RLG:oQsHDf0HZ"6$mbF&U2O$R^QR$",$%<V(#i?A*u.Y<Q:;YI0qQf8\#"^a4M.k),dcgR@sags;]EA6g-ef,(/.fXqp<0#MDQ+nPQPV7p`0jqdGK.qC*R.a>B4<;&o%!(O4dFj`Bk*;Q/'MSe^9(Ae>C2LI<Kd_(5fE])<ua;?@a^%Sf(^!+H-F/c1"&qC:.[84mo8\>gD5*olZf.n?`h^`U3k-%3&Nf)kK7O9DWM(u#_Gfl73X*Aen;pL<g`H$u4PqZ8n1CECA&rg>`Fl`kB^&B4]-FLF`<FaH.c`-,OOX@66C%*U7NO#8]HCafXoIi:ao?_7N/f=Djift5NA8jp;2W>4bWSln@nelGH/&BL,G(377dB4(A83K?]@l$iBGOIUc@^f#!VWSM[-5o84C@Xa=0iC^CgYNP,VfK;]&OkUS:FoW%1Y_3*3oNG]`9XK.hssb3B57rOF5sW%[qM>_rA_+I?KXgAKHJ$b$2'a'rqW[]eul]Y>-k0nIN:/WQ/csug\*YLTj-Tj]K`/hRE6d
sjo3TeH9G"o,Hm`RaTjhoW*S37kYGKo0ch<]?#r*B3toXE^?1LR(]IR^*kQKJmRo>E,7r7QFIF@<1oNgHT&%Xt[HA4##XYa[T86?4W-(3'gFl-W"pkJjl9uJ'qcq8?>Rq&Mh[tR[#at,13j6W,du([jK0<P@[=7,#Flud_A_)8C#1dR<F4:^]mFiHddqJ)9#)9,W!_6K:0)~>
+Gau1/bAs(+'Sc@2$C9p#WpW,rWSR2%kp<aL...@MA>@^6%L?_Dp6EEN;H!Y9Bekkp(n^giir$i[ShDEd01g[CJAG\TJbEG+J\c)Q!bd$[&DMQ%69oc0/2k?-ZanNT8T9^-a?2Jp;VN94Z"n8/`4*/G4C>PKdsS4X<gQU60>ZY-;.]jGNNsqP_@UZW)nAD78fNB8-YMkTBJ0l!c(L28Y&u)G6'?/0CT3jdV/f(8RlX0qOXbKN-+YQO33o:hFr#]i3j@LLbT9:j5s$ZW6U6-6qb!Z0DKB6j(B/X[k89<<$]%.QiY_P9RiK$`0XhT-!=V-XF+g"CB6dBlb^1",jkI%RKTZPa'h\)YTjg:BhI,"6m6H$5Qc3:Q]?sT:!.q_aOWT5LO[0DM%^P,Q4B0[MV32qfE3;X'a&5iL??6hVqR9LuCr5mbdtkC1lJ\>;X&@-X+d]bY?(_nb!p)U3fgP1Zk-V8T*pC(Bp[k4SX2HaeU=+=\`-O7$29$-:cXHZF+k(m`au\@$Do2+j`bRsq/!\9)e2RX8;kG1H,l4W5D/U94I5uLd,a7QeULdiIHO0@KQ35b_+)K5j.eRoX=MW#M'HHoKWDQ4_Mk!2o?t[20rHRcFpm(\VLdK4(r(Jn3N49*IYenZM@'D`U\r^Br[]@e]46CZ_H_P'j[?&itH)MZ8MBk!a4F0bZ=VMF#I8+\ig.+bC\,U'Cj$&`GO$9Ol8PB.tg_\;b!L=r9G$E80Xf#1BH:]uc1Bs@=^?Qh9*2@D<q4+F`W'BG;=ZcU*MiB2ZZRd#53c;6ZI&(1BKXfcVs,t<CQ"r+GhgD\JRt-DQ/-3*NlEm6lgfghF5,#:pZi;/F<'88GLkKsKn,;VQfIq!j)4l#;Vrs>m3+"P:qm[7*2PSCn\'c,..YW$&nAHogYTa1k;9ZVRl&qOLjAN*$);S6#kXhm3PWe)uNgtut8<j,<Hj_9h=eW"a\iHbiH*YOXkA/b*n&&aGV[kFE]QR=
IoknX57LRi&5>Ou><'MC[3\BRC9f'%[S^8Qs+6L;mpJ``2LX3V89Q'0KOa<@T?jK(7crq=hb^r.cbK9G<\NNbDBgQbu:l0-^R<+sO(]uL*5th7d)jtESqcTjtVV@;t)pD,?_@T2^!DDPpY?[IHm>gB*;W!>J\T=0?gU,hK"6X1=o"5jig8-TkVUOmC$JO!Z!QVZA?N~>
endstream
endobj
6 0 obj
@@ -20,10 +20,10 @@
>>
endobj
7 0 obj
-<< /Length 465 /Filter [ /ASCII85Decode /FlateDecode ]
+<< /Length 483 /Filter [ /ASCII85Decode /FlateDecode ]
>>
stream
-GatU.8Pat('Ya/hhQ]6=S:--%gXq-Uj20%Z1U^4SUeh(Lrd3=%^o$Wu3.2=OjXnrs]:;Ne$jd&PO$Mo++QZ7IWiE"n6:D"[>Bh2@(mAI>TN%*Fe*,@PEKl,fKf+RR-nk-(8lERYj(AcR&6tQnlJT5Ij-h),@d"XfMChC@OkPn%UGrkVdDaDnXp]NA(@I-C!]a7Ja3;Lo,PkbBC65"-3JlcKj(r4hn;NT(U/GIa\(kc-1^4.QT-Y=A_`2>POYX[[4[he&40?o^MH)`5hR@YeW!2lH*r_G-#B5tUKhDoF]F)<4qUYSE3D),C$)2=.+HqmI=F1S8ZoUtY2l8.dEPoU/BQTa7hM+CIen^J4p:/<6[#k5"QP+]m_2,C*g.9K:h+nf?S"eIieaOAKdLIuu9S\4ZlbI%DQ[SdrY)@4fod]S*1KiBT8e^R=Raj?ip<J$;jY$FN/*pmB$Uf$kOo~>
+GatUo9i$C,&;KZOME0CCj6j2pm'/JB@#XW.`.KpZXU?F!ZkPCsHP^@,C""n935)ZBq`_>pcK!i!.Y'9"&17L01n[+h97b6_."t&\35k:TGVWc&N8Tpu6/9l06hZg+.7gM2pR`$[6%%o0jDVod4UfH8qc@RBkn&1^)kKkL5JT&qhu4;02c%Z_3":k/*MBsL\M\0g.]b.*QaL-!),P;]W>m6c2D7,?,Fh)9,R6:V$WEiBk#IiK),bM)RLS<>Xae[<+@>L?EQ0\A+Xe6QA\.3=cg$ff.F^sFLQrTmmq+SY`TAHgIT7]s3@Tm?(A0T`+q@Z@5H^L7-rHoGn9/W"9L*/4$#YCeAguprVJpcg'S>HaW=@sK8Vd,c)j?H0;Ru$+(]J.UJQ`D:\/UfY<g.S"?"N8pDZ-<lnn1YC?84d@?H!<,Mt,6>I=5HLop;Yjb-_'p?;6uU"aY$2hMX:=m:j2@eYdi7rpG48R?[G.~>
endstream
endobj
8 0 obj
@@ -87,19 +87,19 @@
xref
0 14
0000000000 65535 f
-0000002683 00000 n
-0000002747 00000 n
-0000002797 00000 n
+0000002696 00000 n
+0000002760 00000 n
+0000002810 00000 n
0000000015 00000 n
0000000071 00000 n
-0000001358 00000 n
-0000001464 00000 n
-0000002020 00000 n
-0000002126 00000 n
-0000002238 00000 n
-0000002348 00000 n
-0000002459 00000 n
-0000002567 00000 n
+0000001353 00000 n
+0000001459 00000 n
+0000002033 00000 n
+0000002139 00000 n
+0000002251 00000 n
+0000002361 00000 n
+0000002472 00000 n
+0000002580 00000 n
trailer
<<
/Size 14
@@ -107,5 +107,5 @@
/Info 4 0 R
>>
startxref
-2919
+2932
%%EOF
Modified: hadoop/core/branches/branch-0.19/docs/mapred_tutorial.html
URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.19/docs/mapred_tutorial.html?rev=706368&r1=706367&r2=706368&view=diff
==============================================================================
--- hadoop/core/branches/branch-0.19/docs/mapred_tutorial.html (original)
+++ hadoop/core/branches/branch-0.19/docs/mapred_tutorial.html Mon Oct 20 12:00:39 2008
@@ -108,49 +108,55 @@
<a href="index.html">Overview</a>
</div>
<div class="menuitem">
-<a href="quickstart.html">Quickstart</a>
+<a href="quickstart.html">Hadoop Quick Start</a>
</div>
<div class="menuitem">
-<a href="cluster_setup.html">Cluster Setup</a>
+<a href="cluster_setup.html">Hadoop Cluster Setup</a>
+</div>
+<div class="menupage">
+<div class="menupagetitle">Hadoop Map/Reduce Tutorial</div>
</div>
<div class="menuitem">
-<a href="hdfs_design.html">HDFS Architecture</a>
+<a href="commands_manual.html">Hadoop Command Guide</a>
</div>
<div class="menuitem">
-<a href="hdfs_user_guide.html">HDFS User Guide</a>
+<a href="hdfs_shell.html">Hadoop FS Shell Guide</a>
</div>
<div class="menuitem">
-<a href="hdfs_permissions_guide.html">HDFS Permissions Guide</a>
+<a href="distcp.html">Hadoop DistCp Guide</a>
</div>
<div class="menuitem">
-<a href="hdfs_quota_admin_guide.html">HDFS Quotas Administrator Guide</a>
+<a href="native_libraries.html">Hadoop Native Libraries</a>
</div>
<div class="menuitem">
-<a href="commands_manual.html">Commands Manual</a>
+<a href="streaming.html">Hadoop Streaming</a>
</div>
<div class="menuitem">
-<a href="hdfs_shell.html">FS Shell Guide</a>
+<a href="hadoop_archives.html">Hadoop Archives</a>
</div>
<div class="menuitem">
-<a href="SLG_user_guide.html">Synthetic Load Generator User Guide</a>
+<a href="hdfs_user_guide.html">HDFS User Guide</a>
</div>
<div class="menuitem">
-<a href="distcp.html">DistCp Guide</a>
+<a href="hdfs_design.html">HDFS Architecture</a>
</div>
-<div class="menupage">
-<div class="menupagetitle">Map-Reduce Tutorial</div>
+<div class="menuitem">
+<a href="hdfs_permissions_guide.html">HDFS Admin Guide: Permissions</a>
</div>
<div class="menuitem">
-<a href="native_libraries.html">Native Hadoop Libraries</a>
+<a href="hdfs_quota_admin_guide.html">HDFS Admin Guide: Quotas</a>
</div>
<div class="menuitem">
-<a href="streaming.html">Streaming</a>
+<a href="SLG_user_guide.html">HDFS Utilities</a>
</div>
<div class="menuitem">
-<a href="hadoop_archives.html">Hadoop Archives</a>
+<a href="hod_user_guide.html">HOD User Guide</a>
</div>
<div class="menuitem">
-<a href="hod.html">Hadoop On Demand</a>
+<a href="hod_admin_guide.html">HOD Admin Guide</a>
+</div>
+<div class="menuitem">
+<a href="hod_config_guide.html">HOD Config Guide</a>
</div>
<div class="menuitem">
<a href="capacity_scheduler.html">Capacity Scheduler</a>
@@ -168,13 +174,10 @@
<a href="http://wiki.apache.org/hadoop/FAQ">FAQ</a>
</div>
<div class="menuitem">
-<a href="http://hadoop.apache.org/core/mailing_lists.html">Mailing Lists</a>
-</div>
-<div class="menuitem">
<a href="releasenotes.html">Release Notes</a>
</div>
<div class="menuitem">
-<a href="changes.html">All Changes</a>
+<a href="changes.html">Change Log</a>
</div>
</div>
<div id="credit"></div>
@@ -345,7 +348,7 @@
<a href="#Example%3A+WordCount+v2.0">Example: WordCount v2.0</a>
<ul class="minitoc">
<li>
-<a href="#Source+Code-N10FB2">Source Code</a>
+<a href="#Source+Code-N10FAA">Source Code</a>
</li>
<li>
<a href="#Sample+Runs">Sample Runs</a>
@@ -376,11 +379,13 @@
<ul>
<li>
- Hadoop <a href="quickstart.html">Quickstart</a> for first-time users.
+
+<a href="quickstart.html">Hadoop Quick Start</a> for first-time users.
</li>
<li>
- Hadoop <a href="cluster_setup.html">Cluster Setup</a> for large,
+
+<a href="cluster_setup.html">Hadoop Cluster Setup</a> for large,
distributed clusters.
</li>
@@ -403,8 +408,8 @@
takes care of scheduling tasks, monitoring them and re-executes the failed
tasks.</p>
<p>Typically the compute nodes and the storage nodes are the same, that is,
- the Map/Reduce framework and the <a href="hdfs_design.html">Distributed
- FileSystem</a> are running on the same set of nodes. This configuration
+ the Map/Reduce framework and the Hadoop Distributed File System (see <a href="hdfs_design.html">HDFS Architecture </a>)
+ are running on the same set of nodes. This configuration
allows the framework to effectively schedule tasks on the nodes where data
is already present, resulting in very high aggregate bandwidth across the
cluster.</p>
@@ -488,12 +493,9 @@
<p>
<span class="codefrag">WordCount</span> is a simple application that counts the number of
occurences of each word in a given input set.</p>
-<p>This works with a
- <a href="quickstart.html#Standalone+Operation">local-standalone</a>,
- <a href="quickstart.html#SingleNodeSetup">pseudo-distributed</a> or
- <a href="quickstart.html#Fully-Distributed+Operation">fully-distributed</a>
- Hadoop installation.</p>
-<a name="N100EA"></a><a name="Source+Code"></a>
+<p>This works with a local-standalone, pseudo-distributed or fully-distributed
+ Hadoop installation(see <a href="quickstart.html"> Hadoop Quick Start</a>).</p>
+<a name="N100E2"></a><a name="Source+Code"></a>
<h3 class="h4">Source Code</h3>
<table class="ForrestTable" cellspacing="1" cellpadding="4">
@@ -1056,7 +1058,7 @@
</tr>
</table>
-<a name="N1046C"></a><a name="Usage"></a>
+<a name="N10464"></a><a name="Usage"></a>
<h3 class="h4">Usage</h3>
<p>Assuming <span class="codefrag">HADOOP_HOME</span> is the root of the installation and
<span class="codefrag">HADOOP_VERSION</span> is the Hadoop version installed, compile
@@ -1159,7 +1161,7 @@
as arguments that are unzipped/unjarred and a link with name of the
jar/zip are created in the current working directory of tasks. More
details about the command line options are available at
- <a href="commands_manual.html">Commands manual</a>
+ <a href="commands_manual.html"> Hadoop Command Guide.</a>
</p>
<p>Running <span class="codefrag">wordcount</span> example with
<span class="codefrag">-libjars</span> and <span class="codefrag">-files</span>:<br>
@@ -1168,7 +1170,7 @@
-libjars mylib.jar input output </span>
</p>
-<a name="N1050C"></a><a name="Walk-through"></a>
+<a name="N10504"></a><a name="Walk-through"></a>
<h3 class="h4">Walk-through</h3>
<p>The <span class="codefrag">WordCount</span> application is quite straight-forward.</p>
<p>The <span class="codefrag">Mapper</span> implementation (lines 14-26), via the
@@ -1278,7 +1280,7 @@
</div>
-<a name="N105C3"></a><a name="Map%2FReduce+-+User+Interfaces"></a>
+<a name="N105BB"></a><a name="Map%2FReduce+-+User+Interfaces"></a>
<h2 class="h3">Map/Reduce - User Interfaces</h2>
<div class="section">
<p>This section provides a reasonable amount of detail on every user-facing
@@ -1298,12 +1300,12 @@
<p>Finally, we will wrap up by discussing some useful features of the
framework such as the <span class="codefrag">DistributedCache</span>,
<span class="codefrag">IsolationRunner</span> etc.</p>
-<a name="N105FF"></a><a name="Payload"></a>
+<a name="N105F7"></a><a name="Payload"></a>
<h3 class="h4">Payload</h3>
<p>Applications typically implement the <span class="codefrag">Mapper</span> and
<span class="codefrag">Reducer</span> interfaces to provide the <span class="codefrag">map</span> and
<span class="codefrag">reduce</span> methods. These form the core of the job.</p>
-<a name="N10614"></a><a name="Mapper"></a>
+<a name="N1060C"></a><a name="Mapper"></a>
<h4>Mapper</h4>
<p>
<a href="api/org/apache/hadoop/mapred/Mapper.html">
@@ -1359,7 +1361,7 @@
<a href="api/org/apache/hadoop/io/compress/CompressionCodec.html">
CompressionCodec</a> to be used via the <span class="codefrag">JobConf</span>.
</p>
-<a name="N1068A"></a><a name="How+Many+Maps%3F"></a>
+<a name="N10682"></a><a name="How+Many+Maps%3F"></a>
<h5>How Many Maps?</h5>
<p>The number of maps is usually driven by the total size of the
inputs, that is, the total number of blocks of the input files.</p>
@@ -1372,7 +1374,7 @@
<a href="api/org/apache/hadoop/mapred/JobConf.html#setNumMapTasks(int)">
setNumMapTasks(int)</a> (which only provides a hint to the framework)
is used to set it even higher.</p>
-<a name="N106A2"></a><a name="Reducer"></a>
+<a name="N1069A"></a><a name="Reducer"></a>
<h4>Reducer</h4>
<p>
<a href="api/org/apache/hadoop/mapred/Reducer.html">
@@ -1395,18 +1397,18 @@
<p>
<span class="codefrag">Reducer</span> has 3 primary phases: shuffle, sort and reduce.
</p>
-<a name="N106D2"></a><a name="Shuffle"></a>
+<a name="N106CA"></a><a name="Shuffle"></a>
<h5>Shuffle</h5>
<p>Input to the <span class="codefrag">Reducer</span> is the sorted output of the
mappers. In this phase the framework fetches the relevant partition
of the output of all the mappers, via HTTP.</p>
-<a name="N106DF"></a><a name="Sort"></a>
+<a name="N106D7"></a><a name="Sort"></a>
<h5>Sort</h5>
<p>The framework groups <span class="codefrag">Reducer</span> inputs by keys (since
different mappers may have output the same key) in this stage.</p>
<p>The shuffle and sort phases occur simultaneously; while
map-outputs are being fetched they are merged.</p>
-<a name="N106EE"></a><a name="Secondary+Sort"></a>
+<a name="N106E6"></a><a name="Secondary+Sort"></a>
<h5>Secondary Sort</h5>
<p>If equivalence rules for grouping the intermediate keys are
required to be different from those for grouping keys before
@@ -1417,7 +1419,7 @@
JobConf.setOutputKeyComparatorClass(Class)</a> can be used to
control how intermediate keys are grouped, these can be used in
conjunction to simulate <em>secondary sort on values</em>.</p>
-<a name="N10707"></a><a name="Reduce"></a>
+<a name="N106FF"></a><a name="Reduce"></a>
<h5>Reduce</h5>
<p>In this phase the
<a href="api/org/apache/hadoop/mapred/Reducer.html#reduce(K2, java.util.Iterator, org.apache.hadoop.mapred.OutputCollector, org.apache.hadoop.mapred.Reporter)">
@@ -1433,7 +1435,7 @@
progress, set application-level status messages and update
<span class="codefrag">Counters</span>, or just indicate that they are alive.</p>
<p>The output of the <span class="codefrag">Reducer</span> is <em>not sorted</em>.</p>
-<a name="N10735"></a><a name="How+Many+Reduces%3F"></a>
+<a name="N1072D"></a><a name="How+Many+Reduces%3F"></a>
<h5>How Many Reduces?</h5>
<p>The right number of reduces seems to be <span class="codefrag">0.95</span> or
<span class="codefrag">1.75</span> multiplied by (<<em>no. of nodes</em>> *
@@ -1448,7 +1450,7 @@
<p>The scaling factors above are slightly less than whole numbers to
reserve a few reduce slots in the framework for speculative-tasks and
failed tasks.</p>
-<a name="N1075A"></a><a name="Reducer+NONE"></a>
+<a name="N10752"></a><a name="Reducer+NONE"></a>
<h5>Reducer NONE</h5>
<p>It is legal to set the number of reduce-tasks to <em>zero</em> if
no reduction is desired.</p>
@@ -1458,7 +1460,7 @@
setOutputPath(Path)</a>. The framework does not sort the
map-outputs before writing them out to the <span class="codefrag">FileSystem</span>.
</p>
-<a name="N10775"></a><a name="Partitioner"></a>
+<a name="N1076D"></a><a name="Partitioner"></a>
<h4>Partitioner</h4>
<p>
<a href="api/org/apache/hadoop/mapred/Partitioner.html">
@@ -1472,7 +1474,7 @@
<p>
<a href="api/org/apache/hadoop/mapred/lib/HashPartitioner.html">
HashPartitioner</a> is the default <span class="codefrag">Partitioner</span>.</p>
-<a name="N10794"></a><a name="Reporter"></a>
+<a name="N1078C"></a><a name="Reporter"></a>
<h4>Reporter</h4>
<p>
<a href="api/org/apache/hadoop/mapred/Reporter.html">
@@ -1491,7 +1493,7 @@
</p>
<p>Applications can also update <span class="codefrag">Counters</span> using the
<span class="codefrag">Reporter</span>.</p>
-<a name="N107BE"></a><a name="OutputCollector"></a>
+<a name="N107B6"></a><a name="OutputCollector"></a>
<h4>OutputCollector</h4>
<p>
<a href="api/org/apache/hadoop/mapred/OutputCollector.html">
@@ -1502,7 +1504,7 @@
<p>Hadoop Map/Reduce comes bundled with a
<a href="api/org/apache/hadoop/mapred/lib/package-summary.html">
library</a> of generally useful mappers, reducers, and partitioners.</p>
-<a name="N107D9"></a><a name="Job+Configuration"></a>
+<a name="N107D1"></a><a name="Job+Configuration"></a>
<h3 class="h4">Job Configuration</h3>
<p>
<a href="api/org/apache/hadoop/mapred/JobConf.html">
@@ -1561,7 +1563,7 @@
<a href="api/org/apache/hadoop/conf/Configuration.html#set(java.lang.String, java.lang.String)">set(String, String)</a>/<a href="api/org/apache/hadoop/conf/Configuration.html#get(java.lang.String, java.lang.String)">get(String, String)</a>
to set/get arbitrary parameters needed by applications. However, use the
<span class="codefrag">DistributedCache</span> for large amounts of (read-only) data.</p>
-<a name="N1086E"></a><a name="Task+Execution+%26+Environment"></a>
+<a name="N10866"></a><a name="Task+Execution+%26+Environment"></a>
<h3 class="h4">Task Execution & Environment</h3>
<p>The <span class="codefrag">TaskTracker</span> executes the <span class="codefrag">Mapper</span>/
<span class="codefrag">Reducer</span> <em>task</em> as a child process in a separate jvm.
@@ -1603,7 +1605,7 @@
<span class="codefrag"></property></span>
</p>
-<a name="N108BF"></a><a name="Memory+management"></a>
+<a name="N108B7"></a><a name="Memory+management"></a>
<h4> Memory management</h4>
<p>Users/admins can also specify the maximum virtual memory
of the launched child-task, and any sub-process it launches
@@ -1646,7 +1648,7 @@
counters for a job- particularly relative to byte counts from the map
and into the reduce- is invaluable to the tuning of these
parameters.</p>
-<a name="N108F0"></a><a name="Map+Parameters"></a>
+<a name="N108E8"></a><a name="Map+Parameters"></a>
<h4>Map Parameters</h4>
<p>A record emitted from a map will be serialized into a buffer and
metadata will be stored into accounting buffers. As described in the
@@ -1720,7 +1722,7 @@
combiner.</li>
</ul>
-<a name="N1095C"></a><a name="Shuffle%2FReduce+Parameters"></a>
+<a name="N10954"></a><a name="Shuffle%2FReduce+Parameters"></a>
<h4>Shuffle/Reduce Parameters</h4>
<p>As described previously, each reduce fetches the output assigned
to it by the Partitioner via HTTP into memory and periodically
@@ -1816,7 +1818,7 @@
of the intermediate merge.</li>
</ul>
-<a name="N109D7"></a><a name="Directory+Structure"></a>
+<a name="N109CF"></a><a name="Directory+Structure"></a>
<h4> Directory Structure </h4>
<p>The task tracker has local directory,
<span class="codefrag"> ${mapred.local.dir}/taskTracker/</span> to create localized
@@ -1917,7 +1919,7 @@
</li>
</ul>
-<a name="N10A46"></a><a name="Task+JVM+Reuse"></a>
+<a name="N10A3E"></a><a name="Task+JVM+Reuse"></a>
<h4>Task JVM Reuse</h4>
<p>Jobs can enable task JVMs to be reused by specifying the job
configuration <span class="codefrag">mapred.job.reuse.jvm.num.tasks</span>. If the
@@ -2009,7 +2011,7 @@
<a href="native_libraries.html#Loading+native+libraries+through+DistributedCache">
native_libraries.html</a>
</p>
-<a name="N10B2F"></a><a name="Job+Submission+and+Monitoring"></a>
+<a name="N10B27"></a><a name="Job+Submission+and+Monitoring"></a>
<h3 class="h4">Job Submission and Monitoring</h3>
<p>
<a href="api/org/apache/hadoop/mapred/JobClient.html">
@@ -2070,7 +2072,7 @@
<p>Normally the user creates the application, describes various facets
of the job via <span class="codefrag">JobConf</span>, and then uses the
<span class="codefrag">JobClient</span> to submit the job and monitor its progress.</p>
-<a name="N10B8F"></a><a name="Job+Control"></a>
+<a name="N10B87"></a><a name="Job+Control"></a>
<h4>Job Control</h4>
<p>Users may need to chain Map/Reduce jobs to accomplish complex
tasks which cannot be done via a single Map/Reduce job. This is fairly
@@ -2106,7 +2108,7 @@
</li>
</ul>
-<a name="N10BB9"></a><a name="Job+Input"></a>
+<a name="N10BB1"></a><a name="Job+Input"></a>
<h3 class="h4">Job Input</h3>
<p>
<a href="api/org/apache/hadoop/mapred/InputFormat.html">
@@ -2154,7 +2156,7 @@
appropriate <span class="codefrag">CompressionCodec</span>. However, it must be noted that
compressed files with the above extensions cannot be <em>split</em> and
each compressed file is processed in its entirety by a single mapper.</p>
-<a name="N10C23"></a><a name="InputSplit"></a>
+<a name="N10C1B"></a><a name="InputSplit"></a>
<h4>InputSplit</h4>
<p>
<a href="api/org/apache/hadoop/mapred/InputSplit.html">
@@ -2168,7 +2170,7 @@
FileSplit</a> is the default <span class="codefrag">InputSplit</span>. It sets
<span class="codefrag">map.input.file</span> to the path of the input file for the
logical split.</p>
-<a name="N10C48"></a><a name="RecordReader"></a>
+<a name="N10C40"></a><a name="RecordReader"></a>
<h4>RecordReader</h4>
<p>
<a href="api/org/apache/hadoop/mapred/RecordReader.html">
@@ -2180,7 +2182,7 @@
for processing. <span class="codefrag">RecordReader</span> thus assumes the
responsibility of processing record boundaries and presents the tasks
with keys and values.</p>
-<a name="N10C6B"></a><a name="Job+Output"></a>
+<a name="N10C63"></a><a name="Job+Output"></a>
<h3 class="h4">Job Output</h3>
<p>
<a href="api/org/apache/hadoop/mapred/OutputFormat.html">
@@ -2205,7 +2207,7 @@
<p>
<span class="codefrag">TextOutputFormat</span> is the default
<span class="codefrag">OutputFormat</span>.</p>
-<a name="N10C94"></a><a name="OutputCommitter"></a>
+<a name="N10C8C"></a><a name="OutputCommitter"></a>
<h4>OutputCommitter</h4>
<p>
<a href="api/org/apache/hadoop/mapred/OutputCommitter.html">
@@ -2247,7 +2249,7 @@
<p>
<span class="codefrag">FileOutputCommitter</span> is the default
<span class="codefrag">OutputCommitter</span>.</p>
-<a name="N10CC4"></a><a name="Task+Side-Effect+Files"></a>
+<a name="N10CBC"></a><a name="Task+Side-Effect+Files"></a>
<h4>Task Side-Effect Files</h4>
<p>In some applications, component tasks need to create and/or write to
side-files, which differ from the actual job-output files.</p>
@@ -2288,7 +2290,7 @@
<p>The entire discussion holds true for maps of jobs with
reducer=NONE (i.e. 0 reduces) since output of the map, in that case,
goes directly to HDFS.</p>
-<a name="N10D12"></a><a name="RecordWriter"></a>
+<a name="N10D0A"></a><a name="RecordWriter"></a>
<h4>RecordWriter</h4>
<p>
<a href="api/org/apache/hadoop/mapred/RecordWriter.html">
@@ -2296,9 +2298,9 @@
pairs to an output file.</p>
<p>RecordWriter implementations write the job outputs to the
<span class="codefrag">FileSystem</span>.</p>
-<a name="N10D29"></a><a name="Other+Useful+Features"></a>
+<a name="N10D21"></a><a name="Other+Useful+Features"></a>
<h3 class="h4">Other Useful Features</h3>
-<a name="N10D2F"></a><a name="Submitting+Jobs+to+a+Queue"></a>
+<a name="N10D27"></a><a name="Submitting+Jobs+to+a+Queue"></a>
<h4>Submitting Jobs to a Queue</h4>
<p>Some job schedulers supported in Hadoop, like the
<a href="capacity_scheduler.html">Capacity
@@ -2314,7 +2316,7 @@
given user. In that case, if the job is not submitted
to one of the queues where the user has access,
the job would be rejected.</p>
-<a name="N10D47"></a><a name="Counters"></a>
+<a name="N10D3F"></a><a name="Counters"></a>
<h4>Counters</h4>
<p>
<span class="codefrag">Counters</span> represent global counters, defined either by
@@ -2331,7 +2333,7 @@
in the <span class="codefrag">map</span> and/or
<span class="codefrag">reduce</span> methods. These counters are then globally
aggregated by the framework.</p>
-<a name="N10D76"></a><a name="DistributedCache"></a>
+<a name="N10D6E"></a><a name="DistributedCache"></a>
<h4>DistributedCache</h4>
<p>
<a href="api/org/apache/hadoop/filecache/DistributedCache.html">
@@ -2402,7 +2404,7 @@
<span class="codefrag">mapred.job.classpath.{files|archives}</span>. Similarly the
cached files that are symlinked into the working directory of the
task can be used to distribute native libraries and load them.</p>
-<a name="N10DF9"></a><a name="Tool"></a>
+<a name="N10DF1"></a><a name="Tool"></a>
<h4>Tool</h4>
<p>The <a href="api/org/apache/hadoop/util/Tool.html">Tool</a>
interface supports the handling of generic Hadoop command-line options.
@@ -2442,7 +2444,7 @@
</span>
</p>
-<a name="N10E2B"></a><a name="IsolationRunner"></a>
+<a name="N10E23"></a><a name="IsolationRunner"></a>
<h4>IsolationRunner</h4>
<p>
<a href="api/org/apache/hadoop/mapred/IsolationRunner.html">
@@ -2466,7 +2468,7 @@
<p>
<span class="codefrag">IsolationRunner</span> will run the failed task in a single
jvm, which can be in the debugger, over precisely the same input.</p>
-<a name="N10E5E"></a><a name="Profiling"></a>
+<a name="N10E56"></a><a name="Profiling"></a>
<h4>Profiling</h4>
<p>Profiling is a utility to get a representative (2 or 3) sample
of built-in java profiler for a sample of maps and reduces. </p>
@@ -2499,7 +2501,7 @@
<span class="codefrag">-agentlib:hprof=cpu=samples,heap=sites,force=n,thread=y,verbose=n,file=%s</span>
</p>
-<a name="N10E92"></a><a name="Debugging"></a>
+<a name="N10E8A"></a><a name="Debugging"></a>
<h4>Debugging</h4>
<p>Map/Reduce framework provides a facility to run user-provided
scripts for debugging. When map/reduce task fails, user can run
@@ -2510,14 +2512,14 @@
<p> In the following sections we discuss how to submit debug script
along with the job. For submitting debug script, first it has to
distributed. Then the script has to supplied in Configuration. </p>
-<a name="N10E9E"></a><a name="How+to+distribute+script+file%3A"></a>
+<a name="N10E96"></a><a name="How+to+distribute+script+file%3A"></a>
<h5> How to distribute script file: </h5>
<p>
The user has to use
<a href="mapred_tutorial.html#DistributedCache">DistributedCache</a>
mechanism to <em>distribute</em> and <em>symlink</em> the
debug script file.</p>
-<a name="N10EB2"></a><a name="How+to+submit+script%3A"></a>
+<a name="N10EAA"></a><a name="How+to+submit+script%3A"></a>
<h5> How to submit script: </h5>
<p> A quick way to submit debug script is to set values for the
properties "mapred.map.task.debug.script" and
@@ -2541,17 +2543,17 @@
<span class="codefrag">$script $stdout $stderr $syslog $jobconf $program </span>
</p>
-<a name="N10ED4"></a><a name="Default+Behavior%3A"></a>
+<a name="N10ECC"></a><a name="Default+Behavior%3A"></a>
<h5> Default Behavior: </h5>
<p> For pipes, a default script is run to process core dumps under
gdb, prints stack trace and gives info about running threads. </p>
-<a name="N10EDF"></a><a name="JobControl"></a>
+<a name="N10ED7"></a><a name="JobControl"></a>
<h4>JobControl</h4>
<p>
<a href="api/org/apache/hadoop/mapred/jobcontrol/package-summary.html">
JobControl</a> is a utility which encapsulates a set of Map/Reduce jobs
and their dependencies.</p>
-<a name="N10EEC"></a><a name="Data+Compression"></a>
+<a name="N10EE4"></a><a name="Data+Compression"></a>
<h4>Data Compression</h4>
<p>Hadoop Map/Reduce provides facilities for the application-writer to
specify compression for both intermediate map-outputs and the
@@ -2565,7 +2567,7 @@
codecs for reasons of both performance (zlib) and non-availability of
Java libraries (lzo). More details on their usage and availability are
available <a href="native_libraries.html">here</a>.</p>
-<a name="N10F0C"></a><a name="Intermediate+Outputs"></a>
+<a name="N10F04"></a><a name="Intermediate+Outputs"></a>
<h5>Intermediate Outputs</h5>
<p>Applications can control compression of intermediate map-outputs
via the
@@ -2574,7 +2576,7 @@
<span class="codefrag">CompressionCodec</span> to be used via the
<a href="api/org/apache/hadoop/mapred/JobConf.html#setMapOutputCompressorClass(java.lang.Class)">
JobConf.setMapOutputCompressorClass(Class)</a> api.</p>
-<a name="N10F21"></a><a name="Job+Outputs"></a>
+<a name="N10F19"></a><a name="Job+Outputs"></a>
<h5>Job Outputs</h5>
<p>Applications can control compression of job-outputs via the
<a href="api/org/apache/hadoop/mapred/FileOutputFormat.html#setCompressOutput(org.apache.hadoop.mapred.JobConf,%20boolean)">
@@ -2591,7 +2593,7 @@
<a href="api/org/apache/hadoop/mapred/SequenceFileOutputFormat.html#setOutputCompressionType(org.apache.hadoop.mapred.JobConf,%20org.apache.hadoop.io.SequenceFile.CompressionType)">
SequenceFileOutputFormat.setOutputCompressionType(JobConf,
SequenceFile.CompressionType)</a> api.</p>
-<a name="N10F4E"></a><a name="Skipping+Bad+Records"></a>
+<a name="N10F46"></a><a name="Skipping+Bad+Records"></a>
<h4>Skipping Bad Records</h4>
<p>Hadoop provides an optional mode of execution in which the bad
records are detected and skipped in further attempts.
@@ -2665,7 +2667,7 @@
</div>
-<a name="N10F98"></a><a name="Example%3A+WordCount+v2.0"></a>
+<a name="N10F90"></a><a name="Example%3A+WordCount+v2.0"></a>
<h2 class="h3">Example: WordCount v2.0</h2>
<div class="section">
<p>Here is a more complete <span class="codefrag">WordCount</span> which uses many of the
@@ -2675,7 +2677,7 @@
<a href="quickstart.html#SingleNodeSetup">pseudo-distributed</a> or
<a href="quickstart.html#Fully-Distributed+Operation">fully-distributed</a>
Hadoop installation.</p>
-<a name="N10FB2"></a><a name="Source+Code-N10FB2"></a>
+<a name="N10FAA"></a><a name="Source+Code-N10FAA"></a>
<h3 class="h4">Source Code</h3>
<table class="ForrestTable" cellspacing="1" cellpadding="4">
@@ -3885,7 +3887,7 @@
</tr>
</table>
-<a name="N11714"></a><a name="Sample+Runs"></a>
+<a name="N1170C"></a><a name="Sample+Runs"></a>
<h3 class="h4">Sample Runs</h3>
<p>Sample text-files as input:</p>
<p>
@@ -4053,7 +4055,7 @@
<br>
</p>
-<a name="N117E8"></a><a name="Highlights"></a>
+<a name="N117E0"></a><a name="Highlights"></a>
<h3 class="h4">Highlights</h3>
<p>The second version of <span class="codefrag">WordCount</span> improves upon the
previous one by using some features offered by the Map/Reduce framework: