You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by cu...@apache.org on 2006/02/10 00:49:41 UTC
svn commit: r376492 -
/lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/JobConf.java
Author: cutting
Date: Thu Feb 9 15:49:39 2006
New Revision: 376492
URL: http://svn.apache.org/viewcvs?rev=376492&view=rev
Log:
Further improvements from Bryan A. Pendleton.
Modified:
lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/JobConf.java
Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/JobConf.java
URL: http://svn.apache.org/viewcvs/lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/JobConf.java?rev=376492&r1=376491&r2=376492&view=diff
==============================================================================
--- lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/JobConf.java (original)
+++ lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/JobConf.java Thu Feb 9 15:49:39 2006
@@ -69,7 +69,10 @@
*/
public JobConf(Configuration conf, Class aClass) {
this(conf);
- setJar(findContainingJar(aClass));
+ String jar = findContainingJar(aClass);
+ if (jar != null) {
+ setJar(jar);
+ }
}
@@ -302,7 +305,11 @@
itr.hasMoreElements();) {
URL url = (URL) itr.nextElement();
if ("jar".equals(url.getProtocol())) {
- return url.getPath().replaceFirst("file:", "").replaceAll("!.*$", "");
+ String toReturn = url.getPath();
+ if (toReturn.startsWith("file:")) {
+ toReturn = toReturn.substring("file:".length());
+ }
+ return toReturn.replaceAll("!.*$", "");
}
}
} catch (IOException e) {
Re: svn commit: r376492 - /lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/JobConf.java
Posted by Owen O'Malley <ow...@yahoo-inc.com>.
On Feb 9, 2006, at 4:21 PM, Bryan A. Pendleton wrote:
> I agree that the first fix is better (handling the file: part of the
> URL).
>
> I don't quite agree about not setting a Jar file masking the problem -
> certainly, testing things out in debugging environments makes it not
> guaranteed that there's a Jar there. In any case, it would certainly be
> better to fail-fast there, but, at this point, fail-fast by obscure NPE
> isn't necessarily better. Is there something better?
Given the problems with error checking, I'd suggest making the
findContainingJar a public static method of JobConf, and letting the
user application call it explicitly. Otherwise, we would need to throw
an exception (RuntimeException?) with a message about the class not
being found in a jar file.
The only case where not having the jar file works is either that both
your map and reduce class are in hadoop.jar, which is how Grep worked,
or you are using the local runner. So, you should pretty much always
have a jar. *smile*
-- Owen
Re: svn commit: r376492 - /lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/JobConf.java
Posted by "Bryan A. Pendleton" <bp...@geekdom.net>.
I agree that the first fix is better (handling the file: part of the URL).
I don't quite agree about not setting a Jar file masking the problem -
certainly, testing things out in debugging environments makes it not
guaranteed that there's a Jar there. In any case, it would certainly be
better to fail-fast there, but, at this point, fail-fast by obscure NPE
isn't necessarily better. Is there something better?
On 2/9/06, Owen O'Malley <ow...@yahoo-inc.com> wrote:
>
> Actually, I liked the first fix for the java 1.4 problem better (489
> rather than 492), since it was shorter and more concise.
>
> Furthermore, I think that it is a serious mistake to mask the problem
> of not finding the jar file. For map/reduce, the application will fail
> without the jar set. Having it fail at the right spot in the driver is
> much better than having it fail in the task tracker with Class Not
> Found.
>
> -- Owen
>
>
--
Bryan A. Pendleton
Ph: (877) geek-1-bp
Re: svn commit: r376492 - /lucene/hadoop/trunk/src/java/org/apache/hadoop/mapred/JobConf.java
Posted by Owen O'Malley <ow...@yahoo-inc.com>.
Actually, I liked the first fix for the java 1.4 problem better (489
rather than 492), since it was shorter and more concise.
Furthermore, I think that it is a serious mistake to mask the problem
of not finding the jar file. For map/reduce, the application will fail
without the jar set. Having it fail at the right spot in the driver is
much better than having it fail in the task tracker with Class Not
Found.
-- Owen