You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by jun zhang <zh...@gmail.com> on 2013/02/04 08:15:27 UTC
Use vaidya but error in parsing conf file
I’m try to use vaidya to check my mr job, but always get the error
info like the below
what's the home here? Need I setting any things
./vaidya_new.sh -jobconf
file://home/jt1_1359122958375_job_201301252209_1384_conf.xml -joblog
file://home/job_201301252209_1384_1359959201318_b -testconf
/opt/hadoop/contrib/vaidya/conf/postex_diagnosis_tests.xml -report
./report.xml
13/02/04 15:06:04 FATAL conf.Configuration: error parsing conf file:
java.net.UnknownHostException: home
Exception:java.lang.RuntimeException: java.net.UnknownHostException:
homejava.lang.RuntimeException: java.net.UnknownHostException: home
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1395)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1269)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1200)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:501)
at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:131)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:242)
at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:225)
at org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.readJobInformation(PostExPerformanceDiagnoser.java:138)
at org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.<init>(PostExPerformanceDiagnoser.java:112)
at org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.main(PostExPerformanceDiagnoser.java:220)
Caused by: java.net.UnknownHostException: home
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
at sun.net.NetworkClient.openServer(NetworkClient.java:118)
at sun.net.ftp.FtpClient.openServer(FtpClient.java:488)
at sun.net.ftp.FtpClient.openServer(FtpClient.java:475)
at sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:270)
at sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:352)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1300)
... 9 more
Re: Use vaidya but error in parsing conf file
Posted by jun zhang <zh...@gmail.com>.
You can find it with google "vaidya github hadoop"
link is https://github.com/facebook/hadoop-20/tree/master/src/contrib/vaidya
But these is only 5 rules will be checked.
It was not as useful as I wished.
And my problem is fixed by change file://home to file:/home
2013/2/5 Dhanasekaran Anbalagan <bu...@gmail.com>
> Hi jun,
>
> I am very much interested with vaidya project. to analysis the mapreduce
> job, output. I read some weblinks, We have already using CDH4, where you
> can get from source vaidya. Please guide me How to test my MR jon to vaidya.
>
> -Dhanasekaran
>
> Did I learn something today? If not, I wasted it.
>
>
> On Mon, Feb 4, 2013 at 2:15 AM, jun zhang <zh...@gmail.com>wrote:
>
>> I’m try to use vaidya to check my mr job, but always get the error
>> info like the below
>>
>> what's the home here? Need I setting any things
>>
>> ./vaidya_new.sh -jobconf
>> file://home/jt1_1359122958375_job_201301252209_1384_conf.xml -joblog
>> file://home/job_201301252209_1384_1359959201318_b -testconf
>> /opt/hadoop/contrib/vaidya/conf/postex_diagnosis_tests.xml -report
>> ./report.xml
>>
>> 13/02/04 15:06:04 FATAL conf.Configuration: error parsing conf file:
>> java.net.UnknownHostException: home
>> Exception:java.lang.RuntimeException: java.net.UnknownHostException:
>> homejava.lang.RuntimeException: java.net.UnknownHostException: home
>> at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1395)
>> at
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1269)
>> at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1200)
>> at
>> org.apache.hadoop.conf.Configuration.get(Configuration.java:501)
>> at
>> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:131)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:242)
>> at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:225)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.readJobInformation(PostExPerformanceDiagnoser.java:138)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.<init>(PostExPerformanceDiagnoser.java:112)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.main(PostExPerformanceDiagnoser.java:220)
>> Caused by: java.net.UnknownHostException: home
>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>> at java.net.Socket.connect(Socket.java:529)
>> at java.net.Socket.connect(Socket.java:478)
>> at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
>> at sun.net.NetworkClient.openServer(NetworkClient.java:118)
>> at sun.net.ftp.FtpClient.openServer(FtpClient.java:488)
>> at sun.net.ftp.FtpClient.openServer(FtpClient.java:475)
>> at
>> sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:270)
>> at
>> sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:352)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
>> at
>> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
>> at
>> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
>> at
>> javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
>> at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1300)
>> ... 9 more
>>
>
>
Re: HyperThreading in TaskTracker nodes?
Posted by Todd Lipcon <to...@cloudera.com>.
Power issues aside, I've seen similar sorts of performance gains for MR
workloads - around 15-20%.
I think a fair bit of it is due to poor CPU cache utilization in various
parts of Hadoop - hyperthreading gets some extra parallelism there while
the core is waiting on round trips to DRAM.
-Todd
On Tue, Feb 5, 2013 at 10:03 AM, Brad Sarsfield <br...@bing.com> wrote:
> Hate to say it, but HyperThreading can have either positive or negative
> performance characteristics. It all depends on your workload. You have to
> measure very careful; it may not even be a bottleneck(!) :)
>
> I hit a pretty significant power issue when I enable HyperThreading at
> multi-thousand node scale. We hit a ~8-10% power utilization increase,
> which, if rolled out to the entire cluster, would put me a few %'ge over
> our max spec power. In this case, for our workload, we actually saw a 15%
> increase in processing throughput / job latency. We ended up literally
> turning off machines and enabling HyperThreading on the remaining and saw
> an overall ~10% efficiency gain in the cluster, with a few less machines,
> but running hot on power.
>
> ~Brad
>
> -----Original Message-----
> From: Terry Healy [mailto:thealy@bnl.gov]
> Sent: Tuesday, February 5, 2013 7:20 AM
> To: user@hadoop.apache.org
> Subject: HyperThreading in TaskTracker nodes?
>
> I would like to get some opinions / recommendations about the pros and
> cons of enabling HyperThreading on TaskTracker nodes. Presumably memory
> could be an issue, but is there anything to be gained, perhaps because of
> I/O wait? My small cluster is made of relatively slow and old systems,
> which mostly are quite slow to/from disk, if that matters.
>
> Thanks,
>
> Terry
>
>
>
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: HyperThreading in TaskTracker nodes?
Posted by Todd Lipcon <to...@cloudera.com>.
Power issues aside, I've seen similar sorts of performance gains for MR
workloads - around 15-20%.
I think a fair bit of it is due to poor CPU cache utilization in various
parts of Hadoop - hyperthreading gets some extra parallelism there while
the core is waiting on round trips to DRAM.
-Todd
On Tue, Feb 5, 2013 at 10:03 AM, Brad Sarsfield <br...@bing.com> wrote:
> Hate to say it, but HyperThreading can have either positive or negative
> performance characteristics. It all depends on your workload. You have to
> measure very careful; it may not even be a bottleneck(!) :)
>
> I hit a pretty significant power issue when I enable HyperThreading at
> multi-thousand node scale. We hit a ~8-10% power utilization increase,
> which, if rolled out to the entire cluster, would put me a few %'ge over
> our max spec power. In this case, for our workload, we actually saw a 15%
> increase in processing throughput / job latency. We ended up literally
> turning off machines and enabling HyperThreading on the remaining and saw
> an overall ~10% efficiency gain in the cluster, with a few less machines,
> but running hot on power.
>
> ~Brad
>
> -----Original Message-----
> From: Terry Healy [mailto:thealy@bnl.gov]
> Sent: Tuesday, February 5, 2013 7:20 AM
> To: user@hadoop.apache.org
> Subject: HyperThreading in TaskTracker nodes?
>
> I would like to get some opinions / recommendations about the pros and
> cons of enabling HyperThreading on TaskTracker nodes. Presumably memory
> could be an issue, but is there anything to be gained, perhaps because of
> I/O wait? My small cluster is made of relatively slow and old systems,
> which mostly are quite slow to/from disk, if that matters.
>
> Thanks,
>
> Terry
>
>
>
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: HyperThreading in TaskTracker nodes?
Posted by Todd Lipcon <to...@cloudera.com>.
Power issues aside, I've seen similar sorts of performance gains for MR
workloads - around 15-20%.
I think a fair bit of it is due to poor CPU cache utilization in various
parts of Hadoop - hyperthreading gets some extra parallelism there while
the core is waiting on round trips to DRAM.
-Todd
On Tue, Feb 5, 2013 at 10:03 AM, Brad Sarsfield <br...@bing.com> wrote:
> Hate to say it, but HyperThreading can have either positive or negative
> performance characteristics. It all depends on your workload. You have to
> measure very careful; it may not even be a bottleneck(!) :)
>
> I hit a pretty significant power issue when I enable HyperThreading at
> multi-thousand node scale. We hit a ~8-10% power utilization increase,
> which, if rolled out to the entire cluster, would put me a few %'ge over
> our max spec power. In this case, for our workload, we actually saw a 15%
> increase in processing throughput / job latency. We ended up literally
> turning off machines and enabling HyperThreading on the remaining and saw
> an overall ~10% efficiency gain in the cluster, with a few less machines,
> but running hot on power.
>
> ~Brad
>
> -----Original Message-----
> From: Terry Healy [mailto:thealy@bnl.gov]
> Sent: Tuesday, February 5, 2013 7:20 AM
> To: user@hadoop.apache.org
> Subject: HyperThreading in TaskTracker nodes?
>
> I would like to get some opinions / recommendations about the pros and
> cons of enabling HyperThreading on TaskTracker nodes. Presumably memory
> could be an issue, but is there anything to be gained, perhaps because of
> I/O wait? My small cluster is made of relatively slow and old systems,
> which mostly are quite slow to/from disk, if that matters.
>
> Thanks,
>
> Terry
>
>
>
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: HyperThreading in TaskTracker nodes?
Posted by Todd Lipcon <to...@cloudera.com>.
Power issues aside, I've seen similar sorts of performance gains for MR
workloads - around 15-20%.
I think a fair bit of it is due to poor CPU cache utilization in various
parts of Hadoop - hyperthreading gets some extra parallelism there while
the core is waiting on round trips to DRAM.
-Todd
On Tue, Feb 5, 2013 at 10:03 AM, Brad Sarsfield <br...@bing.com> wrote:
> Hate to say it, but HyperThreading can have either positive or negative
> performance characteristics. It all depends on your workload. You have to
> measure very careful; it may not even be a bottleneck(!) :)
>
> I hit a pretty significant power issue when I enable HyperThreading at
> multi-thousand node scale. We hit a ~8-10% power utilization increase,
> which, if rolled out to the entire cluster, would put me a few %'ge over
> our max spec power. In this case, for our workload, we actually saw a 15%
> increase in processing throughput / job latency. We ended up literally
> turning off machines and enabling HyperThreading on the remaining and saw
> an overall ~10% efficiency gain in the cluster, with a few less machines,
> but running hot on power.
>
> ~Brad
>
> -----Original Message-----
> From: Terry Healy [mailto:thealy@bnl.gov]
> Sent: Tuesday, February 5, 2013 7:20 AM
> To: user@hadoop.apache.org
> Subject: HyperThreading in TaskTracker nodes?
>
> I would like to get some opinions / recommendations about the pros and
> cons of enabling HyperThreading on TaskTracker nodes. Presumably memory
> could be an issue, but is there anything to be gained, perhaps because of
> I/O wait? My small cluster is made of relatively slow and old systems,
> which mostly are quite slow to/from disk, if that matters.
>
> Thanks,
>
> Terry
>
>
>
>
--
Todd Lipcon
Software Engineer, Cloudera
RE: HyperThreading in TaskTracker nodes?
Posted by Brad Sarsfield <br...@bing.com>.
Hate to say it, but HyperThreading can have either positive or negative performance characteristics. It all depends on your workload. You have to measure very careful; it may not even be a bottleneck(!) :)
I hit a pretty significant power issue when I enable HyperThreading at multi-thousand node scale. We hit a ~8-10% power utilization increase, which, if rolled out to the entire cluster, would put me a few %'ge over our max spec power. In this case, for our workload, we actually saw a 15% increase in processing throughput / job latency. We ended up literally turning off machines and enabling HyperThreading on the remaining and saw an overall ~10% efficiency gain in the cluster, with a few less machines, but running hot on power.
~Brad
-----Original Message-----
From: Terry Healy [mailto:thealy@bnl.gov]
Sent: Tuesday, February 5, 2013 7:20 AM
To: user@hadoop.apache.org
Subject: HyperThreading in TaskTracker nodes?
I would like to get some opinions / recommendations about the pros and cons of enabling HyperThreading on TaskTracker nodes. Presumably memory could be an issue, but is there anything to be gained, perhaps because of I/O wait? My small cluster is made of relatively slow and old systems, which mostly are quite slow to/from disk, if that matters.
Thanks,
Terry
RE: HyperThreading in TaskTracker nodes?
Posted by Brad Sarsfield <br...@bing.com>.
Hate to say it, but HyperThreading can have either positive or negative performance characteristics. It all depends on your workload. You have to measure very careful; it may not even be a bottleneck(!) :)
I hit a pretty significant power issue when I enable HyperThreading at multi-thousand node scale. We hit a ~8-10% power utilization increase, which, if rolled out to the entire cluster, would put me a few %'ge over our max spec power. In this case, for our workload, we actually saw a 15% increase in processing throughput / job latency. We ended up literally turning off machines and enabling HyperThreading on the remaining and saw an overall ~10% efficiency gain in the cluster, with a few less machines, but running hot on power.
~Brad
-----Original Message-----
From: Terry Healy [mailto:thealy@bnl.gov]
Sent: Tuesday, February 5, 2013 7:20 AM
To: user@hadoop.apache.org
Subject: HyperThreading in TaskTracker nodes?
I would like to get some opinions / recommendations about the pros and cons of enabling HyperThreading on TaskTracker nodes. Presumably memory could be an issue, but is there anything to be gained, perhaps because of I/O wait? My small cluster is made of relatively slow and old systems, which mostly are quite slow to/from disk, if that matters.
Thanks,
Terry
RE: HyperThreading in TaskTracker nodes?
Posted by Brad Sarsfield <br...@bing.com>.
Hate to say it, but HyperThreading can have either positive or negative performance characteristics. It all depends on your workload. You have to measure very careful; it may not even be a bottleneck(!) :)
I hit a pretty significant power issue when I enable HyperThreading at multi-thousand node scale. We hit a ~8-10% power utilization increase, which, if rolled out to the entire cluster, would put me a few %'ge over our max spec power. In this case, for our workload, we actually saw a 15% increase in processing throughput / job latency. We ended up literally turning off machines and enabling HyperThreading on the remaining and saw an overall ~10% efficiency gain in the cluster, with a few less machines, but running hot on power.
~Brad
-----Original Message-----
From: Terry Healy [mailto:thealy@bnl.gov]
Sent: Tuesday, February 5, 2013 7:20 AM
To: user@hadoop.apache.org
Subject: HyperThreading in TaskTracker nodes?
I would like to get some opinions / recommendations about the pros and cons of enabling HyperThreading on TaskTracker nodes. Presumably memory could be an issue, but is there anything to be gained, perhaps because of I/O wait? My small cluster is made of relatively slow and old systems, which mostly are quite slow to/from disk, if that matters.
Thanks,
Terry
RE: HyperThreading in TaskTracker nodes?
Posted by Brad Sarsfield <br...@bing.com>.
Hate to say it, but HyperThreading can have either positive or negative performance characteristics. It all depends on your workload. You have to measure very careful; it may not even be a bottleneck(!) :)
I hit a pretty significant power issue when I enable HyperThreading at multi-thousand node scale. We hit a ~8-10% power utilization increase, which, if rolled out to the entire cluster, would put me a few %'ge over our max spec power. In this case, for our workload, we actually saw a 15% increase in processing throughput / job latency. We ended up literally turning off machines and enabling HyperThreading on the remaining and saw an overall ~10% efficiency gain in the cluster, with a few less machines, but running hot on power.
~Brad
-----Original Message-----
From: Terry Healy [mailto:thealy@bnl.gov]
Sent: Tuesday, February 5, 2013 7:20 AM
To: user@hadoop.apache.org
Subject: HyperThreading in TaskTracker nodes?
I would like to get some opinions / recommendations about the pros and cons of enabling HyperThreading on TaskTracker nodes. Presumably memory could be an issue, but is there anything to be gained, perhaps because of I/O wait? My small cluster is made of relatively slow and old systems, which mostly are quite slow to/from disk, if that matters.
Thanks,
Terry
HyperThreading in TaskTracker nodes?
Posted by Terry Healy <th...@bnl.gov>.
I would like to get some opinions / recommendations about the pros and
cons of enabling HyperThreading on TaskTracker nodes. Presumably memory
could be an issue, but is there anything to be gained, perhaps because
of I/O wait? My small cluster is made of relatively slow and old
systems, which mostly are quite slow to/from disk, if that matters.
Thanks,
Terry
Re: Use vaidya but error in parsing conf file
Posted by jun zhang <zh...@gmail.com>.
You can find it with google "vaidya github hadoop"
link is https://github.com/facebook/hadoop-20/tree/master/src/contrib/vaidya
But these is only 5 rules will be checked.
It was not as useful as I wished.
And my problem is fixed by change file://home to file:/home
2013/2/5 Dhanasekaran Anbalagan <bu...@gmail.com>
> Hi jun,
>
> I am very much interested with vaidya project. to analysis the mapreduce
> job, output. I read some weblinks, We have already using CDH4, where you
> can get from source vaidya. Please guide me How to test my MR jon to vaidya.
>
> -Dhanasekaran
>
> Did I learn something today? If not, I wasted it.
>
>
> On Mon, Feb 4, 2013 at 2:15 AM, jun zhang <zh...@gmail.com>wrote:
>
>> I’m try to use vaidya to check my mr job, but always get the error
>> info like the below
>>
>> what's the home here? Need I setting any things
>>
>> ./vaidya_new.sh -jobconf
>> file://home/jt1_1359122958375_job_201301252209_1384_conf.xml -joblog
>> file://home/job_201301252209_1384_1359959201318_b -testconf
>> /opt/hadoop/contrib/vaidya/conf/postex_diagnosis_tests.xml -report
>> ./report.xml
>>
>> 13/02/04 15:06:04 FATAL conf.Configuration: error parsing conf file:
>> java.net.UnknownHostException: home
>> Exception:java.lang.RuntimeException: java.net.UnknownHostException:
>> homejava.lang.RuntimeException: java.net.UnknownHostException: home
>> at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1395)
>> at
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1269)
>> at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1200)
>> at
>> org.apache.hadoop.conf.Configuration.get(Configuration.java:501)
>> at
>> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:131)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:242)
>> at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:225)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.readJobInformation(PostExPerformanceDiagnoser.java:138)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.<init>(PostExPerformanceDiagnoser.java:112)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.main(PostExPerformanceDiagnoser.java:220)
>> Caused by: java.net.UnknownHostException: home
>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>> at java.net.Socket.connect(Socket.java:529)
>> at java.net.Socket.connect(Socket.java:478)
>> at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
>> at sun.net.NetworkClient.openServer(NetworkClient.java:118)
>> at sun.net.ftp.FtpClient.openServer(FtpClient.java:488)
>> at sun.net.ftp.FtpClient.openServer(FtpClient.java:475)
>> at
>> sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:270)
>> at
>> sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:352)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
>> at
>> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
>> at
>> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
>> at
>> javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
>> at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1300)
>> ... 9 more
>>
>
>
Re: Use vaidya but error in parsing conf file
Posted by jun zhang <zh...@gmail.com>.
You can find it with google "vaidya github hadoop"
link is https://github.com/facebook/hadoop-20/tree/master/src/contrib/vaidya
But these is only 5 rules will be checked.
It was not as useful as I wished.
And my problem is fixed by change file://home to file:/home
2013/2/5 Dhanasekaran Anbalagan <bu...@gmail.com>
> Hi jun,
>
> I am very much interested with vaidya project. to analysis the mapreduce
> job, output. I read some weblinks, We have already using CDH4, where you
> can get from source vaidya. Please guide me How to test my MR jon to vaidya.
>
> -Dhanasekaran
>
> Did I learn something today? If not, I wasted it.
>
>
> On Mon, Feb 4, 2013 at 2:15 AM, jun zhang <zh...@gmail.com>wrote:
>
>> I’m try to use vaidya to check my mr job, but always get the error
>> info like the below
>>
>> what's the home here? Need I setting any things
>>
>> ./vaidya_new.sh -jobconf
>> file://home/jt1_1359122958375_job_201301252209_1384_conf.xml -joblog
>> file://home/job_201301252209_1384_1359959201318_b -testconf
>> /opt/hadoop/contrib/vaidya/conf/postex_diagnosis_tests.xml -report
>> ./report.xml
>>
>> 13/02/04 15:06:04 FATAL conf.Configuration: error parsing conf file:
>> java.net.UnknownHostException: home
>> Exception:java.lang.RuntimeException: java.net.UnknownHostException:
>> homejava.lang.RuntimeException: java.net.UnknownHostException: home
>> at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1395)
>> at
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1269)
>> at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1200)
>> at
>> org.apache.hadoop.conf.Configuration.get(Configuration.java:501)
>> at
>> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:131)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:242)
>> at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:225)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.readJobInformation(PostExPerformanceDiagnoser.java:138)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.<init>(PostExPerformanceDiagnoser.java:112)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.main(PostExPerformanceDiagnoser.java:220)
>> Caused by: java.net.UnknownHostException: home
>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>> at java.net.Socket.connect(Socket.java:529)
>> at java.net.Socket.connect(Socket.java:478)
>> at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
>> at sun.net.NetworkClient.openServer(NetworkClient.java:118)
>> at sun.net.ftp.FtpClient.openServer(FtpClient.java:488)
>> at sun.net.ftp.FtpClient.openServer(FtpClient.java:475)
>> at
>> sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:270)
>> at
>> sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:352)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
>> at
>> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
>> at
>> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
>> at
>> javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
>> at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1300)
>> ... 9 more
>>
>
>
HyperThreading in TaskTracker nodes?
Posted by Terry Healy <th...@bnl.gov>.
I would like to get some opinions / recommendations about the pros and
cons of enabling HyperThreading on TaskTracker nodes. Presumably memory
could be an issue, but is there anything to be gained, perhaps because
of I/O wait? My small cluster is made of relatively slow and old
systems, which mostly are quite slow to/from disk, if that matters.
Thanks,
Terry
HyperThreading in TaskTracker nodes?
Posted by Terry Healy <th...@bnl.gov>.
I would like to get some opinions / recommendations about the pros and
cons of enabling HyperThreading on TaskTracker nodes. Presumably memory
could be an issue, but is there anything to be gained, perhaps because
of I/O wait? My small cluster is made of relatively slow and old
systems, which mostly are quite slow to/from disk, if that matters.
Thanks,
Terry
Re: Use vaidya but error in parsing conf file
Posted by jun zhang <zh...@gmail.com>.
You can find it with google "vaidya github hadoop"
link is https://github.com/facebook/hadoop-20/tree/master/src/contrib/vaidya
But these is only 5 rules will be checked.
It was not as useful as I wished.
And my problem is fixed by change file://home to file:/home
2013/2/5 Dhanasekaran Anbalagan <bu...@gmail.com>
> Hi jun,
>
> I am very much interested with vaidya project. to analysis the mapreduce
> job, output. I read some weblinks, We have already using CDH4, where you
> can get from source vaidya. Please guide me How to test my MR jon to vaidya.
>
> -Dhanasekaran
>
> Did I learn something today? If not, I wasted it.
>
>
> On Mon, Feb 4, 2013 at 2:15 AM, jun zhang <zh...@gmail.com>wrote:
>
>> I’m try to use vaidya to check my mr job, but always get the error
>> info like the below
>>
>> what's the home here? Need I setting any things
>>
>> ./vaidya_new.sh -jobconf
>> file://home/jt1_1359122958375_job_201301252209_1384_conf.xml -joblog
>> file://home/job_201301252209_1384_1359959201318_b -testconf
>> /opt/hadoop/contrib/vaidya/conf/postex_diagnosis_tests.xml -report
>> ./report.xml
>>
>> 13/02/04 15:06:04 FATAL conf.Configuration: error parsing conf file:
>> java.net.UnknownHostException: home
>> Exception:java.lang.RuntimeException: java.net.UnknownHostException:
>> homejava.lang.RuntimeException: java.net.UnknownHostException: home
>> at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1395)
>> at
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1269)
>> at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1200)
>> at
>> org.apache.hadoop.conf.Configuration.get(Configuration.java:501)
>> at
>> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:131)
>> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:242)
>> at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:225)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.readJobInformation(PostExPerformanceDiagnoser.java:138)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.<init>(PostExPerformanceDiagnoser.java:112)
>> at
>> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.main(PostExPerformanceDiagnoser.java:220)
>> Caused by: java.net.UnknownHostException: home
>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>> at java.net.Socket.connect(Socket.java:529)
>> at java.net.Socket.connect(Socket.java:478)
>> at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
>> at sun.net.NetworkClient.openServer(NetworkClient.java:118)
>> at sun.net.ftp.FtpClient.openServer(FtpClient.java:488)
>> at sun.net.ftp.FtpClient.openServer(FtpClient.java:475)
>> at
>> sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:270)
>> at
>> sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:352)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
>> at
>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
>> at
>> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
>> at
>> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
>> at
>> javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
>> at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1300)
>> ... 9 more
>>
>
>
HyperThreading in TaskTracker nodes?
Posted by Terry Healy <th...@bnl.gov>.
I would like to get some opinions / recommendations about the pros and
cons of enabling HyperThreading on TaskTracker nodes. Presumably memory
could be an issue, but is there anything to be gained, perhaps because
of I/O wait? My small cluster is made of relatively slow and old
systems, which mostly are quite slow to/from disk, if that matters.
Thanks,
Terry
Re: Use vaidya but error in parsing conf file
Posted by Dhanasekaran Anbalagan <bu...@gmail.com>.
Hi jun,
I am very much interested with vaidya project. to analysis the mapreduce
job, output. I read some weblinks, We have already using CDH4, where you
can get from source vaidya. Please guide me How to test my MR jon to vaidya.
-Dhanasekaran
Did I learn something today? If not, I wasted it.
On Mon, Feb 4, 2013 at 2:15 AM, jun zhang <zh...@gmail.com> wrote:
> I’m try to use vaidya to check my mr job, but always get the error
> info like the below
>
> what's the home here? Need I setting any things
>
> ./vaidya_new.sh -jobconf
> file://home/jt1_1359122958375_job_201301252209_1384_conf.xml -joblog
> file://home/job_201301252209_1384_1359959201318_b -testconf
> /opt/hadoop/contrib/vaidya/conf/postex_diagnosis_tests.xml -report
> ./report.xml
>
> 13/02/04 15:06:04 FATAL conf.Configuration: error parsing conf file:
> java.net.UnknownHostException: home
> Exception:java.lang.RuntimeException: java.net.UnknownHostException:
> homejava.lang.RuntimeException: java.net.UnknownHostException: home
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1395)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1269)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1200)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:501)
> at
> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:131)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:242)
> at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:225)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.readJobInformation(PostExPerformanceDiagnoser.java:138)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.<init>(PostExPerformanceDiagnoser.java:112)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.main(PostExPerformanceDiagnoser.java:220)
> Caused by: java.net.UnknownHostException: home
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:529)
> at java.net.Socket.connect(Socket.java:478)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
> at sun.net.NetworkClient.openServer(NetworkClient.java:118)
> at sun.net.ftp.FtpClient.openServer(FtpClient.java:488)
> at sun.net.ftp.FtpClient.openServer(FtpClient.java:475)
> at
> sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:270)
> at
> sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:352)
> at
> com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
> at
> com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at
> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at
> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at
> javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1300)
> ... 9 more
>
Re: Use vaidya but error in parsing conf file
Posted by Dhanasekaran Anbalagan <bu...@gmail.com>.
Hi jun,
I am very much interested with vaidya project. to analysis the mapreduce
job, output. I read some weblinks, We have already using CDH4, where you
can get from source vaidya. Please guide me How to test my MR jon to vaidya.
-Dhanasekaran
Did I learn something today? If not, I wasted it.
On Mon, Feb 4, 2013 at 2:15 AM, jun zhang <zh...@gmail.com> wrote:
> I’m try to use vaidya to check my mr job, but always get the error
> info like the below
>
> what's the home here? Need I setting any things
>
> ./vaidya_new.sh -jobconf
> file://home/jt1_1359122958375_job_201301252209_1384_conf.xml -joblog
> file://home/job_201301252209_1384_1359959201318_b -testconf
> /opt/hadoop/contrib/vaidya/conf/postex_diagnosis_tests.xml -report
> ./report.xml
>
> 13/02/04 15:06:04 FATAL conf.Configuration: error parsing conf file:
> java.net.UnknownHostException: home
> Exception:java.lang.RuntimeException: java.net.UnknownHostException:
> homejava.lang.RuntimeException: java.net.UnknownHostException: home
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1395)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1269)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1200)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:501)
> at
> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:131)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:242)
> at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:225)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.readJobInformation(PostExPerformanceDiagnoser.java:138)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.<init>(PostExPerformanceDiagnoser.java:112)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.main(PostExPerformanceDiagnoser.java:220)
> Caused by: java.net.UnknownHostException: home
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:529)
> at java.net.Socket.connect(Socket.java:478)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
> at sun.net.NetworkClient.openServer(NetworkClient.java:118)
> at sun.net.ftp.FtpClient.openServer(FtpClient.java:488)
> at sun.net.ftp.FtpClient.openServer(FtpClient.java:475)
> at
> sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:270)
> at
> sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:352)
> at
> com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
> at
> com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at
> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at
> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at
> javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1300)
> ... 9 more
>
Re: Use vaidya but error in parsing conf file
Posted by Dhanasekaran Anbalagan <bu...@gmail.com>.
Hi jun,
I am very much interested with vaidya project. to analysis the mapreduce
job, output. I read some weblinks, We have already using CDH4, where you
can get from source vaidya. Please guide me How to test my MR jon to vaidya.
-Dhanasekaran
Did I learn something today? If not, I wasted it.
On Mon, Feb 4, 2013 at 2:15 AM, jun zhang <zh...@gmail.com> wrote:
> I’m try to use vaidya to check my mr job, but always get the error
> info like the below
>
> what's the home here? Need I setting any things
>
> ./vaidya_new.sh -jobconf
> file://home/jt1_1359122958375_job_201301252209_1384_conf.xml -joblog
> file://home/job_201301252209_1384_1359959201318_b -testconf
> /opt/hadoop/contrib/vaidya/conf/postex_diagnosis_tests.xml -report
> ./report.xml
>
> 13/02/04 15:06:04 FATAL conf.Configuration: error parsing conf file:
> java.net.UnknownHostException: home
> Exception:java.lang.RuntimeException: java.net.UnknownHostException:
> homejava.lang.RuntimeException: java.net.UnknownHostException: home
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1395)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1269)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1200)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:501)
> at
> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:131)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:242)
> at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:225)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.readJobInformation(PostExPerformanceDiagnoser.java:138)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.<init>(PostExPerformanceDiagnoser.java:112)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.main(PostExPerformanceDiagnoser.java:220)
> Caused by: java.net.UnknownHostException: home
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:529)
> at java.net.Socket.connect(Socket.java:478)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
> at sun.net.NetworkClient.openServer(NetworkClient.java:118)
> at sun.net.ftp.FtpClient.openServer(FtpClient.java:488)
> at sun.net.ftp.FtpClient.openServer(FtpClient.java:475)
> at
> sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:270)
> at
> sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:352)
> at
> com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
> at
> com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at
> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at
> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at
> javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1300)
> ... 9 more
>
Re: Use vaidya but error in parsing conf file
Posted by Dhanasekaran Anbalagan <bu...@gmail.com>.
Hi jun,
I am very much interested with vaidya project. to analysis the mapreduce
job, output. I read some weblinks, We have already using CDH4, where you
can get from source vaidya. Please guide me How to test my MR jon to vaidya.
-Dhanasekaran
Did I learn something today? If not, I wasted it.
On Mon, Feb 4, 2013 at 2:15 AM, jun zhang <zh...@gmail.com> wrote:
> I’m try to use vaidya to check my mr job, but always get the error
> info like the below
>
> what's the home here? Need I setting any things
>
> ./vaidya_new.sh -jobconf
> file://home/jt1_1359122958375_job_201301252209_1384_conf.xml -joblog
> file://home/job_201301252209_1384_1359959201318_b -testconf
> /opt/hadoop/contrib/vaidya/conf/postex_diagnosis_tests.xml -report
> ./report.xml
>
> 13/02/04 15:06:04 FATAL conf.Configuration: error parsing conf file:
> java.net.UnknownHostException: home
> Exception:java.lang.RuntimeException: java.net.UnknownHostException:
> homejava.lang.RuntimeException: java.net.UnknownHostException: home
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1395)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1269)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1200)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:501)
> at
> org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:131)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:242)
> at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:225)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.readJobInformation(PostExPerformanceDiagnoser.java:138)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.<init>(PostExPerformanceDiagnoser.java:112)
> at
> org.apache.hadoop.vaidya.postexdiagnosis.PostExPerformanceDiagnoser.main(PostExPerformanceDiagnoser.java:220)
> Caused by: java.net.UnknownHostException: home
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:529)
> at java.net.Socket.connect(Socket.java:478)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
> at sun.net.NetworkClient.openServer(NetworkClient.java:118)
> at sun.net.ftp.FtpClient.openServer(FtpClient.java:488)
> at sun.net.ftp.FtpClient.openServer(FtpClient.java:475)
> at
> sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:270)
> at
> sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:352)
> at
> com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
> at
> com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at
> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> at
> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> at
> javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1300)
> ... 9 more
>