You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by "real great.." <gr...@gmail.com> on 2011/04/20 11:28:53 UTC
Execution time.
Hi,
I had asked a question about predicting map times in hadoop.
Thanks a lot for the encouraging response.
I want to know if anybody has a code or any idea on how to calculate the
execution time? I mean a small estimation.q
--
Regards,
R.V.
Re: Execution time.
Posted by "real great.." <gr...@gmail.com>.
Am concentrating on block placement strategy. And the second point is what
am involved in.
On Wed, Apr 27, 2011 at 3:28 PM, Steve Loughran <st...@apache.org> wrote:
> On 26/1 14:16, real great.. wrote:
>
>> Thanks a lot.I have managed to do it.
>> And my final year project is on power aware Hadoop. i do realise its
>> against
>> ethics to get the code that way..:)
>>
>
> Good.
>
> What do you mean by power aware
>
> -awareness of the topology of UPS sources inside a datacentre
> -awareness of CPU voltage level/power drain to schedule work where CPUs are
> capable of being most efficiently used, rather than scheduling work on a CPU
> that will have to ramp up to its full voltage and so be least efficient?
>
> either would be interesting. You could use the existing rack topology
> scripts for UPS topology, but really there should be two topologies, as it's
> block placement where you need the UPS topology
>
--
Regards,
R.V.
Re: Execution time.
Posted by Steve Loughran <st...@apache.org>.
On 26/04/11 14:16, real great.. wrote:
> Thanks a lot.I have managed to do it.
> And my final year project is on power aware Hadoop. i do realise its against
> ethics to get the code that way..:)
Good.
What do you mean by power aware
-awareness of the topology of UPS sources inside a datacentre
-awareness of CPU voltage level/power drain to schedule work where CPUs
are capable of being most efficiently used, rather than scheduling work
on a CPU that will have to ramp up to its full voltage and so be least
efficient?
either would be interesting. You could use the existing rack topology
scripts for UPS topology, but really there should be two topologies, as
it's block placement where you need the UPS topology
Re: Execution time.
Posted by "real great.." <gr...@gmail.com>.
Thanks a lot.I have managed to do it.
And my final year project is on power aware Hadoop. i do realise its against
ethics to get the code that way..:)
On Tue, Apr 26, 2011 at 4:24 PM, Steve Loughran <st...@apache.org> wrote:
> On 20/04/11 10:28, real great.. wrote:
>
>> Hi,
>> I had asked a question about predicting map times in hadoop.
>> Thanks a lot for the encouraging response.
>> I want to know if anybody has a code or any idea on how to calculate the
>> execution time? I mean a small estimation.q
>>
>>
>>
> 1. Surely this is what your final year project should do? It's one thing to
> provide pointers, but to give you code, isn't that, well, against the rules.
>
> 2. If you read the papers, the key finding is you can't estimate execution
> time except from historical data or some subsampling. You will need to write
> code to do one of these
>
--
Regards,
R.V.
Re: Execution time.
Posted by Steve Loughran <st...@apache.org>.
On 20/04/11 10:28, real great.. wrote:
> Hi,
> I had asked a question about predicting map times in hadoop.
> Thanks a lot for the encouraging response.
> I want to know if anybody has a code or any idea on how to calculate the
> execution time? I mean a small estimation.q
>
>
1. Surely this is what your final year project should do? It's one thing
to provide pointers, but to give you code, isn't that, well, against the
rules.
2. If you read the papers, the key finding is you can't estimate
execution time except from historical data or some subsampling. You will
need to write code to do one of these
RE: Execution time.
Posted by Viraj Bhat <vi...@yahoo-inc.com>.
Hi RV,
We have literature which does this in research from the University of Washington. Pig is planning to use this framework, in its progress indicators.
The below papers give you some idea of how to do this estimation.
http://www.cs.washington.edu/education/courses/cse544/11wi/examples/abe-kristi.pdf
ftp://ftp.cs.washington.edu/tr/2009/07/UW-CSE-09-07-01.PDF
Regards
Viraj
-----Original Message-----
From: real great.. [mailto:greatness.hardness@gmail.com]
Sent: Wednesday, April 20, 2011 2:29 AM
To: common-user
Subject: Execution time.
Hi,
I had asked a question about predicting map times in hadoop.
Thanks a lot for the encouraging response.
I want to know if anybody has a code or any idea on how to calculate the
execution time? I mean a small estimation.q
--
Regards,
R.V.