You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by bilwa st <st...@gmail.com> on 2019/11/22 13:08:49 UTC

Performance issue

Hi,

Tera sort is slow by 9mins(26%) in 3.1.1 compared to 2.7.2

I could see some JIRA open for the same issue ie MAPREDUCE-6735

Has anyone else faced the same issue?

Thanks,
Bilwa ST

Re: Performance issue

Posted by HarshaKiran Reddy Boreddy <bh...@gmail.com>.
I used same environment and same configurations..Not used EC..

On Friday, December 13, 2019, Julien Laurenceau <
julien.laurenceau@pepitedata.com> wrote:

> Hi
> Could you please share more details ?
> We can assume it is the same hardware and same OS / JVM.
> What about the hadoop configuration ? Are you using Erasure Coding ?
>
> Regards
>
>
> Le jeu. 12 déc. 2019 à 14:12, HarshaKiran Reddy Boreddy <
> bharshait@gmail.com> a écrit :
>
>> Yes, Even in HDFS(3.1.1), I have seen the performance degradation
>> compared with 2.7.2. I have done the DFSIO test following are the details.
>>
>> Test:
>>
>> DATA SIZE: 1TB
>> No of Files : 1000
>> File size : 1GB
>>
>> *Hadoop-3.1.1::*
>>
>> *DFSIO Write : 1420.99 sec*
>>
>> *Hadoop-2.7.2::*
>>
>> *DFSIO Write : 1297.971 sec*
>>
>> so,there is degradation compare to 2.7.2..Any pointers to this??
>>
>> Thanks
>> Harshakiran Reddy
>>
>>
>>
>>
>> From: bilwa st <st...@gmail.com>
>> Date: Fri, 22 Nov 2019 at 9:09 PM
>> Subject: Performance issue
>> To: <us...@hadoop.apache.org>
>>
>>
>> Hi,
>>
>> Tera sort is slow by 9mins(26%) in 3.1.1 compared to 2.7.2
>>
>> I could see some JIRA open for the same issue ie MAPREDUCE-6735
>>
>> Has anyone else faced the same issue?
>>
>> Thanks,
>> Bilwa ST
>>
>>
>>
>> --
>> *Regards*
>> Harshakiranreddy.Boreddy
>>
>>
>>

-- 
*Regards*
Harshakiranreddy.Boreddy

Re: Performance issue

Posted by Julien Laurenceau <ju...@pepitedata.com>.
Hi
Could you please share more details ?
We can assume it is the same hardware and same OS / JVM.
What about the hadoop configuration ? Are you using Erasure Coding ?

Regards


Le jeu. 12 déc. 2019 à 14:12, HarshaKiran Reddy Boreddy <bh...@gmail.com>
a écrit :

> Yes, Even in HDFS(3.1.1), I have seen the performance degradation compared
> with 2.7.2. I have done the DFSIO test following are the details.
>
> Test:
>
> DATA SIZE: 1TB
> No of Files : 1000
> File size : 1GB
>
> *Hadoop-3.1.1::*
>
> *DFSIO Write : 1420.99 sec*
>
> *Hadoop-2.7.2::*
>
> *DFSIO Write : 1297.971 sec*
>
> so,there is degradation compare to 2.7.2..Any pointers to this??
>
> Thanks
> Harshakiran Reddy
>
>
>
>
> From: bilwa st <st...@gmail.com>
> Date: Fri, 22 Nov 2019 at 9:09 PM
> Subject: Performance issue
> To: <us...@hadoop.apache.org>
>
>
> Hi,
>
> Tera sort is slow by 9mins(26%) in 3.1.1 compared to 2.7.2
>
> I could see some JIRA open for the same issue ie MAPREDUCE-6735
>
> Has anyone else faced the same issue?
>
> Thanks,
> Bilwa ST
>
>
>
> --
> *Regards*
> Harshakiranreddy.Boreddy
>
>
>

Re: Performance issue

Posted by HarshaKiran Reddy Boreddy <bh...@gmail.com>.
Yes, Even in HDFS(3.1.1), I have seen the performance degradation compared
with 2.7.2. I have done the DFSIO test following are the details.

Test:

DATA SIZE: 1TB
No of Files : 1000
File size : 1GB

*Hadoop-3.1.1::*

*DFSIO Write : 1420.99 sec*

*Hadoop-2.7.2::*

*DFSIO Write : 1297.971 sec*

so,there is degradation compare to 2.7.2..Any pointers to this??

Thanks
Harshakiran Reddy




From: bilwa st <st...@gmail.com>
Date: Fri, 22 Nov 2019 at 9:09 PM
Subject: Performance issue
To: <us...@hadoop.apache.org>


Hi,

Tera sort is slow by 9mins(26%) in 3.1.1 compared to 2.7.2

I could see some JIRA open for the same issue ie MAPREDUCE-6735

Has anyone else faced the same issue?

Thanks,
Bilwa ST



-- 
*Regards*
Harshakiranreddy.Boreddy