You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon" <ed...@apache.org> on 2011/06/08 07:22:29 UTC
[DISCUSS] Rearrange our roadmap
Hi all,
I would suggest that we have to focus on performance improvements and
basic communication parts of BSP framework. So I've little edited our
roadmap page.
http://wiki.apache.org/hama/RoadMap
If you have any comments or other opinions, let's discuss here.
Thanks.
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: [DISCUSS] Rearrange our roadmap
Posted by "Edward J. Yoon" <ed...@apache.org>.
I was trying to say ..
> I think this is a major need before
> advancing forward to fault tolerance and I/O things.
:D
On Wed, Jun 8, 2011 at 4:29 PM, Tommaso Teofili
<to...@gmail.com> wrote:
> 2011/6/8 Thomas Jungblut <th...@googlemail.com>
>
>> Feel free to profile it :)
>>
>> http://codingwiththomas.blogspot.com/2011/04/profiling-apache-hama-bsps.html
>>
>> +1 for the new roadmap of 0.3.0. I think this is a major need before
>> advancing forward to fault tolerance and I/O things.
>>
>
> correct, +1 from me too.
> Tommaso
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: [DISCUSS] Rearrange our roadmap
Posted by Tommaso Teofili <to...@gmail.com>.
2011/6/8 Thomas Jungblut <th...@googlemail.com>
> Feel free to profile it :)
>
> http://codingwiththomas.blogspot.com/2011/04/profiling-apache-hama-bsps.html
>
> +1 for the new roadmap of 0.3.0. I think this is a major need before
> advancing forward to fault tolerance and I/O things.
>
correct, +1 from me too.
Tommaso
Re: [DISCUSS] Rearrange our roadmap
Posted by Thomas Jungblut <th...@googlemail.com>.
Feel free to profile it :)
http://codingwiththomas.blogspot.com/2011/04/profiling-apache-hama-bsps.html
+1 for the new roadmap of 0.3.0. I think this is a major need before
advancing forward to fault tolerance and I/O things.
Best Regards
2011/6/8 ChiaHung Lin <ch...@nuk.edu.tw>:
> Regarding to performance improvement, http://wiki.apache.org/hama/Benchmarks contains some statistics. Is there any chance that we can pick up some points (data serialization, scheduling, etc.) to gauge in detail the time consumed (or how long the time elapses) by the framework? Also, some more parameters such as how the test is executed, etc. might help community to spot on something that may not work well or as expected.
>
> -----Original message-----
> From:Edward J. Yoon <ed...@apache.org>
> To:hama-dev@incubator.apache.org
> Date:Wed, 8 Jun 2011 14:22:29 +0900
> Subject:[DISCUSS] Rearrange our roadmap
>
> Hi all,
>
> I would suggest that we have to focus on performance improvements and
> basic communication parts of BSP framework. So I've little edited our
> roadmap page.
>
> http://wiki.apache.org/hama/RoadMap
>
> If you have any comments or other opinions, let's discuss here.
>
> Thanks.
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
>
> --
> ChiaHung Lin
> Department of Information Management
> National University of Kaohsiung
> Taiwan
>
--
Thomas Jungblut
Berlin
mobile: 0170-3081070
business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com
Re: [DISCUSS] Rearrange our roadmap
Posted by ChiaHung Lin <ch...@nuk.edu.tw>.
Regarding to performance improvement, http://wiki.apache.org/hama/Benchmarks contains some statistics. Is there any chance that we can pick up some points (data serialization, scheduling, etc.) to gauge in detail the time consumed (or how long the time elapses) by the framework? Also, some more parameters such as how the test is executed, etc. might help community to spot on something that may not work well or as expected.
-----Original message-----
From:Edward J. Yoon <ed...@apache.org>
To:hama-dev@incubator.apache.org
Date:Wed, 8 Jun 2011 14:22:29 +0900
Subject:[DISCUSS] Rearrange our roadmap
Hi all,
I would suggest that we have to focus on performance improvements and
basic communication parts of BSP framework. So I've little edited our
roadmap page.
http://wiki.apache.org/hama/RoadMap
If you have any comments or other opinions, let's discuss here.
Thanks.
--
Best Regards, Edward J. Yoon
@eddieyoon
--
ChiaHung Lin
Department of Information Management
National University of Kaohsiung
Taiwan
Re: [DISCUSS] Rearrange our roadmap
Posted by Thomas Jungblut <th...@googlemail.com>.
Without know what is inside MR-279. The "ϕ accrual failure detector"
sounds like a reasonable way to detect faults.
Can someone please point out what features the MR-279 branch has?
Links to Jira issues would be very cool.
Thanks in advance :)
2011/6/10 Edward J. Yoon <ed...@apache.org>:
> As far as I can recall, there was no conclusion.
>
> http://www.mail-archive.com/hama-dev@incubator.apache.org/msg03742.html
>
> and "ϕ accrual failure detector" also.
>
> On Fri, Jun 10, 2011 at 11:46 AM, ChiaHung Lin <ch...@nuk.edu.tw> wrote:
>> I remember there was discussion that Hama would not adapt to MR-279. Is this still the case?
>>
>> -----Original message-----
>> From:Steve Loughran <st...@apache.org>
>> To:hama-dev@incubator.apache.org
>> Date:Wed, 08 Jun 2011 12:15:14 +0100
>> Subject:Re: [DISCUSS] Rearrange our roadmap
>>
>> On 06/08/2011 06:22 AM, Edward J. Yoon wrote:
>>> Hi all,
>>>
>>> I would suggest that we have to focus on performance improvements and
>>> basic communication parts of BSP framework. So I've little edited our
>>> roadmap page.
>>>
>>> http://wiki.apache.org/hama/RoadMap
>>>
>>> If you have any comments or other opinions, let's discuss here.
>>>
>>> Thanks.
>>
>> I'd recommend targeting running the fault tolerant code under the MR-279
>> engine (hadoop 0.22 or .23?), as this will deploy more than just the MR
>> engine, and handle liveness -so you don't need to replicate core modules.
>>
>> the big thing for BSP engines is checkpointing, where the sync point
>> becomes the thing you can checkpoint.
>>
>>
>> --
>> ChiaHung Lin
>> Department of Information Management
>> National University of Kaohsiung
>> Taiwan
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
--
Thomas Jungblut
Berlin
mobile: 0170-3081070
business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com
Re: [DISCUSS] Rearrange our roadmap
Posted by "Edward J. Yoon" <ed...@apache.org>.
As far as I can recall, there was no conclusion.
http://www.mail-archive.com/hama-dev@incubator.apache.org/msg03742.html
and "ϕ accrual failure detector" also.
On Fri, Jun 10, 2011 at 11:46 AM, ChiaHung Lin <ch...@nuk.edu.tw> wrote:
> I remember there was discussion that Hama would not adapt to MR-279. Is this still the case?
>
> -----Original message-----
> From:Steve Loughran <st...@apache.org>
> To:hama-dev@incubator.apache.org
> Date:Wed, 08 Jun 2011 12:15:14 +0100
> Subject:Re: [DISCUSS] Rearrange our roadmap
>
> On 06/08/2011 06:22 AM, Edward J. Yoon wrote:
>> Hi all,
>>
>> I would suggest that we have to focus on performance improvements and
>> basic communication parts of BSP framework. So I've little edited our
>> roadmap page.
>>
>> http://wiki.apache.org/hama/RoadMap
>>
>> If you have any comments or other opinions, let's discuss here.
>>
>> Thanks.
>
> I'd recommend targeting running the fault tolerant code under the MR-279
> engine (hadoop 0.22 or .23?), as this will deploy more than just the MR
> engine, and handle liveness -so you don't need to replicate core modules.
>
> the big thing for BSP engines is checkpointing, where the sync point
> becomes the thing you can checkpoint.
>
>
> --
> ChiaHung Lin
> Department of Information Management
> National University of Kaohsiung
> Taiwan
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: [DISCUSS] Rearrange our roadmap
Posted by ChiaHung Lin <ch...@nuk.edu.tw>.
I remember there was discussion that Hama would not adapt to MR-279. Is this still the case?
-----Original message-----
From:Steve Loughran <st...@apache.org>
To:hama-dev@incubator.apache.org
Date:Wed, 08 Jun 2011 12:15:14 +0100
Subject:Re: [DISCUSS] Rearrange our roadmap
On 06/08/2011 06:22 AM, Edward J. Yoon wrote:
> Hi all,
>
> I would suggest that we have to focus on performance improvements and
> basic communication parts of BSP framework. So I've little edited our
> roadmap page.
>
> http://wiki.apache.org/hama/RoadMap
>
> If you have any comments or other opinions, let's discuss here.
>
> Thanks.
I'd recommend targeting running the fault tolerant code under the MR-279
engine (hadoop 0.22 or .23?), as this will deploy more than just the MR
engine, and handle liveness -so you don't need to replicate core modules.
the big thing for BSP engines is checkpointing, where the sync point
becomes the thing you can checkpoint.
--
ChiaHung Lin
Department of Information Management
National University of Kaohsiung
Taiwan
Re: [DISCUSS] Rearrange our roadmap
Posted by "Edward J. Yoon" <ed...@apache.org>.
>> the big thing for BSP engines is checkpointing, where the sync point becomes the thing you can checkpoint.
Heavily agree with you!
On Wed, Jun 8, 2011 at 8:15 PM, Steve Loughran <st...@apache.org> wrote:
> On 06/08/2011 06:22 AM, Edward J. Yoon wrote:
>>
>> Hi all,
>>
>> I would suggest that we have to focus on performance improvements and
>> basic communication parts of BSP framework. So I've little edited our
>> roadmap page.
>>
>> http://wiki.apache.org/hama/RoadMap
>>
>> If you have any comments or other opinions, let's discuss here.
>>
>> Thanks.
>
> I'd recommend targeting running the fault tolerant code under the MR-279
> engine (hadoop 0.22 or .23?), as this will deploy more than just the MR
> engine, and handle liveness -so you don't need to replicate core modules.
>
> the big thing for BSP engines is checkpointing, where the sync point becomes
> the thing you can checkpoint.
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: [DISCUSS] Rearrange our roadmap
Posted by Steve Loughran <st...@apache.org>.
On 06/08/2011 06:22 AM, Edward J. Yoon wrote:
> Hi all,
>
> I would suggest that we have to focus on performance improvements and
> basic communication parts of BSP framework. So I've little edited our
> roadmap page.
>
> http://wiki.apache.org/hama/RoadMap
>
> If you have any comments or other opinions, let's discuss here.
>
> Thanks.
I'd recommend targeting running the fault tolerant code under the MR-279
engine (hadoop 0.22 or .23?), as this will deploy more than just the MR
engine, and handle liveness -so you don't need to replicate core modules.
the big thing for BSP engines is checkpointing, where the sync point
becomes the thing you can checkpoint.