You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon" <ed...@apache.org> on 2011/07/04 02:13:18 UTC
Our TODO things
Hi all,
Here's our TODO things:
1. Improve lock and barrier synchronization
2. Support configurable task slot
3. Add Pregel-like graph computing framework based on BSP engine
4. Support checkpoint based on HDFS
5. Support Fault Tolerant based on checkpoint
6. Support Input/OutputFormatter like MapReduce
7. Support compatible job with MapReduce2
8. Support send and receive large messages or many messages
9. Support message compression
10. Advanced job scheduler
* Performance and Fault Tolerant issues requires a lot of experiences
and tests.
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Our TODO things
Posted by ChiaHung Lin <ch...@nuk.edu.tw>.
Barrier synchronization seemingly can be improved by applying tree-based dissemination structure[1].
[1]. Practical Barrier Synchronisation. ftp://ftp.comlab.ox.ac.uk/pub/Documents/techpapers/Jonathan.Hill/HillSkill_barrier.ps.Z
-----Original message-----
From:Edward J. Yoon <ed...@apache.org>
To:hama-dev@incubator.apache.org
Date:Mon, 4 Jul 2011 09:13:18 +0900
Subject:Our TODO things
Hi all,
Here's our TODO things:
1. Improve lock and barrier synchronization
2. Support configurable task slot
3. Add Pregel-like graph computing framework based on BSP engine
4. Support checkpoint based on HDFS
5. Support Fault Tolerant based on checkpoint
6. Support Input/OutputFormatter like MapReduce
7. Support compatible job with MapReduce2
8. Support send and receive large messages or many messages
9. Support message compression
10. Advanced job scheduler
* Performance and Fault Tolerant issues requires a lot of experiences
and tests.
--
Best Regards, Edward J. Yoon
@eddieyoon
--
ChiaHung Lin
Department of Information Management
National University of Kaohsiung
Taiwan
Re: Our TODO things
Posted by Thomas Jungblut <th...@googlemail.com>.
Ah I wanted to add the "asynchronous message sending".
This could speed up things very much.
Let's discuss it over here:
https://issues.apache.org/jira/browse/HAMA-358
2011/7/4 Thomas Jungblut <th...@googlemail.com>:
> Hi Edward,
>
> should we open issues for that, scheduling to 0.4.0?
>
> Regards,
> Thomas
>
> 2011/7/4 Edward J. Yoon <ed...@apache.org>:
>> Hi all,
>>
>> Here's our TODO things:
>>
>> 1. Improve lock and barrier synchronization
>> 2. Support configurable task slot
>> 3. Add Pregel-like graph computing framework based on BSP engine
>> 4. Support checkpoint based on HDFS
>> 5. Support Fault Tolerant based on checkpoint
>> 6. Support Input/OutputFormatter like MapReduce
>> 7. Support compatible job with MapReduce2
>> 8. Support send and receive large messages or many messages
>> 9. Support message compression
>> 10. Advanced job scheduler
>>
>> * Performance and Fault Tolerant issues requires a lot of experiences
>> and tests.
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
>
>
> --
> Thomas Jungblut
> Berlin
>
> mobile: 0170-3081070
>
> business: thomas.jungblut@testberichte.de
> private: thomas.jungblut@gmail.com
>
--
Thomas Jungblut
Berlin
mobile: 0170-3081070
business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com
Re: Our TODO things
Posted by Thomas Jungblut <th...@googlemail.com>.
We have to add various things to the webfrontend, I have a ton of ideas.
But the web UI for a groom server is a good beginning.
2011/7/5 Edward J. Yoon <ed...@apache.org>:
> Plus,
>
> We have to add web UI for groom servers. Right?
>
> 2011/7/5 Edward J. Yoon <ed...@apache.org>:
>> 0.x.0
>>
>> Sent from my iPhone
>>
>> On 2011. 7. 4., at 오후 11:39, Thomas Jungblut <th...@googlemail.com> wrote:
>>
>>> Hi Edward,
>>>
>>> should we open issues for that, scheduling to 0.4.0?
>>>
>>> Regards,
>>> Thomas
>>>
>>> 2011/7/4 Edward J. Yoon <ed...@apache.org>:
>>>> Hi all,
>>>>
>>>> Here's our TODO things:
>>>>
>>>> 1. Improve lock and barrier synchronization
>>>> 2. Support configurable task slot
>>>> 3. Add Pregel-like graph computing framework based on BSP engine
>>>> 4. Support checkpoint based on HDFS
>>>> 5. Support Fault Tolerant based on checkpoint
>>>> 6. Support Input/OutputFormatter like MapReduce
>>>> 7. Support compatible job with MapReduce2
>>>> 8. Support send and receive large messages or many messages
>>>> 9. Support message compression
>>>> 10. Advanced job scheduler
>>>>
>>>> * Performance and Fault Tolerant issues requires a lot of experiences
>>>> and tests.
>>>>
>>>> --
>>>> Best Regards, Edward J. Yoon
>>>> @eddieyoon
>>>>
>>>
>>>
>>>
>>> --
>>> Thomas Jungblut
>>> Berlin
>>>
>>> mobile: 0170-3081070
>>>
>>> business: thomas.jungblut@testberichte.de
>>> private: thomas.jungblut@gmail.com
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
--
Thomas Jungblut
Berlin
mobile: 0170-3081070
business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com
Re: Our TODO things
Posted by "Edward J. Yoon" <ed...@apache.org>.
Plus,
We have to add web UI for groom servers. Right?
2011/7/5 Edward J. Yoon <ed...@apache.org>:
> 0.x.0
>
> Sent from my iPhone
>
> On 2011. 7. 4., at 오후 11:39, Thomas Jungblut <th...@googlemail.com> wrote:
>
>> Hi Edward,
>>
>> should we open issues for that, scheduling to 0.4.0?
>>
>> Regards,
>> Thomas
>>
>> 2011/7/4 Edward J. Yoon <ed...@apache.org>:
>>> Hi all,
>>>
>>> Here's our TODO things:
>>>
>>> 1. Improve lock and barrier synchronization
>>> 2. Support configurable task slot
>>> 3. Add Pregel-like graph computing framework based on BSP engine
>>> 4. Support checkpoint based on HDFS
>>> 5. Support Fault Tolerant based on checkpoint
>>> 6. Support Input/OutputFormatter like MapReduce
>>> 7. Support compatible job with MapReduce2
>>> 8. Support send and receive large messages or many messages
>>> 9. Support message compression
>>> 10. Advanced job scheduler
>>>
>>> * Performance and Fault Tolerant issues requires a lot of experiences
>>> and tests.
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>> @eddieyoon
>>>
>>
>>
>>
>> --
>> Thomas Jungblut
>> Berlin
>>
>> mobile: 0170-3081070
>>
>> business: thomas.jungblut@testberichte.de
>> private: thomas.jungblut@gmail.com
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Our TODO things
Posted by "Edward J. Yoon" <ed...@apache.org>.
0.x.0
Sent from my iPhone
On 2011. 7. 4., at 오후 11:39, Thomas Jungblut <th...@googlemail.com> wrote:
> Hi Edward,
>
> should we open issues for that, scheduling to 0.4.0?
>
> Regards,
> Thomas
>
> 2011/7/4 Edward J. Yoon <ed...@apache.org>:
>> Hi all,
>>
>> Here's our TODO things:
>>
>> 1. Improve lock and barrier synchronization
>> 2. Support configurable task slot
>> 3. Add Pregel-like graph computing framework based on BSP engine
>> 4. Support checkpoint based on HDFS
>> 5. Support Fault Tolerant based on checkpoint
>> 6. Support Input/OutputFormatter like MapReduce
>> 7. Support compatible job with MapReduce2
>> 8. Support send and receive large messages or many messages
>> 9. Support message compression
>> 10. Advanced job scheduler
>>
>> * Performance and Fault Tolerant issues requires a lot of experiences
>> and tests.
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
>
>
> --
> Thomas Jungblut
> Berlin
>
> mobile: 0170-3081070
>
> business: thomas.jungblut@testberichte.de
> private: thomas.jungblut@gmail.com
Re: Our TODO things
Posted by Thomas Jungblut <th...@googlemail.com>.
Hi Edward,
should we open issues for that, scheduling to 0.4.0?
Regards,
Thomas
2011/7/4 Edward J. Yoon <ed...@apache.org>:
> Hi all,
>
> Here's our TODO things:
>
> 1. Improve lock and barrier synchronization
> 2. Support configurable task slot
> 3. Add Pregel-like graph computing framework based on BSP engine
> 4. Support checkpoint based on HDFS
> 5. Support Fault Tolerant based on checkpoint
> 6. Support Input/OutputFormatter like MapReduce
> 7. Support compatible job with MapReduce2
> 8. Support send and receive large messages or many messages
> 9. Support message compression
> 10. Advanced job scheduler
>
> * Performance and Fault Tolerant issues requires a lot of experiences
> and tests.
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
--
Thomas Jungblut
Berlin
mobile: 0170-3081070
business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com