2.0 Roadmap ?

[Julien Vermillard <jv...@archean.fr>]

Hi,

After meeting Emmanuel IRL, we agreed we need to discuss some kind of
little roadmap for 2.0, and sort JIRA issues

Code features :

- integrating proxy patch
- composite buffers and streams based on it
- killing IoBuffer
- fixing logging filter
- removing netty2 codec module (who want to use it for Mina 2.0 ?) 
- test coverage (yeah painfull at this point)
- write traffic throttling

Doc feature : 
- testing and perhaps tuning APR based transport
- doc on reqres filter
- finishing doc on transports (core/APR/serial)
- overall documentation (tutorial, concepts)
- new website, clearer, and with better integration/links to
subprojects like ftpserver and asyncweb
- state of the art release tarballs

The discussion is open, the idea is to gather wish and ideas, reach some
consensus and clear the list of issue in JIRA.

WDYT ?

Julien

Re: 2.0 Roadmap ?

[Emmanuel Lecharny <el...@gmail.com>]

Hi guys,

sorry for not having replied to this mail yesturday, I was busy with 
personal stuff.

Thanks Julien for having done this summarize. I have to add that we 
didn't discussed all of the items on the list with Julien, but this is a 
very good thing that Julien added the important items. We badly need a 
roadmap for 2.0, taht's for sure.

Comments in line

Julien Vermillard wrote:
> Hi,
>
> next step is to create a wiki page due to the size of the list :)
>   
Certainly.
>   
>> <snip/>
>>> Code features :
>>>
>>> - integrating proxy patch
>>>       
>> Don't know what this is about.
>>     
> https://issues.apache.org/jira/browse/DIRMINA-415
>
> Edouard big patch for supporting proxy such as simple port forwarding
> and SOCKS.
> It's somewhat a big patch to shallow.
>   
ok. We have to look it closely.
>   
>>     
>>> - composite buffers and streams based on it
>>>       
>> I definitely want to get involved here but I need to catch up on the
>> conversations had in the past about it.
>>     
We need to discuss this item on the ML, as it will imply a hell lot of 
modifications.
>>
>>     
>>> - killing IoBuffer
>>>       
>> +1
>>     
Sure, +1.
>>     
>>> - fixing logging filter
>>>       
>> +1
>>     
I will create a JIRA plus submit at least two proposals (code)
>>     
>>> - removing netty2 codec module (who want to use it for Mina 2.0 ?)
>>>       
>> +1
>>     
+1
>>     
>>> - test coverage (yeah painfull at this point)
>>>       
>> Yeah test coverage is important but luckily MINA is not that large.
>> Test coverage makes it easier for people to get into the code to try
>> to add features or fix a bug while knowing their changes/fixes will
>> not cause regressions or breakage in the behavior of the API.  Plus
>> it's the best example code to use for how to learn to use different
>> features of MINA: I always look at test code when learning how to use
>> a new piece of OSS since the docs may not be there or may not be
>> totally up to date.
>>
>>     
>
> Emmanuel said me exactly the things word for word ;)
>   
We are sharing a lot with Alex, but more important, we have empowered 
each other to work better : removing bad habits ("Human habits ..." :) 
by learning from the other, and listening to criticisms. Here, we were 
on the very same page. It's also something we have gone through as we 
are not young bloods... call it experience !
>   
>>> - write traffic throttling
>>>       
>> Yes this is a concern of mine as well.  I've been experimenting in
>> the last few days specifically with ApacheDS trying to get her to
>> behave like a lady (u know not giving everything she's got and
>> overwhelming clients).  I realized we need to support the MINA
>> community by making the move to MINA 2 ASAP.  Was going to do this
>> last night but was wrestling with Maven issues. We'll probably make
>> the move shortly and can supply a lot of feedback with respect to
>> this aspect.
>>     
>
> I wrote a little resume here of ADS problem : 
> http://cwiki.apache.org/MINA/traffic-throttling.html
>
> I don't understand why it's not doable using WriteFuture.
> Would be very kind if someone of ADS can take 5 minutes to explain me
> where I'm wrong.
>   
I will try to explain the problem we have. May be I'm also missing some 
points.
>   
>>     
>>> Doc feature :
>>> - testing and perhaps tuning APR based transport
>>>       
>> +1
>>     
+1
>>     
>>> - doc on reqres filter
>>>       
>> Have not had the pleasure of looking at this filter at all - don't
>> know what it's for.
>>     
>
> it's for request/response protocols, so when you send a request you can
> listen for the response in a non blocking way.
> The whole module is doc less.
>   
ok. So +1
>   
>>> - finishing doc on transports (core/APR/serial)
>>>       
>> +1
>>     
+1
>>     
>>> - overall documentation (tutorial, concepts)
>>>       
>> +1
>>     
We badly need tutorials. Also, a while back, we discussed about buidling 
a kind of Ethereal on top of MINA. The biggest advantage of doing so 
(having a protocol codec for many well know protocols) is that people 
who want to learn about MINA will have real life samples on how to 
implement a protocol on top of MINA. Good candidates for such codec are 
obviously HTTP, FTP or LDAP (which is quite complex).
>>     
>>> - new website, clearer, and with better integration/links to
>>> subprojects like ftpserver and asyncweb
>>>       
>> +1
>>     
+1 !
>>     
>>> - state of the art release tarballs
>>>       
>> +1
>>
>> I think might we have some reusable infrastructure over at Directory
>> that might help with this and things like deb, pkg, rpm installers.
>> We're not starting completely from scratch in other words.
>>
>>     
>
> Well MINA is a lib, so perhaps releasing linux package is a bit extreme.
> Some of the work was already contributed, we just need to find a
> workaround for a maven assembly bug.
>   
Yeah, may be not necessary to deliver a rmp/deb. Maybe interesting for 
AsyncWeb or FTPServer.
>>> The discussion is open, the idea is to gather wish and ideas, reach
>>> some consensus and clear the list of issue in JIRA.
>>>
>>>       
>> Personally I'm disappointed by MINA's internals and how complex it is.
>> Another committer and I were discussing this complexity on IM a while
>> back. He said, "I have not read a single project that was great that
>> did not have lucid crazy simple code.  It's like what Feynman said
>> about complex subjects: you don't really understand it.  Same goes
>> for code.  If it's complex and obfuscated you don't understand the
>> problem domain."  I whole heartedly agree and there might be a need
>> for some rearchitecting for the sake of simplification and clarity.
>>
>> This can be approached in phases with subtle refactorings of the
>> internals without impacting the API.  In general this will improve:
>>
>>
>>    - our ability to respond to bugs,
>>    - requests for new features,
>>    - increase the developer adoption rate with lucid code,
>>    - have better more discrete test cases for internals,
>>    - and better manage and interpret issues when internals are
>> explicit in what they are designed to do
>>
>> After realizing that I need to step up, I've started reaclimating
>> myself with the internals.  I could not believe how disorderly it is:
>> I thought it was in much better shape. 
>>     
>
> What is amazing, it's the simplicity of the API and the complexity of
> the internals, it's sure NIO is not helping us here, but the is a lot
> of things to fix/document in core. Emmanuel raised a good example with
> LoggingFilter.
>
> The death of IoBuffer will probably be the first big simplification of
> the code base.
>   
Damn yes ! IoBuffer represent itself around 15% of the code.

There is another aspect which is cumbersome and complex : the filter 
chain. I personnally don't think we need to have such a complex 
implementation (the chaining concept is ok). For instance, we have a 
HeadFilter and a TailFilter, I don't think we need them. Plus I also 
think that we should distinguish between the incoming chain and the 
outgoing chain. last, not least, we may need to have more than one 
chain, as events may not have the same pathways. This is something we 
have to think about.
Another reason why the current implementation sucks, is taht when it 
comes to debug it, it's really a PITA.

>   
>> In the present state, it's
>> very hard to see people with limited time frames looking at the code
>> and understanding it in time to actually affect some change.  The
>> barrier is too high but we can fix that one step at a time.
>>
>> Alex
>>     
>
> Yep actually MINA contributors are all hobbyist in the number of hours
> allocated, so we need reasonable target.
>   
The current community is pretty wealthy, and I'm quite please to see 
that ithere is a lot of positive energy, and new blood. Also we are not 
in a hurry, as we don't have to deliver MINA 2.0 in 3 months. We can 
spend a few extra months in order to get a good 2.0 MINA. And I don't 
want to have a 2.0 out if this is just to start working on a 3.0 which 
will be almost a complete rewrite.

Thanks Julien !
> Julien
>   


-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org

Re: 2.0 Roadmap ?

[Julien Vermillard <jv...@archean.fr>]

Hi,

next step is to create a wiki page due to the size of the list :)

comments inline .. 

On Mon, 25 Aug 2008 14:41:27 -0400
"Alex Karasulu" <ak...@apache.org> wrote:

> Hi all,
> 
> On Mon, Aug 25, 2008 at 5:18 AM, Julien Vermillard
> <jv...@archean.fr>wrote:
> 
> > Hi,
> >
> > After meeting Emmanuel IRL, we agreed we need to discuss some kind
> > of little roadmap for 2.0, and sort JIRA issues
> >
> 
> Excellent, thanks for summarizing here.
> 
> 
> >
> > Code features :
> >
> > - integrating proxy patch
> 
> 
> Don't know what this is about.
https://issues.apache.org/jira/browse/DIRMINA-415

Edouard big patch for supporting proxy such as simple port forwarding
and SOCKS.
It's somewhat a big patch to shallow.

> 
> 
> >
> > - composite buffers and streams based on it
> 
> 
> I definitely want to get involved here but I need to catch up on the
> conversations had in the past about it.
> 
> 
> >
> > - killing IoBuffer
> 
> +1
> 
> >
> > - fixing logging filter
> 
> +1
> 
> >
> > - removing netty2 codec module (who want to use it for Mina 2.0 ?)
> 
> +1
> 
> >
> > - test coverage (yeah painfull at this point)
> 
> 
> Yeah test coverage is important but luckily MINA is not that large.
> Test coverage makes it easier for people to get into the code to try
> to add features or fix a bug while knowing their changes/fixes will
> not cause regressions or breakage in the behavior of the API.  Plus
> it's the best example code to use for how to learn to use different
> features of MINA: I always look at test code when learning how to use
> a new piece of OSS since the docs may not be there or may not be
> totally up to date.
> 

Emmanuel said me exactly the things word for word ;)

> 
> >
> > - write traffic throttling
> 
> 
> Yes this is a concern of mine as well.  I've been experimenting in
> the last few days specifically with ApacheDS trying to get her to
> behave like a lady (u know not giving everything she's got and
> overwhelming clients).  I realized we need to support the MINA
> community by making the move to MINA 2 ASAP.  Was going to do this
> last night but was wrestling with Maven issues. We'll probably make
> the move shortly and can supply a lot of feedback with respect to
> this aspect.

I wrote a little resume here of ADS problem : 
http://cwiki.apache.org/MINA/traffic-throttling.html

I don't understand why it's not doable using WriteFuture.
Would be very kind if someone of ADS can take 5 minutes to explain me
where I'm wrong.

> 
> 
> >
> >
> > Doc feature :
> > - testing and perhaps tuning APR based transport
> 
> +1
> 
> >
> > - doc on reqres filter
> 
> Have not had the pleasure of looking at this filter at all - don't
> know what it's for.

it's for request/response protocols, so when you send a request you can
listen for the response in a non blocking way.
The whole module is doc less.

> 
> >
> > - finishing doc on transports (core/APR/serial)
> 
> +1
> 
> >
> > - overall documentation (tutorial, concepts)
> 
> +1
> 
> >
> > - new website, clearer, and with better integration/links to
> > subprojects like ftpserver and asyncweb
> 
> +1
> 
> >
> > - state of the art release tarballs
> 
> +1
> 
> I think might we have some reusable infrastructure over at Directory
> that might help with this and things like deb, pkg, rpm installers.
> We're not starting completely from scratch in other words.
> 

Well MINA is a lib, so perhaps releasing linux package is a bit extreme.
Some of the work was already contributed, we just need to find a
workaround for a maven assembly bug.
> 
> >
> > The discussion is open, the idea is to gather wish and ideas, reach
> > some consensus and clear the list of issue in JIRA.
> >
> 
> Personally I'm disappointed by MINA's internals and how complex it is.
> Another committer and I were discussing this complexity on IM a while
> back. He said, "I have not read a single project that was great that
> did not have lucid crazy simple code.  It's like what Feynman said
> about complex subjects: you don't really understand it.  Same goes
> for code.  If it's complex and obfuscated you don't understand the
> problem domain."  I whole heartedly agree and there might be a need
> for some rearchitecting for the sake of simplification and clarity.
> 
> This can be approached in phases with subtle refactorings of the
> internals without impacting the API.  In general this will improve:
> 
> 
>    - our ability to respond to bugs,
>    - requests for new features,
>    - increase the developer adoption rate with lucid code,
>    - have better more discrete test cases for internals,
>    - and better manage and interpret issues when internals are
> explicit in what they are designed to do
> 
> After realizing that I need to step up, I've started reaclimating
> myself with the internals.  I could not believe how disorderly it is:
> I thought it was in much better shape. 

What is amazing, it's the simplicity of the API and the complexity of
the internals, it's sure NIO is not helping us here, but the is a lot
of things to fix/document in core. Emmanuel raised a good example with
LoggingFilter.

The death of IoBuffer will probably be the first big simplification of
the code base.

> In the present state, it's
> very hard to see people with limited time frames looking at the code
> and understanding it in time to actually affect some change.  The
> barrier is too high but we can fix that one step at a time.
> 
> Alex

Yep actually MINA contributors are all hobbyist in the number of hours
allocated, so we need reasonable target.

Julien

Re: 2.0 Roadmap ?

[Alex Karasulu <ak...@apache.org>]

Hi all,

On Mon, Aug 25, 2008 at 5:18 AM, Julien Vermillard
<jv...@archean.fr>wrote:

> Hi,
>
> After meeting Emmanuel IRL, we agreed we need to discuss some kind of
> little roadmap for 2.0, and sort JIRA issues
>

Excellent, thanks for summarizing here.


>
> Code features :
>
> - integrating proxy patch


Don't know what this is about.


>
> - composite buffers and streams based on it


I definitely want to get involved here but I need to catch up on the
conversations had in the past about it.


>
> - killing IoBuffer

+1

>
> - fixing logging filter

+1

>
> - removing netty2 codec module (who want to use it for Mina 2.0 ?)

+1

>
> - test coverage (yeah painfull at this point)


Yeah test coverage is important but luckily MINA is not that large.  Test
coverage makes it easier for people to get into the code to try to add
features or fix a bug while knowing their changes/fixes will not cause
regressions or breakage in the behavior of the API.  Plus it's the best
example code to use for how to learn to use different features of MINA: I
always look at test code when learning how to use a new piece of OSS since
the docs may not be there or may not be totally up to date.


>
> - write traffic throttling


Yes this is a concern of mine as well.  I've been experimenting in the last
few days specifically with ApacheDS trying to get her to behave like a lady
(u know not giving everything she's got and overwhelming clients).  I
realized we need to support the MINA community by making the move to MINA 2
ASAP.  Was going to do this last night but was wrestling with Maven issues.
We'll probably make the move shortly and can supply a lot of feedback with
respect to this aspect.


>
>
> Doc feature :
> - testing and perhaps tuning APR based transport

+1

>
> - doc on reqres filter

Have not had the pleasure of looking at this filter at all - don't know what
it's for.

>
> - finishing doc on transports (core/APR/serial)

+1

>
> - overall documentation (tutorial, concepts)

+1

>
> - new website, clearer, and with better integration/links to
> subprojects like ftpserver and asyncweb

+1

>
> - state of the art release tarballs

+1

I think might we have some reusable infrastructure over at Directory that
might help with this and things like deb, pkg, rpm installers. We're not
starting completely from scratch in other words.


>
> The discussion is open, the idea is to gather wish and ideas, reach some
> consensus and clear the list of issue in JIRA.
>

Personally I'm disappointed by MINA's internals and how complex it is.
Another committer and I were discussing this complexity on IM a while back.
He said, "I have not read a single project that was great that did not have
lucid crazy simple code.  It's like what Feynman said about complex
subjects: you don't really understand it.  Same goes for code.  If it's
complex and obfuscated you don't understand the problem domain."  I whole
heartedly agree and there might be a need for some rearchitecting for the
sake of simplification and clarity.

This can be approached in phases with subtle refactorings of the internals
without impacting the API.  In general this will improve:


   - our ability to respond to bugs,
   - requests for new features,
   - increase the developer adoption rate with lucid code,
   - have better more discrete test cases for internals,
   - and better manage and interpret issues when internals are explicit in
   what they are designed to do

After realizing that I need to step up, I've started reaclimating myself
with the internals.  I could not believe how disorderly it is: I thought it
was in much better shape.  In the present state, it's very hard to see
people with limited time frames looking at the code and understanding it in
time to actually affect some change.  The barrier is too high but we can fix
that one step at a time.

Alex