You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by Bobby Evans <ev...@yahoo-inc.com.INVALID> on 2016/08/01 13:27:50 UTC

Re: [DISCUSSION] Move out porting tests to after JStorm merger (Phase 3 or even 2.x)

The plan was that we would all concentrate on the migration and knock it out quickly, but stuff happened and we were not able to concentrate on it as fully as I would have liked.  I agree that the tests can wait.  What is more many of the tests because of local mode have to wait until all of the daemons have been ported over.  I really wish there were more hours in the day for me to help out more on this. - Bobby 

    On Sunday, July 31, 2016 6:51 PM, Jungtaek Lim <ka...@gmail.com> wrote:
 

 Hi devs,

Porting to Java seems to be longer than we expected, and now working on
storm-core requires working for Java and Clojure, say twice.
One of reason for port is "lowering learning curve to have more
contributors", but now contributors need to know Clojure to contribute
storm-core targeted for 1.x and "port" to master. It's even harder for me
since I need some time to investigate how things are ported.

Bobby suggested "feature freeze" for 1.x and move to 2.x quickly, but IMO
it will work only when we have due date for 2.0.0 and we all concentrate on
this to release ASAP.
(Sorry I had to break "feature freeze" since metrics improvements are
needed within months.)
Since we're individual and also several different teams, setting a due date
for this seems to be just ideal. Need to find other ways to make transition
faster.

Looking into progress of porting to Java, I found that we have ported many
places on source so we only left some (still huge) daemons, but still have
lots of files on test.
Given that ported files should pass Clojure tests, I think porting tests is
not urgent and we can move this out of phase 1.

What do you think?

Thanks,
Jungtaek Lim (HeartSaVioR)

ps. Just two cents, I even think it might be valid way to set milestone for
2.0.0 to only phase 1 (with well tested). If Storm 2.0.0 is just a Java
port of Storm 1.x, 1.x users can easily move to 2.0.0 and we can minimize
supporting multi-versions with main language difference. Phase 2 and other
improvements can be on top of 2.0.0. (We may need to have 3.0.0 for that
but I think version bump is not a big deal.)


  

Re: [DISCUSSION] Move out porting tests to after JStorm merger (Phase 3 or even 2.x)

Posted by Jungtaek Lim <ka...@gmail.com>.
Yes they should be included. Maybe all clojure files in src directory
should be ported as well, or be moved to test directory if it's OK to not
exposed to users.

2016년 8월 2일 (화) 오전 1:12, Abhishek Agarwal <ab...@gmail.com>님이 작성:

> Tests themselves can wait but helper code (testing.clj and testing4j.clj)
> is part of storm-core jar (both are assigned to me). That could be ported
> at least. These classes depend on other daemons which are yet to be ported.
>
> On Mon, Aug 1, 2016 at 6:57 PM, Bobby Evans <ev...@yahoo-inc.com.invalid>
> wrote:
>
> > The plan was that we would all concentrate on the migration and knock it
> > out quickly, but stuff happened and we were not able to concentrate on it
> > as fully as I would have liked.  I agree that the tests can wait.  What
> is
> > more many of the tests because of local mode have to wait until all of
> the
> > daemons have been ported over.  I really wish there were more hours in
> the
> > day for me to help out more on this. - Bobby
> >
> >     On Sunday, July 31, 2016 6:51 PM, Jungtaek Lim <ka...@gmail.com>
> > wrote:
> >
> >
> >  Hi devs,
> >
> > Porting to Java seems to be longer than we expected, and now working on
> > storm-core requires working for Java and Clojure, say twice.
> > One of reason for port is "lowering learning curve to have more
> > contributors", but now contributors need to know Clojure to contribute
> > storm-core targeted for 1.x and "port" to master. It's even harder for me
> > since I need some time to investigate how things are ported.
> >
> > Bobby suggested "feature freeze" for 1.x and move to 2.x quickly, but IMO
> > it will work only when we have due date for 2.0.0 and we all concentrate
> on
> > this to release ASAP.
> > (Sorry I had to break "feature freeze" since metrics improvements are
> > needed within months.)
> > Since we're individual and also several different teams, setting a due
> date
> > for this seems to be just ideal. Need to find other ways to make
> transition
> > faster.
> >
> > Looking into progress of porting to Java, I found that we have ported
> many
> > places on source so we only left some (still huge) daemons, but still
> have
> > lots of files on test.
> > Given that ported files should pass Clojure tests, I think porting tests
> is
> > not urgent and we can move this out of phase 1.
> >
> > What do you think?
> >
> > Thanks,
> > Jungtaek Lim (HeartSaVioR)
> >
> > ps. Just two cents, I even think it might be valid way to set milestone
> for
> > 2.0.0 to only phase 1 (with well tested). If Storm 2.0.0 is just a Java
> > port of Storm 1.x, 1.x users can easily move to 2.0.0 and we can minimize
> > supporting multi-versions with main language difference. Phase 2 and
> other
> > improvements can be on top of 2.0.0. (We may need to have 3.0.0 for that
> > but I think version bump is not a big deal.)
> >
> >
> >
> >
>
>
>
> --
> Regards,
> Abhishek Agarwal
>

Re: [DISCUSSION] Move out porting tests to after JStorm merger (Phase 3 or even 2.x)

Posted by Abhishek Agarwal <ab...@gmail.com>.
Tests themselves can wait but helper code (testing.clj and testing4j.clj)
is part of storm-core jar (both are assigned to me). That could be ported
at least. These classes depend on other daemons which are yet to be ported.

On Mon, Aug 1, 2016 at 6:57 PM, Bobby Evans <ev...@yahoo-inc.com.invalid>
wrote:

> The plan was that we would all concentrate on the migration and knock it
> out quickly, but stuff happened and we were not able to concentrate on it
> as fully as I would have liked.  I agree that the tests can wait.  What is
> more many of the tests because of local mode have to wait until all of the
> daemons have been ported over.  I really wish there were more hours in the
> day for me to help out more on this. - Bobby
>
>     On Sunday, July 31, 2016 6:51 PM, Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>
>  Hi devs,
>
> Porting to Java seems to be longer than we expected, and now working on
> storm-core requires working for Java and Clojure, say twice.
> One of reason for port is "lowering learning curve to have more
> contributors", but now contributors need to know Clojure to contribute
> storm-core targeted for 1.x and "port" to master. It's even harder for me
> since I need some time to investigate how things are ported.
>
> Bobby suggested "feature freeze" for 1.x and move to 2.x quickly, but IMO
> it will work only when we have due date for 2.0.0 and we all concentrate on
> this to release ASAP.
> (Sorry I had to break "feature freeze" since metrics improvements are
> needed within months.)
> Since we're individual and also several different teams, setting a due date
> for this seems to be just ideal. Need to find other ways to make transition
> faster.
>
> Looking into progress of porting to Java, I found that we have ported many
> places on source so we only left some (still huge) daemons, but still have
> lots of files on test.
> Given that ported files should pass Clojure tests, I think porting tests is
> not urgent and we can move this out of phase 1.
>
> What do you think?
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> ps. Just two cents, I even think it might be valid way to set milestone for
> 2.0.0 to only phase 1 (with well tested). If Storm 2.0.0 is just a Java
> port of Storm 1.x, 1.x users can easily move to 2.0.0 and we can minimize
> supporting multi-versions with main language difference. Phase 2 and other
> improvements can be on top of 2.0.0. (We may need to have 3.0.0 for that
> but I think version bump is not a big deal.)
>
>
>
>



-- 
Regards,
Abhishek Agarwal

Re: [DISCUSSION] Move out porting tests to after JStorm merger (Phase 3 or even 2.x)

Posted by Jungtaek Lim <ka...@gmail.com>.
Yeah I think your plan would be perfect once we concentrate migration
works. But things could be happen for all of us, and individual of us need
to focus other thing at that time. In many chance it's not that we could
control.

Let's back to now and find the way to go forward with our plan for Storm
2.0.0. At least I'd like to see port to be finished ASAP, since it makes
huge divergence between branches, and we could even lose the flow when it's
getting longer.

I'm still not familiar with Clojure so not easy to participate huge part of
the work, but would like to help out porting smaller parts, and reviewing
relatively bigger part.

- Jungtaek Lim (HeartSaVioR)

2016년 8월 1일 (월) 오후 10:28, Bobby Evans <ev...@yahoo-inc.com.invalid>님이 작성:

> The plan was that we would all concentrate on the migration and knock it
> out quickly, but stuff happened and we were not able to concentrate on it
> as fully as I would have liked.  I agree that the tests can wait.  What is
> more many of the tests because of local mode have to wait until all of the
> daemons have been ported over.  I really wish there were more hours in the
> day for me to help out more on this. - Bobby
>
>     On Sunday, July 31, 2016 6:51 PM, Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>
>  Hi devs,
>
> Porting to Java seems to be longer than we expected, and now working on
> storm-core requires working for Java and Clojure, say twice.
> One of reason for port is "lowering learning curve to have more
> contributors", but now contributors need to know Clojure to contribute
> storm-core targeted for 1.x and "port" to master. It's even harder for me
> since I need some time to investigate how things are ported.
>
> Bobby suggested "feature freeze" for 1.x and move to 2.x quickly, but IMO
> it will work only when we have due date for 2.0.0 and we all concentrate on
> this to release ASAP.
> (Sorry I had to break "feature freeze" since metrics improvements are
> needed within months.)
> Since we're individual and also several different teams, setting a due date
> for this seems to be just ideal. Need to find other ways to make transition
> faster.
>
> Looking into progress of porting to Java, I found that we have ported many
> places on source so we only left some (still huge) daemons, but still have
> lots of files on test.
> Given that ported files should pass Clojure tests, I think porting tests is
> not urgent and we can move this out of phase 1.
>
> What do you think?
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> ps. Just two cents, I even think it might be valid way to set milestone for
> 2.0.0 to only phase 1 (with well tested). If Storm 2.0.0 is just a Java
> port of Storm 1.x, 1.x users can easily move to 2.0.0 and we can minimize
> supporting multi-versions with main language difference. Phase 2 and other
> improvements can be on top of 2.0.0. (We may need to have 3.0.0 for that
> but I think version bump is not a big deal.)
>
>
>