You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by Roshan Naik <ro...@hortonworks.com> on 2017/03/01 00:21:43 UTC
Storm 1.x-branch won't build in IntelliJ anymore
Lately (about a week and half maybe) it has not been possible to get the 1.x-branch to build inside IntelliJ. None of the modules are able to locate the LocalCluster class (which is a clojure class). Previously, every once in a while, I used to get this problem in the storm-starter module and was able to get around it by doing a mvn clean install –DskipTests on the cmd line and then doing a full rebuild of the project in IntelliJ.
Now the problem has become a lot more endemic (modules like storm-sql, storm-*-examples, etc.). And the above workaround doesn’t help. Spent many futile hours trying to work around the build issue within Intellij (both 2015 and 2016 versions and using different Clojure plugins).
There seems to possible ways to move forward:
- Somebody here knows what magic to do work address this. And we can all use that.
- Bring in the java port of LocalCluster.clj from master branch to 1.x (STORM-1281). I spend a little time to see if this was easy to do, but appears kind of complicated due to number of files involved and dependencies on prior patches. Somebody familiar with the original porting effort may be better person to take this up.
Thoughts ?
-Roshan
Re: Storm 1.x-branch won't build in IntelliJ anymore
Posted by Roshan Naik <ro...@hortonworks.com>.
That workaround of running a clean mvn build on the cmd line before building in IntellJ does not work for me anymore. It used sometime back (a month ago?).
No matter what I try… I cant get it to build in IntelliJ.
-roshan
On 3/1/17, 5:20 PM, "Jungtaek Lim" <ka...@gmail.com> wrote:
The original plan was not adding anything related storm-core in 1.x version
line and move out 2.0.0 fast. (That's what Bobby proposed for JStorm merger
phases.)
We didn't make it faster (sorry I couldn't contribute for this side at
all), and we broke the plan, and it makes harder and harder to move toward
2.0.0.
What we're adding is what we need to maintain, and so as what we're
releasing. I have other project which has similar plan and it didn't reach
major release for years. I don't want to see same thing for Storm given
that merger plan was started more than 1 year.
Moving to Java codebase gets rid of hard thing what you talked about, and
let end-users being able to dig the code and becomes contributors. It's
invaluable change.
Btw I can still be able to play with 1.x version line with workaround (mvn
clean install -DskipTests). Does workaround not work any more? Or do you
see some spots which workaround doesn't work?
- Jungtaek Lim (HeartSaVioR)
On Wed, Mar 1, 2017 at 11:34 PM, Roshan Naik <ro...@hortonworks.com> wrote:
> If we leave 1.x in this state it will forever remain un-usable within
> IntelliJ .. hard to debug, UT etc. And 1.x will live for a while. Is
> rolling back some of the non-critical offending commits that are causing
> this an option ? i tried rolling back a bit but was unable to identify the
> offending commit.
> -roshan
>
>
> On 2/28/17, 9:13 PM, "Arun Iyer on behalf of Arun Mahadevan" <
> aiyer@hortonworks.com on behalf of arunm@apache.org> wrote:
>
> >If we really need long time to discuss above, I'm even OK to revert
> DRPC
> >port and start 2.0.0 with webservices unported, (DRPC, UI, Logviewer)
> and
> >address them at 2.1 or other minor versions.
> >
> >Any other opinions?
> >
>
> Yes, it may be better to move to 2.0 sooner and migrate the pending
> components in 2.1. May be what we need is more rigorous testing of the
> master branch before we release 2.0.
>
> We could also consider moving to JDK 8 for the next 1.x release and it
> would make porting patches to 1.x branch much easier (unless we want to do
> a JDK upgrade only with 2.0).
>
> Thanks,
> Arun
>
> On 3/1/17, 7:40 AM, "Jungtaek Lim" <ka...@gmail.com> wrote:
>
> >Thanks Roshan for bring this up.
> >
> >For me moving toward 2.0.0 makes more sense.
> >
> >I know master branch has similar issue (DRPC) and it also has
> >not-yet-ported things but if we are going to struggle with 1.x branch
> issue
> >again and again, Storm 2.0.0 will never come. Another recent headache
> issue
> >is JDK 7 vs JDK 8. Patch for master easily breaks for 1.x branch due
> to
> >this, and we just had to cancel another 1.1.0 RC vote.
> >
> >We might want to discuss how to handle webservice like DRPC (My
> feeling is
> >that current approach is somewhat kinda hacky.), but we can initiate
> >different thread for more details.
> >
> >If we really need long time to discuss above, I'm even OK to revert
> DRPC
> >port and start 2.0.0 with webservices unported, (DRPC, UI, Logviewer)
> and
> >address them at 2.1 or other minor versions.
> >
> >Any other opinions?
> >
> >- Jungtaek Lim (HeartSaVioR)
> >
> >2017년 3월 1일 (수) 오전 9:27, Roshan Naik <ro...@hortonworks.com>님이 작성:
> >
> >Lately (about a week and half maybe) it has not been possible to get
> the
> >1.x-branch to build inside IntelliJ. None of the modules are able to
> locate
> >the LocalCluster class (which is a clojure class). Previously, every
> once
> >in a while, I used to get this problem in the storm-starter module
> and was
> >able to get around it by doing a mvn clean install –DskipTests on the
> cmd
> >line and then doing a full rebuild of the project in IntelliJ.
> >
> >Now the problem has become a lot more endemic (modules like storm-sql,
> >storm-*-examples, etc.). And the above workaround doesn’t help. Spent
> many
> >futile hours trying to work around the build issue within Intellij
> (both
> >2015 and 2016 versions and using different Clojure plugins).
> >
> >There seems to possible ways to move forward:
> >
> >
> >- Somebody here knows what magic to do work address this.
> And we
> >can all use that.
> >
> >- Bring in the java port of LocalCluster.clj from master
> branch to
> >1.x (STORM-1281). I spend a little time to see if this was easy to
> do, but
> >appears kind of complicated due to number of files involved and
> >dependencies on prior patches. Somebody familiar with the original
> porting
> >effort may be better person to take this up.
> >
> >Thoughts ?
> >
> >-Roshan
>
>
>
>
>
>
--
Name : Jungtaek Lim
Blog : http://medium.com/@heartsavior
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior
Re: Storm 1.x-branch won't build in IntelliJ anymore
Posted by Jungtaek Lim <ka...@gmail.com>.
The original plan was not adding anything related storm-core in 1.x version
line and move out 2.0.0 fast. (That's what Bobby proposed for JStorm merger
phases.)
We didn't make it faster (sorry I couldn't contribute for this side at
all), and we broke the plan, and it makes harder and harder to move toward
2.0.0.
What we're adding is what we need to maintain, and so as what we're
releasing. I have other project which has similar plan and it didn't reach
major release for years. I don't want to see same thing for Storm given
that merger plan was started more than 1 year.
Moving to Java codebase gets rid of hard thing what you talked about, and
let end-users being able to dig the code and becomes contributors. It's
invaluable change.
Btw I can still be able to play with 1.x version line with workaround (mvn
clean install -DskipTests). Does workaround not work any more? Or do you
see some spots which workaround doesn't work?
- Jungtaek Lim (HeartSaVioR)
On Wed, Mar 1, 2017 at 11:34 PM, Roshan Naik <ro...@hortonworks.com> wrote:
> If we leave 1.x in this state it will forever remain un-usable within
> IntelliJ .. hard to debug, UT etc. And 1.x will live for a while. Is
> rolling back some of the non-critical offending commits that are causing
> this an option ? i tried rolling back a bit but was unable to identify the
> offending commit.
> -roshan
>
>
> On 2/28/17, 9:13 PM, "Arun Iyer on behalf of Arun Mahadevan" <
> aiyer@hortonworks.com on behalf of arunm@apache.org> wrote:
>
> >If we really need long time to discuss above, I'm even OK to revert
> DRPC
> >port and start 2.0.0 with webservices unported, (DRPC, UI, Logviewer)
> and
> >address them at 2.1 or other minor versions.
> >
> >Any other opinions?
> >
>
> Yes, it may be better to move to 2.0 sooner and migrate the pending
> components in 2.1. May be what we need is more rigorous testing of the
> master branch before we release 2.0.
>
> We could also consider moving to JDK 8 for the next 1.x release and it
> would make porting patches to 1.x branch much easier (unless we want to do
> a JDK upgrade only with 2.0).
>
> Thanks,
> Arun
>
> On 3/1/17, 7:40 AM, "Jungtaek Lim" <ka...@gmail.com> wrote:
>
> >Thanks Roshan for bring this up.
> >
> >For me moving toward 2.0.0 makes more sense.
> >
> >I know master branch has similar issue (DRPC) and it also has
> >not-yet-ported things but if we are going to struggle with 1.x branch
> issue
> >again and again, Storm 2.0.0 will never come. Another recent headache
> issue
> >is JDK 7 vs JDK 8. Patch for master easily breaks for 1.x branch due
> to
> >this, and we just had to cancel another 1.1.0 RC vote.
> >
> >We might want to discuss how to handle webservice like DRPC (My
> feeling is
> >that current approach is somewhat kinda hacky.), but we can initiate
> >different thread for more details.
> >
> >If we really need long time to discuss above, I'm even OK to revert
> DRPC
> >port and start 2.0.0 with webservices unported, (DRPC, UI, Logviewer)
> and
> >address them at 2.1 or other minor versions.
> >
> >Any other opinions?
> >
> >- Jungtaek Lim (HeartSaVioR)
> >
> >2017년 3월 1일 (수) 오전 9:27, Roshan Naik <ro...@hortonworks.com>님이 작성:
> >
> >Lately (about a week and half maybe) it has not been possible to get
> the
> >1.x-branch to build inside IntelliJ. None of the modules are able to
> locate
> >the LocalCluster class (which is a clojure class). Previously, every
> once
> >in a while, I used to get this problem in the storm-starter module
> and was
> >able to get around it by doing a mvn clean install –DskipTests on the
> cmd
> >line and then doing a full rebuild of the project in IntelliJ.
> >
> >Now the problem has become a lot more endemic (modules like storm-sql,
> >storm-*-examples, etc.). And the above workaround doesn’t help. Spent
> many
> >futile hours trying to work around the build issue within Intellij
> (both
> >2015 and 2016 versions and using different Clojure plugins).
> >
> >There seems to possible ways to move forward:
> >
> >
> >- Somebody here knows what magic to do work address this.
> And we
> >can all use that.
> >
> >- Bring in the java port of LocalCluster.clj from master
> branch to
> >1.x (STORM-1281). I spend a little time to see if this was easy to
> do, but
> >appears kind of complicated due to number of files involved and
> >dependencies on prior patches. Somebody familiar with the original
> porting
> >effort may be better person to take this up.
> >
> >Thoughts ?
> >
> >-Roshan
>
>
>
>
>
>
--
Name : Jungtaek Lim
Blog : http://medium.com/@heartsavior
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior
Re: Storm 1.x-branch won't build in IntelliJ anymore
Posted by Roshan Naik <ro...@hortonworks.com>.
If we leave 1.x in this state it will forever remain un-usable within IntelliJ .. hard to debug, UT etc. And 1.x will live for a while. Is rolling back some of the non-critical offending commits that are causing this an option ? i tried rolling back a bit but was unable to identify the offending commit.
-roshan
On 2/28/17, 9:13 PM, "Arun Iyer on behalf of Arun Mahadevan" <aiyer@hortonworks.com on behalf of arunm@apache.org> wrote:
>If we really need long time to discuss above, I'm even OK to revert DRPC
>port and start 2.0.0 with webservices unported, (DRPC, UI, Logviewer) and
>address them at 2.1 or other minor versions.
>
>Any other opinions?
>
Yes, it may be better to move to 2.0 sooner and migrate the pending components in 2.1. May be what we need is more rigorous testing of the master branch before we release 2.0.
We could also consider moving to JDK 8 for the next 1.x release and it would make porting patches to 1.x branch much easier (unless we want to do a JDK upgrade only with 2.0).
Thanks,
Arun
On 3/1/17, 7:40 AM, "Jungtaek Lim" <ka...@gmail.com> wrote:
>Thanks Roshan for bring this up.
>
>For me moving toward 2.0.0 makes more sense.
>
>I know master branch has similar issue (DRPC) and it also has
>not-yet-ported things but if we are going to struggle with 1.x branch issue
>again and again, Storm 2.0.0 will never come. Another recent headache issue
>is JDK 7 vs JDK 8. Patch for master easily breaks for 1.x branch due to
>this, and we just had to cancel another 1.1.0 RC vote.
>
>We might want to discuss how to handle webservice like DRPC (My feeling is
>that current approach is somewhat kinda hacky.), but we can initiate
>different thread for more details.
>
>If we really need long time to discuss above, I'm even OK to revert DRPC
>port and start 2.0.0 with webservices unported, (DRPC, UI, Logviewer) and
>address them at 2.1 or other minor versions.
>
>Any other opinions?
>
>- Jungtaek Lim (HeartSaVioR)
>
>2017년 3월 1일 (수) 오전 9:27, Roshan Naik <ro...@hortonworks.com>님이 작성:
>
>Lately (about a week and half maybe) it has not been possible to get the
>1.x-branch to build inside IntelliJ. None of the modules are able to locate
>the LocalCluster class (which is a clojure class). Previously, every once
>in a while, I used to get this problem in the storm-starter module and was
>able to get around it by doing a mvn clean install –DskipTests on the cmd
>line and then doing a full rebuild of the project in IntelliJ.
>
>Now the problem has become a lot more endemic (modules like storm-sql,
>storm-*-examples, etc.). And the above workaround doesn’t help. Spent many
>futile hours trying to work around the build issue within Intellij (both
>2015 and 2016 versions and using different Clojure plugins).
>
>There seems to possible ways to move forward:
>
>
>- Somebody here knows what magic to do work address this. And we
>can all use that.
>
>- Bring in the java port of LocalCluster.clj from master branch to
>1.x (STORM-1281). I spend a little time to see if this was easy to do, but
>appears kind of complicated due to number of files involved and
>dependencies on prior patches. Somebody familiar with the original porting
>effort may be better person to take this up.
>
>Thoughts ?
>
>-Roshan
Re: Storm 1.x-branch won't build in IntelliJ anymore
Posted by Arun Mahadevan <ar...@apache.org>.
>If we really need long time to discuss above, I'm even OK to revert DRPC
>port and start 2.0.0 with webservices unported, (DRPC, UI, Logviewer) and
>address them at 2.1 or other minor versions.
>
>Any other opinions?
>
Yes, it may be better to move to 2.0 sooner and migrate the pending components in 2.1. May be what we need is more rigorous testing of the master branch before we release 2.0.
We could also consider moving to JDK 8 for the next 1.x release and it would make porting patches to 1.x branch much easier (unless we want to do a JDK upgrade only with 2.0).
Thanks,
Arun
On 3/1/17, 7:40 AM, "Jungtaek Lim" <ka...@gmail.com> wrote:
>Thanks Roshan for bring this up.
>
>For me moving toward 2.0.0 makes more sense.
>
>I know master branch has similar issue (DRPC) and it also has
>not-yet-ported things but if we are going to struggle with 1.x branch issue
>again and again, Storm 2.0.0 will never come. Another recent headache issue
>is JDK 7 vs JDK 8. Patch for master easily breaks for 1.x branch due to
>this, and we just had to cancel another 1.1.0 RC vote.
>
>We might want to discuss how to handle webservice like DRPC (My feeling is
>that current approach is somewhat kinda hacky.), but we can initiate
>different thread for more details.
>
>If we really need long time to discuss above, I'm even OK to revert DRPC
>port and start 2.0.0 with webservices unported, (DRPC, UI, Logviewer) and
>address them at 2.1 or other minor versions.
>
>Any other opinions?
>
>- Jungtaek Lim (HeartSaVioR)
>
>2017년 3월 1일 (수) 오전 9:27, Roshan Naik <ro...@hortonworks.com>님이 작성:
>
>Lately (about a week and half maybe) it has not been possible to get the
>1.x-branch to build inside IntelliJ. None of the modules are able to locate
>the LocalCluster class (which is a clojure class). Previously, every once
>in a while, I used to get this problem in the storm-starter module and was
>able to get around it by doing a mvn clean install –DskipTests on the cmd
>line and then doing a full rebuild of the project in IntelliJ.
>
>Now the problem has become a lot more endemic (modules like storm-sql,
>storm-*-examples, etc.). And the above workaround doesn’t help. Spent many
>futile hours trying to work around the build issue within Intellij (both
>2015 and 2016 versions and using different Clojure plugins).
>
>There seems to possible ways to move forward:
>
>
>- Somebody here knows what magic to do work address this. And we
>can all use that.
>
>- Bring in the java port of LocalCluster.clj from master branch to
>1.x (STORM-1281). I spend a little time to see if this was easy to do, but
>appears kind of complicated due to number of files involved and
>dependencies on prior patches. Somebody familiar with the original porting
>effort may be better person to take this up.
>
>Thoughts ?
>
>-Roshan
Re: Storm 1.x-branch won't build in IntelliJ anymore
Posted by Jungtaek Lim <ka...@gmail.com>.
Thanks Roshan for bring this up.
For me moving toward 2.0.0 makes more sense.
I know master branch has similar issue (DRPC) and it also has
not-yet-ported things but if we are going to struggle with 1.x branch issue
again and again, Storm 2.0.0 will never come. Another recent headache issue
is JDK 7 vs JDK 8. Patch for master easily breaks for 1.x branch due to
this, and we just had to cancel another 1.1.0 RC vote.
We might want to discuss how to handle webservice like DRPC (My feeling is
that current approach is somewhat kinda hacky.), but we can initiate
different thread for more details.
If we really need long time to discuss above, I'm even OK to revert DRPC
port and start 2.0.0 with webservices unported, (DRPC, UI, Logviewer) and
address them at 2.1 or other minor versions.
Any other opinions?
- Jungtaek Lim (HeartSaVioR)
2017년 3월 1일 (수) 오전 9:27, Roshan Naik <ro...@hortonworks.com>님이 작성:
Lately (about a week and half maybe) it has not been possible to get the
1.x-branch to build inside IntelliJ. None of the modules are able to locate
the LocalCluster class (which is a clojure class). Previously, every once
in a while, I used to get this problem in the storm-starter module and was
able to get around it by doing a mvn clean install –DskipTests on the cmd
line and then doing a full rebuild of the project in IntelliJ.
Now the problem has become a lot more endemic (modules like storm-sql,
storm-*-examples, etc.). And the above workaround doesn’t help. Spent many
futile hours trying to work around the build issue within Intellij (both
2015 and 2016 versions and using different Clojure plugins).
There seems to possible ways to move forward:
- Somebody here knows what magic to do work address this. And we
can all use that.
- Bring in the java port of LocalCluster.clj from master branch to
1.x (STORM-1281). I spend a little time to see if this was easy to do, but
appears kind of complicated due to number of files involved and
dependencies on prior patches. Somebody familiar with the original porting
effort may be better person to take this up.
Thoughts ?
-Roshan