You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by William Wong <wi...@gmail.com> on 2019/04/14 09:15:52 UTC
Antlr plugin for sql/catalyst project
Dear all,
I tried to follow the guide at 'http://spark.apache.org/developer-tools.html'
to setup an IntelliJ project for Spark. However, the project was failed to
build. It was due to missing classes generated via antlr on sql/catalyst
project.
I would like to enhance the document to hint other new joiners to run
'build/mvn antlr4 -f sql/catalyst/pom.xml' if hitting missing ANTLR4
classes files. However, Spark's project structure is very new to me. I hope
I did not miss any guideline on spark's documentation about this issue, if
so pls let me know. Thanks in advance.
Regards,
William
Re: Antlr plugin for sql/catalyst project
Posted by William Wong <wi...@gmail.com>.
Hi Sean,
I just submitted a PR for updating the develop-tools.html. (
https://github.com/apache/spark-website/pull/195). May i know if you may
help review it? Many thanks for your help.
Best regards,
William
On Mon, Apr 15, 2019 at 7:04 AM William Wong <wi...@gmail.com> wrote:
> I built the spark with build/mvn via terminal. It also downloaded maven
> with the right version. After changing the maven to the right version in
> IntelliJ, everything worked perfectly again.
>
> Thanks for the info. I will enhance that a bit later. Hope it helps the
> community.
>
> On Mon, 15 Apr 2019 at 1:57 AM, Sean Owen <sr...@gmail.com> wrote:
>
>> That's right, Spark needs Maven 3.6.0. Just install it locally and
>> then configure IntelliJ to use the local Maven, not built-in 3.3.9.
>> The docs you're looking at are actually in
>> github.com/apache/spark-website as they aren't version-specific and
>> tied to a release.
>>
>> On Sun, Apr 14, 2019 at 12:00 PM William Wong <wi...@gmail.com>
>> wrote:
>> >
>> > Hi Sean,
>> >
>> > I would like to open a PR for updating the documentation. However, i
>> cannot share any file for 'http://spark.apache.org/developer-tools.html'
>> It seems that this file is not a part of the documentation (under folder
>> docs)...
>> >
>> > Thanks and regards,
>> > William
>> >
>> > On Sun, Apr 14, 2019 at 11:58 PM William Wong <wi...@gmail.com>
>> wrote:
>> >>
>> >> Hi Sean,
>> >>
>> >> I tried the button, but antlr4 source was not generate as expected. I
>> checked the IntelliJ log and found some error message like:
>> >>
>> >> 2019-04-14 16:05:24,796 [ 314609] INFO -
>> #org.jetbrains.idea.maven - [WARNING] Rule 0:
>> org.apache.maven.plugins.enforcer.RequireMavenVersion failed with message:
>> >> Detected Maven Version: 3.3.9 is not in the allowed range 3.6.0.
>> >> 2019-04-14 16:05:24,813 [ 314626] INFO -
>> #org.jetbrains.idea.maven -
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce
>> (enforce-versions) on project spark-parent_2.12: Some Enforcer rules have
>> failed. Look above for specific messages explaining why the rule failed.
>> >> java.lang.RuntimeException:
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce
>> (enforce-versions) on project spark-parent_2.12: Some Enforcer rules have
>> failed. Look above for specific messages explaining why the rule failed.
>> >>
>> >>
>> >> It seems that the source file generation process failed sliently due
>> to incorrect maven version. My IntelliJ bundled maven 3.3.9 while Spark's
>> master branch requires maven 3.6.0. Be honest, failing an action sliently
>> should be an IntelliJ bug. But a note to guide spark developers to set the
>> maven version for IntelliJ should be helpful. I just create a JIRA (
>> https://issues.apache.org/jira/browse/SPARK-27458) for this.
>> >>
>> >> Thanks and regards,
>> >> William
>> >>
>> >> On Sun, 14 Apr 2019 at 8:57 PM, Sean Owen <sr...@gmail.com> wrote:
>> >>>
>> >>> For IntelliJ, in the Maven pane, there's a button to generate all
>> >>> sources and resources that the build creates. That's the easier
>> >>> option. You can open a PR to add a note about it along with other docs
>> >>> for IntelliJ users.
>> >>>
>> >>> On Sun, Apr 14, 2019 at 4:24 AM William Wong <wi...@gmail.com>
>> wrote:
>> >>> >
>> >>> > Dear all,
>> >>> >
>> >>> > I tried to follow the guide at '
>> http://spark.apache.org/developer-tools.html' to setup an IntelliJ
>> project for Spark. However, the project was failed to build. It was due to
>> missing classes generated via antlr on sql/catalyst project.
>> >>> >
>> >>> > I would like to enhance the document to hint other new joiners to
>> run 'build/mvn antlr4 -f sql/catalyst/pom.xml' if hitting missing ANTLR4
>> classes files. However, Spark's project structure is very new to me. I hope
>> I did not miss any guideline on spark's documentation about this issue, if
>> so pls let me know. Thanks in advance.
>> >>> >
>> >>> > Regards,
>> >>> > William
>> >>> >
>> >>> >
>>
>
Re: Antlr plugin for sql/catalyst project
Posted by William Wong <wi...@gmail.com>.
I built the spark with build/mvn via terminal. It also downloaded maven
with the right version. After changing the maven to the right version in
IntelliJ, everything worked perfectly again.
Thanks for the info. I will enhance that a bit later. Hope it helps the
community.
On Mon, 15 Apr 2019 at 1:57 AM, Sean Owen <sr...@gmail.com> wrote:
> That's right, Spark needs Maven 3.6.0. Just install it locally and
> then configure IntelliJ to use the local Maven, not built-in 3.3.9.
> The docs you're looking at are actually in
> github.com/apache/spark-website as they aren't version-specific and
> tied to a release.
>
> On Sun, Apr 14, 2019 at 12:00 PM William Wong <wi...@gmail.com>
> wrote:
> >
> > Hi Sean,
> >
> > I would like to open a PR for updating the documentation. However, i
> cannot share any file for 'http://spark.apache.org/developer-tools.html'
> It seems that this file is not a part of the documentation (under folder
> docs)...
> >
> > Thanks and regards,
> > William
> >
> > On Sun, Apr 14, 2019 at 11:58 PM William Wong <wi...@gmail.com>
> wrote:
> >>
> >> Hi Sean,
> >>
> >> I tried the button, but antlr4 source was not generate as expected. I
> checked the IntelliJ log and found some error message like:
> >>
> >> 2019-04-14 16:05:24,796 [ 314609] INFO -
> #org.jetbrains.idea.maven - [WARNING] Rule 0:
> org.apache.maven.plugins.enforcer.RequireMavenVersion failed with message:
> >> Detected Maven Version: 3.3.9 is not in the allowed range 3.6.0.
> >> 2019-04-14 16:05:24,813 [ 314626] INFO -
> #org.jetbrains.idea.maven -
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce
> (enforce-versions) on project spark-parent_2.12: Some Enforcer rules have
> failed. Look above for specific messages explaining why the rule failed.
> >> java.lang.RuntimeException:
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce
> (enforce-versions) on project spark-parent_2.12: Some Enforcer rules have
> failed. Look above for specific messages explaining why the rule failed.
> >>
> >>
> >> It seems that the source file generation process failed sliently due to
> incorrect maven version. My IntelliJ bundled maven 3.3.9 while Spark's
> master branch requires maven 3.6.0. Be honest, failing an action sliently
> should be an IntelliJ bug. But a note to guide spark developers to set the
> maven version for IntelliJ should be helpful. I just create a JIRA (
> https://issues.apache.org/jira/browse/SPARK-27458) for this.
> >>
> >> Thanks and regards,
> >> William
> >>
> >> On Sun, 14 Apr 2019 at 8:57 PM, Sean Owen <sr...@gmail.com> wrote:
> >>>
> >>> For IntelliJ, in the Maven pane, there's a button to generate all
> >>> sources and resources that the build creates. That's the easier
> >>> option. You can open a PR to add a note about it along with other docs
> >>> for IntelliJ users.
> >>>
> >>> On Sun, Apr 14, 2019 at 4:24 AM William Wong <wi...@gmail.com>
> wrote:
> >>> >
> >>> > Dear all,
> >>> >
> >>> > I tried to follow the guide at '
> http://spark.apache.org/developer-tools.html' to setup an IntelliJ
> project for Spark. However, the project was failed to build. It was due to
> missing classes generated via antlr on sql/catalyst project.
> >>> >
> >>> > I would like to enhance the document to hint other new joiners to
> run 'build/mvn antlr4 -f sql/catalyst/pom.xml' if hitting missing ANTLR4
> classes files. However, Spark's project structure is very new to me. I hope
> I did not miss any guideline on spark's documentation about this issue, if
> so pls let me know. Thanks in advance.
> >>> >
> >>> > Regards,
> >>> > William
> >>> >
> >>> >
>
Re: Antlr plugin for sql/catalyst project
Posted by Sean Owen <sr...@gmail.com>.
That's right, Spark needs Maven 3.6.0. Just install it locally and
then configure IntelliJ to use the local Maven, not built-in 3.3.9.
The docs you're looking at are actually in
github.com/apache/spark-website as they aren't version-specific and
tied to a release.
On Sun, Apr 14, 2019 at 12:00 PM William Wong <wi...@gmail.com> wrote:
>
> Hi Sean,
>
> I would like to open a PR for updating the documentation. However, i cannot share any file for 'http://spark.apache.org/developer-tools.html' It seems that this file is not a part of the documentation (under folder docs)...
>
> Thanks and regards,
> William
>
> On Sun, Apr 14, 2019 at 11:58 PM William Wong <wi...@gmail.com> wrote:
>>
>> Hi Sean,
>>
>> I tried the button, but antlr4 source was not generate as expected. I checked the IntelliJ log and found some error message like:
>>
>> 2019-04-14 16:05:24,796 [ 314609] INFO - #org.jetbrains.idea.maven - [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireMavenVersion failed with message:
>> Detected Maven Version: 3.3.9 is not in the allowed range 3.6.0.
>> 2019-04-14 16:05:24,813 [ 314626] INFO - #org.jetbrains.idea.maven - org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce (enforce-versions) on project spark-parent_2.12: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed.
>> java.lang.RuntimeException: org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce (enforce-versions) on project spark-parent_2.12: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed.
>>
>>
>> It seems that the source file generation process failed sliently due to incorrect maven version. My IntelliJ bundled maven 3.3.9 while Spark's master branch requires maven 3.6.0. Be honest, failing an action sliently should be an IntelliJ bug. But a note to guide spark developers to set the maven version for IntelliJ should be helpful. I just create a JIRA (https://issues.apache.org/jira/browse/SPARK-27458) for this.
>>
>> Thanks and regards,
>> William
>>
>> On Sun, 14 Apr 2019 at 8:57 PM, Sean Owen <sr...@gmail.com> wrote:
>>>
>>> For IntelliJ, in the Maven pane, there's a button to generate all
>>> sources and resources that the build creates. That's the easier
>>> option. You can open a PR to add a note about it along with other docs
>>> for IntelliJ users.
>>>
>>> On Sun, Apr 14, 2019 at 4:24 AM William Wong <wi...@gmail.com> wrote:
>>> >
>>> > Dear all,
>>> >
>>> > I tried to follow the guide at 'http://spark.apache.org/developer-tools.html' to setup an IntelliJ project for Spark. However, the project was failed to build. It was due to missing classes generated via antlr on sql/catalyst project.
>>> >
>>> > I would like to enhance the document to hint other new joiners to run 'build/mvn antlr4 -f sql/catalyst/pom.xml' if hitting missing ANTLR4 classes files. However, Spark's project structure is very new to me. I hope I did not miss any guideline on spark's documentation about this issue, if so pls let me know. Thanks in advance.
>>> >
>>> > Regards,
>>> > William
>>> >
>>> >
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Antlr plugin for sql/catalyst project
Posted by William Wong <wi...@gmail.com>.
Hi Sean,
I would like to open a PR for updating the documentation. However, i cannot
share any file for 'http://spark.apache.org/developer-tools.html' It seems
that this file is not a part of the documentation (under folder docs)...
Thanks and regards,
William
On Sun, Apr 14, 2019 at 11:58 PM William Wong <wi...@gmail.com> wrote:
> Hi Sean,
>
> I tried the button, but antlr4 source was not generate as expected. I
> checked the IntelliJ log and found some error message like:
>
> *2019-04-14 16:05:24,796 [ 314609] INFO - #org.jetbrains.idea.maven
> - [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireMavenVersion
> failed with message:*
> *Detected Maven Version: 3.3.9 is not in the allowed range 3.6.0.*
> *2019-04-14 16:05:24,813 [ 314626] INFO - #org.jetbrains.idea.maven
> - org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce
> (enforce-versions) on project spark-parent_2.12: Some Enforcer rules have
> failed. Look above for specific messages explaining why the rule failed.*
> *java.lang.RuntimeException:
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce
> (enforce-versions) on project spark-parent_2.12: Some Enforcer rules have
> failed. Look above for specific messages explaining why the rule failed.*
>
>
> It seems that the source file generation process failed sliently due to
> incorrect maven version. My IntelliJ bundled maven 3.3.9 while Spark's
> master branch requires maven 3.6.0. Be honest, failing an action sliently
> should be an IntelliJ bug. But a note to guide spark developers to set the
> maven version for IntelliJ should be helpful. I just create a JIRA (
> https://issues.apache.org/jira/browse/SPARK-27458) for this.
>
> Thanks and regards,
> William
>
> On Sun, 14 Apr 2019 at 8:57 PM, Sean Owen <sr...@gmail.com> wrote:
>
>> For IntelliJ, in the Maven pane, there's a button to generate all
>> sources and resources that the build creates. That's the easier
>> option. You can open a PR to add a note about it along with other docs
>> for IntelliJ users.
>>
>> On Sun, Apr 14, 2019 at 4:24 AM William Wong <wi...@gmail.com>
>> wrote:
>> >
>> > Dear all,
>> >
>> > I tried to follow the guide at '
>> http://spark.apache.org/developer-tools.html' to setup an IntelliJ
>> project for Spark. However, the project was failed to build. It was due to
>> missing classes generated via antlr on sql/catalyst project.
>> >
>> > I would like to enhance the document to hint other new joiners to run
>> 'build/mvn antlr4 -f sql/catalyst/pom.xml' if hitting missing ANTLR4
>> classes files. However, Spark's project structure is very new to me. I hope
>> I did not miss any guideline on spark's documentation about this issue, if
>> so pls let me know. Thanks in advance.
>> >
>> > Regards,
>> > William
>> >
>> >
>>
>
Re: Antlr plugin for sql/catalyst project
Posted by William Wong <wi...@gmail.com>.
Hi Sean,
I tried the button, but antlr4 source was not generate as expected. I
checked the IntelliJ log and found some error message like:
*2019-04-14 16:05:24,796 [ 314609] INFO - #org.jetbrains.idea.maven
- [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireMavenVersion
failed with message:*
*Detected Maven Version: 3.3.9 is not in the allowed range 3.6.0.*
*2019-04-14 16:05:24,813 [ 314626] INFO - #org.jetbrains.idea.maven
- org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce
(enforce-versions) on project spark-parent_2.12: Some Enforcer rules have
failed. Look above for specific messages explaining why the rule failed.*
*java.lang.RuntimeException:
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce
(enforce-versions) on project spark-parent_2.12: Some Enforcer rules have
failed. Look above for specific messages explaining why the rule failed.*
It seems that the source file generation process failed sliently due to
incorrect maven version. My IntelliJ bundled maven 3.3.9 while Spark's
master branch requires maven 3.6.0. Be honest, failing an action sliently
should be an IntelliJ bug. But a note to guide spark developers to set the
maven version for IntelliJ should be helpful. I just create a JIRA (
https://issues.apache.org/jira/browse/SPARK-27458) for this.
Thanks and regards,
William
On Sun, 14 Apr 2019 at 8:57 PM, Sean Owen <sr...@gmail.com> wrote:
> For IntelliJ, in the Maven pane, there's a button to generate all
> sources and resources that the build creates. That's the easier
> option. You can open a PR to add a note about it along with other docs
> for IntelliJ users.
>
> On Sun, Apr 14, 2019 at 4:24 AM William Wong <wi...@gmail.com>
> wrote:
> >
> > Dear all,
> >
> > I tried to follow the guide at '
> http://spark.apache.org/developer-tools.html' to setup an IntelliJ
> project for Spark. However, the project was failed to build. It was due to
> missing classes generated via antlr on sql/catalyst project.
> >
> > I would like to enhance the document to hint other new joiners to run
> 'build/mvn antlr4 -f sql/catalyst/pom.xml' if hitting missing ANTLR4
> classes files. However, Spark's project structure is very new to me. I hope
> I did not miss any guideline on spark's documentation about this issue, if
> so pls let me know. Thanks in advance.
> >
> > Regards,
> > William
> >
> >
>
Re: Antlr plugin for sql/catalyst project
Posted by Sean Owen <sr...@gmail.com>.
For IntelliJ, in the Maven pane, there's a button to generate all
sources and resources that the build creates. That's the easier
option. You can open a PR to add a note about it along with other docs
for IntelliJ users.
On Sun, Apr 14, 2019 at 4:24 AM William Wong <wi...@gmail.com> wrote:
>
> Dear all,
>
> I tried to follow the guide at 'http://spark.apache.org/developer-tools.html' to setup an IntelliJ project for Spark. However, the project was failed to build. It was due to missing classes generated via antlr on sql/catalyst project.
>
> I would like to enhance the document to hint other new joiners to run 'build/mvn antlr4 -f sql/catalyst/pom.xml' if hitting missing ANTLR4 classes files. However, Spark's project structure is very new to me. I hope I did not miss any guideline on spark's documentation about this issue, if so pls let me know. Thanks in advance.
>
> Regards,
> William
>
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org