You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Jia Fan <fa...@apache.org> on 2023/03/08 06:01:54 UTC
[Question] Can't start Spark Connect
Hi developers,
I want to contribute some code for Spark Connect. Any doc for starters?
I want to debug SimpleSparkConnectService but I can't start it with IDEA. I
would appreciate any help.
Thanks
________________________
Jia Fan
Re: Undelivered Mail Returned to Sender
Posted by Jia Fan <fa...@apache.org>.
Hi guys,
After I reload and rebuild project. Finally I run test case successfully.
Thanks for you help, Herman and Hyukjin.
Mail Delivery System <MA...@mxout1-he-de.apache.org> 于2023年3月8日周三
22:38写道：
> This is the mail system at host mxout1-he-de.apache.org.
>
> I'm sorry to have to inform you that your message could not
> be delivered to one or more recipients. It's attached below.
>
> For further assistance, please send mail to postmaster.
>
> If you do so, please include this problem report. You can
> delete your own text from the attached returned message.
>
> The mail system
>
> <he...@databricks.com.invalid>: Host or domain name not found. Name
> service
> error for name=databricks.com.invalid type=AAAA: Host not found
>
>
>
> ---------- Forwarded message ----------
> From: Jia Fan <fa...@apache.org>
> To: Herman van Hovell <he...@databricks.com.invalid>
> Cc: Hyukjin Kwon <gu...@gmail.com>, dev@spark.apache.org
> Bcc:
> Date: Wed, 8 Mar 2023 22:37:33 +0800
> Subject: Re: [Question] Can't start Spark Connect
> Hi Herman,
> I just use ./build/mvn -DskipTests clean package, also had try to
> use ./build/mvn -DskipTests clean install.
>
> Herman van Hovell <he...@databricks.com.invalid> 于2023年3月8日周三 21:17写道：
>
>> Hi Jia,
>>
>> How are you building connect?
>>
>> Kind regards,
>> Herman
>>
>> On Wed, Mar 8, 2023 at 8:48 AM Jia Fan <fa...@apache.org> wrote:
>>
>>> Thanks for reply,
>>> I had done clean build with maven few times. But always report
>>>
>>> /Users/xxx/Code/spark/core/target/generated-sources/org/apache/spark/status/protobuf/StoreTypes.java:658:9
>>> java: symbol not found
>>> Symbol: class UnusedPrivateParameter
>>> Location: class org.apache.spark.status.protobuf.StoreTypes.JobData
>>> I think maybe it's protobuf version conflict?
>>> <img width="1149" alt="image" src="
>>> https://user-images.githubusercontent.com/32387433/223716946-85761a34-f86c-4ba1-9557-a59d0d5b9958.png
>>> ">
>>>
>>>
>>> Hyukjin Kwon <gu...@gmail.com> 于2023年3月8日周三 19:09写道：
>>>
>>>> Just doing a clean build with Maven, and running a test case like
>>>> `SparkConnectServiceSuite` in IntelliJ should work.
>>>>
>>>> On Wed, 8 Mar 2023 at 15:02, Jia Fan <fa...@apache.org> wrote:
>>>>
>>>>> Hi developers,
>>>>> I want to contribute some code for Spark Connect. Any doc for
>>>>> starters? I want to debug SimpleSparkConnectService but I can't start it
>>>>> with IDEA. I would appreciate any help.
>>>>>
>>>>> Thanks
>>>>>
>>>>> ________________________
>>>>>
>>>>>
>>>>> Jia Fan
>>>>>
>>>>
Re: [Question] Can't start Spark Connect
Posted by Haejoon Lee <ha...@databricks.com.INVALID>.
Additionally, try deleting the `.idea` in the spark home directory and
restarting IntelliJ if it does not work properly after re-building during
development.
The .idea stores IntelliJ's project configuration and settings, and is
automatically generated when IntelliJ is launched.
>
Re: [Question] Can't start Spark Connect
Posted by yangjie01 <ya...@baidu.com>.
Yeah, after executing `./build/mvn -DskipTests clean package ` on the command line, you may need to manually reload maven projects in intellij, otherwise intellij will not immediately respond to the command line behavior.
Yang Jie
发件人: Lucifer Tyrant <ty...@apache.org>
日期: 2023年3月9日 星期四 00:11
收件人: Jia Fan <fa...@apache.org>
抄送: Herman van Hovell <he...@databricks.com.invalid>, Hyukjin Kwon <gu...@gmail.com>, "dev@spark.apache.org" <de...@spark.apache.org>
主题: Re: [Question] Can't start Spark Connect
It worked. Thank you for your guidance.
Jia Fan <fa...@apache.org>> 于2023年3月8日周三 22:38写道：
Hi Herman,
I just use ./build/mvn -DskipTests clean package, also had try to use ./build/mvn -DskipTests clean install.
Herman van Hovell <he...@databricks.com.invalid> 于2023年3月8日周三 21:17写道：
Hi Jia,
How are you building connect?
Kind regards,
Herman
On Wed, Mar 8, 2023 at 8:48 AM Jia Fan <fa...@apache.org>> wrote:
Thanks for reply,
I had done clean build with maven few times. But always report
/Users/xxx/Code/spark/core/target/generated-sources/org/apache/spark/status/protobuf/StoreTypes.java:658:9
java: symbol not found
Symbol: class UnusedPrivateParameter
Location: class org.apache.spark.status.protobuf.StoreTypes.JobData
I think maybe it's protobuf version conflict?
<img width="1149" alt="image" src="https://user-images.githubusercontent.com/32387433/223716946-85761a34-f86c-4ba1-9557-a59d0d5b9958.png<https://mailshield.baidu.com/check?q=SidJoMcixvn95yt28ggG9Um36qdwq6Y%2flT0198ypCSRr6UYhgHmOUEq5pcoKAkttDuBnvSSPJ6%2bp%2b0%2bUGja7oFKqKzySW0D%2fBV2mcG0%2fEozF7G%2fSJEz9p7BfNXiYQFvRgoOV1VxiKmNU%2bv1x>">
Hyukjin Kwon <gu...@gmail.com>> 于2023年3月8日周三 19:09写道：
Just doing a clean build with Maven, and running a test case like `SparkConnectServiceSuite` in IntelliJ should work.
On Wed, 8 Mar 2023 at 15:02, Jia Fan <fa...@apache.org>> wrote:
Hi developers,
I want to contribute some code for Spark Connect. Any doc for starters? I want to debug SimpleSparkConnectService but I can't start it with IDEA. I would appreciate any help.
Thanks
________________________
Jia Fan
Re: [Question] Can't start Spark Connect
Posted by Lucifer Tyrant <ty...@apache.org>.
It worked. Thank you for your guidance.
Jia Fan <fa...@apache.org> 于2023年3月8日周三 22:38写道：
> Hi Herman,
> I just use ./build/mvn -DskipTests clean package, also had try to
> use ./build/mvn -DskipTests clean install.
>
> Herman van Hovell <he...@databricks.com.invalid> 于2023年3月8日周三 21:17写道：
>
>> Hi Jia,
>>
>> How are you building connect?
>>
>> Kind regards,
>> Herman
>>
>> On Wed, Mar 8, 2023 at 8:48 AM Jia Fan <fa...@apache.org> wrote:
>>
>>> Thanks for reply,
>>> I had done clean build with maven few times. But always report
>>>
>>> /Users/xxx/Code/spark/core/target/generated-sources/org/apache/spark/status/protobuf/StoreTypes.java:658:9
>>> java: symbol not found
>>> Symbol: class UnusedPrivateParameter
>>> Location: class org.apache.spark.status.protobuf.StoreTypes.JobData
>>> I think maybe it's protobuf version conflict?
>>> <img width="1149" alt="image" src="
>>> https://user-images.githubusercontent.com/32387433/223716946-85761a34-f86c-4ba1-9557-a59d0d5b9958.png
>>> ">
>>>
>>>
>>> Hyukjin Kwon <gu...@gmail.com> 于2023年3月8日周三 19:09写道：
>>>
>>>> Just doing a clean build with Maven, and running a test case like
>>>> `SparkConnectServiceSuite` in IntelliJ should work.
>>>>
>>>> On Wed, 8 Mar 2023 at 15:02, Jia Fan <fa...@apache.org> wrote:
>>>>
>>>>> Hi developers,
>>>>> I want to contribute some code for Spark Connect. Any doc for
>>>>> starters? I want to debug SimpleSparkConnectService but I can't start it
>>>>> with IDEA. I would appreciate any help.
>>>>>
>>>>> Thanks
>>>>>
>>>>> ________________________
>>>>>
>>>>>
>>>>> Jia Fan
>>>>>
>>>>
Re: [Question] Can't start Spark Connect
Posted by Jia Fan <fa...@apache.org>.
Hi Herman,
I just use ./build/mvn -DskipTests clean package, also had try to
use ./build/mvn -DskipTests clean install.
Herman van Hovell <he...@databricks.com.invalid> 于2023年3月8日周三 21:17写道：
> Hi Jia,
>
> How are you building connect?
>
> Kind regards,
> Herman
>
> On Wed, Mar 8, 2023 at 8:48 AM Jia Fan <fa...@apache.org> wrote:
>
>> Thanks for reply,
>> I had done clean build with maven few times. But always report
>>
>> /Users/xxx/Code/spark/core/target/generated-sources/org/apache/spark/status/protobuf/StoreTypes.java:658:9
>> java: symbol not found
>> Symbol: class UnusedPrivateParameter
>> Location: class org.apache.spark.status.protobuf.StoreTypes.JobData
>> I think maybe it's protobuf version conflict?
>> <img width="1149" alt="image" src="
>> https://user-images.githubusercontent.com/32387433/223716946-85761a34-f86c-4ba1-9557-a59d0d5b9958.png
>> ">
>>
>>
>> Hyukjin Kwon <gu...@gmail.com> 于2023年3月8日周三 19:09写道：
>>
>>> Just doing a clean build with Maven, and running a test case like
>>> `SparkConnectServiceSuite` in IntelliJ should work.
>>>
>>> On Wed, 8 Mar 2023 at 15:02, Jia Fan <fa...@apache.org> wrote:
>>>
>>>> Hi developers,
>>>> I want to contribute some code for Spark Connect. Any doc for
>>>> starters? I want to debug SimpleSparkConnectService but I can't start it
>>>> with IDEA. I would appreciate any help.
>>>>
>>>> Thanks
>>>>
>>>> ________________________
>>>>
>>>>
>>>> Jia Fan
>>>>
>>>
Re: [Question] Can't start Spark Connect
Posted by Herman van Hovell <he...@databricks.com.INVALID>.
Hi Jia,
How are you building connect?
Kind regards,
Herman
On Wed, Mar 8, 2023 at 8:48 AM Jia Fan <fa...@apache.org> wrote:
> Thanks for reply,
> I had done clean build with maven few times. But always report
>
> /Users/xxx/Code/spark/core/target/generated-sources/org/apache/spark/status/protobuf/StoreTypes.java:658:9
> java: symbol not found
> Symbol: class UnusedPrivateParameter
> Location: class org.apache.spark.status.protobuf.StoreTypes.JobData
> I think maybe it's protobuf version conflict?
> <img width="1149" alt="image" src="
> https://user-images.githubusercontent.com/32387433/223716946-85761a34-f86c-4ba1-9557-a59d0d5b9958.png
> ">
>
>
> Hyukjin Kwon <gu...@gmail.com> 于2023年3月8日周三 19:09写道：
>
>> Just doing a clean build with Maven, and running a test case like
>> `SparkConnectServiceSuite` in IntelliJ should work.
>>
>> On Wed, 8 Mar 2023 at 15:02, Jia Fan <fa...@apache.org> wrote:
>>
>>> Hi developers,
>>> I want to contribute some code for Spark Connect. Any doc for
>>> starters? I want to debug SimpleSparkConnectService but I can't start it
>>> with IDEA. I would appreciate any help.
>>>
>>> Thanks
>>>
>>> ________________________
>>>
>>>
>>> Jia Fan
>>>
>>
Re: [Question] Can't start Spark Connect
Posted by Jia Fan <fa...@apache.org>.
Thanks for reply,
I had done clean build with maven few times. But always report
/Users/xxx/Code/spark/core/target/generated-sources/org/apache/spark/status/protobuf/StoreTypes.java:658:9
java: symbol not found
Symbol: class UnusedPrivateParameter
Location: class org.apache.spark.status.protobuf.StoreTypes.JobData
I think maybe it's protobuf version conflict?
<img width="1149" alt="image" src="
https://user-images.githubusercontent.com/32387433/223716946-85761a34-f86c-4ba1-9557-a59d0d5b9958.png
">
Hyukjin Kwon <gu...@gmail.com> 于2023年3月8日周三 19:09写道：
> Just doing a clean build with Maven, and running a test case like
> `SparkConnectServiceSuite` in IntelliJ should work.
>
> On Wed, 8 Mar 2023 at 15:02, Jia Fan <fa...@apache.org> wrote:
>
>> Hi developers,
>> I want to contribute some code for Spark Connect. Any doc for
>> starters? I want to debug SimpleSparkConnectService but I can't start it
>> with IDEA. I would appreciate any help.
>>
>> Thanks
>>
>> ________________________
>>
>>
>> Jia Fan
>>
>
Re: [Question] Can't start Spark Connect
Posted by Hyukjin Kwon <gu...@gmail.com>.
Just doing a clean build with Maven, and running a test case like
`SparkConnectServiceSuite` in IntelliJ should work.
On Wed, 8 Mar 2023 at 15:02, Jia Fan <fa...@apache.org> wrote:
> Hi developers,
> I want to contribute some code for Spark Connect. Any doc for starters?
> I want to debug SimpleSparkConnectService but I can't start it with IDEA. I
> would appreciate any help.
>
> Thanks
>
> ________________________
>
>
> Jia Fan
>