You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Daniel Filev <da...@graphwise.ai.INVALID> on 2025/05/30 08:40:57 UTC
Inquiry: Extending Spark ML Support via Spark Connect to Scala/Java APIs (SPARK-50812 Analogue)
Dear Apache Spark Community/Development Team,
I hope this message finds you well.
I am writing to inquire about the roadmap and future plans for extending
Spark ML support through Spark Connect to the Scala API in a manner
analogous to SPARK-50812. Specifically, my team is very interested in
leveraging Spark Connect to invoke ML pipelines from a Java-based service;
however, the current implementation primarily targets Python.
Background & Motivation:
-
SPARK-50812 introduced API support for Python ML via Spark Connect.
-
We would like to understand whether there are any ongoing efforts or
planned JIRA tickets for a similar extension enabling Scala (and by
extension Java) ML usage through Spark Connect.
-
Our use case is to embed ML pipelines directly within a Java service,
taking advantage of Spark Connect’s client-server architecture without
resorting to Python.
My Questions:
1.
Roadmap Status: Are there any existing proposals or tickets related to
adding Scala/Java ML bindings in Spark Connect?
2.
Timeline: If so, could you share an anticipated timeline or milestones
for these enhancements?
3.
Contribution Opportunities: Would the community welcome contributions
toward this feature, and if so, are there any guidelines or specifications
we should follow?
References*:*
-
SPARK-50812: https://issues.apache.org/jira/browse/SPARK-50812
Understanding your timeline and priorities will greatly assist us in
planning our integration strategy and, if feasible, contributing to the
effort.
Thank you for your time and for your ongoing work on Apache Spark. I look
forward to your guidance.
--
Daniel Filev
Software Engineer
Ontotext doing business as Graphwise | Making sense of data one triple at a
time.
https://graphwise.ai
<http://graphwise.ai/>
LinkedIn <https://www.linkedin.com/company/graphwise/> | X
<https://x.com/graphwise>
Re: Inquiry: Extending Spark ML Support via Spark Connect to Scala/Java APIs (SPARK-50812 Analogue)
Posted by Daniel Filev <da...@graphwise.ai.INVALID>.
Dear Apache Spark Community/Development Team,
I was wondering whether you had a chance to take a look at my previous
email. I would appreciate any and all information which you could provide
on the aforementioned points.
I hope all is well on your end and do thank you for your time
and consideration.
Best wishes,
Daniel
On Fri, May 30, 2025 at 11:40 AM Daniel Filev <da...@graphwise.ai>
wrote:
> Dear Apache Spark Community/Development Team,
>
> I hope this message finds you well.
>
> I am writing to inquire about the roadmap and future plans for extending
> Spark ML support through Spark Connect to the Scala API in a manner
> analogous to SPARK-50812. Specifically, my team is very interested in
> leveraging Spark Connect to invoke ML pipelines from a Java-based service;
> however, the current implementation primarily targets Python.
>
> Background & Motivation:
>
> -
>
> SPARK-50812 introduced API support for Python ML via Spark Connect.
> -
>
> We would like to understand whether there are any ongoing efforts or
> planned JIRA tickets for a similar extension enabling Scala (and by
> extension Java) ML usage through Spark Connect.
> -
>
> Our use case is to embed ML pipelines directly within a Java service,
> taking advantage of Spark Connect’s client-server architecture without
> resorting to Python.
>
> My Questions:
>
> 1.
>
> Roadmap Status: Are there any existing proposals or tickets related to
> adding Scala/Java ML bindings in Spark Connect?
> 2.
>
> Timeline: If so, could you share an anticipated timeline or milestones
> for these enhancements?
> 3.
>
> Contribution Opportunities: Would the community welcome contributions
> toward this feature, and if so, are there any guidelines or specifications
> we should follow?
>
> References*:*
>
> -
>
> SPARK-50812: https://issues.apache.org/jira/browse/SPARK-50812
>
> Understanding your timeline and priorities will greatly assist us in
> planning our integration strategy and, if feasible, contributing to the
> effort.
>
> Thank you for your time and for your ongoing work on Apache Spark. I look
> forward to your guidance.
> --
> Daniel Filev
> Software Engineer
> Ontotext doing business as Graphwise | Making sense of data one triple at
> a time.
> https://graphwise.ai
> <http://graphwise.ai/>
> LinkedIn <https://www.linkedin.com/company/graphwise/> | X
> <https://x.com/graphwise>
>
--
Daniel Filev
Software Engineer
Ontotext doing business as Graphwise | Making sense of data one triple at a
time.
https://graphwise.ai
<http://graphwise.ai/>
LinkedIn <https://www.linkedin.com/company/graphwise/> | X
<https://x.com/graphwise>
Fwd: Inquiry: Extending Spark ML Support via Spark Connect to Scala/Java APIs (SPARK-50812 Analogue)
Posted by Jules Damji <ju...@gmail.com>.
Excuse the thumb typos
---------- Forwarded message ---------
From: Daniel Filev <da...@graphwise.ai.invalid>
Date: Fri, 30 May 2025 at 5:07 AM
Subject: Inquiry: Extending Spark ML Support via Spark Connect to
Scala/Java APIs (SPARK-50812 Analogue)
To: <us...@spark.apache.org>
Dear Apache Spark Community/Development Team,
I hope this message finds you well.
I am writing to inquire about the roadmap and future plans for extending
Spark ML support through Spark Connect to the Scala API in a manner
analogous to SPARK-50812. Specifically, my team is very interested in
leveraging Spark Connect to invoke ML pipelines from a Java-based service;
however, the current implementation primarily targets Python.
Background & Motivation:
-
SPARK-50812 introduced API support for Python ML via Spark Connect.
-
We would like to understand whether there are any ongoing efforts or
planned JIRA tickets for a similar extension enabling Scala (and by
extension Java) ML usage through Spark Connect.
-
Our use case is to embed ML pipelines directly within a Java service,
taking advantage of Spark Connect’s client-server architecture without
resorting to Python.
My Questions:
1.
Roadmap Status: Are there any existing proposals or tickets related to
adding Scala/Java ML bindings in Spark Connect?
2.
Timeline: If so, could you share an anticipated timeline or milestones
for these enhancements?
3.
Contribution Opportunities: Would the community welcome contributions
toward this feature, and if so, are there any guidelines or specifications
we should follow?
References*:*
-
SPARK-50812: https://issues.apache.org/jira/browse/SPARK-50812
Understanding your timeline and priorities will greatly assist us in
planning our integration strategy and, if feasible, contributing to the
effort.
Thank you for your time and for your ongoing work on Apache Spark. I look
forward to your guidance.
--
Daniel Filev
Software Engineer
Ontotext doing business as Graphwise | Making sense of data one triple at a
time.
https://graphwise.ai
<http://graphwise.ai/>
LinkedIn <https://www.linkedin.com/company/graphwise/> | X
<https://x.com/graphwise>