You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Reynold Xin <rx...@databricks.com.INVALID> on 2025/02/06 01:36:35 UTC
Re: [Spark SQL]: Are SQL User-Defined Functions on the Roadmap?
There's already one here https://issues.apache.org/jira/browse/SPARK-46057
On Wed, Feb 5, 2025 at 5:16 PM Soumasish <so...@gmail.com> wrote:
> Here I create one, https://issues.apache.org/jira/browse/SPARK-51102
>
> Best Regards
> Soumasish Goswami
> in: www.linkedin.com/in/soumasish
> # (415) 530-0405
>
> -
>
>
>
> On Wed, Feb 5, 2025 at 4:49 PM Frank Bertsch <fb...@netflix.com.invalid>
> wrote:
>
>> Thank you Mich.
>>
>> Hi Folks, any lead on this? Just a pointer to a Jira ticket or email
>> discussion would be great!
>>
>> -Frank
>>
>> On Fri, Jan 31, 2025 at 10:06 AM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>> Hi Frank,
>>>
>>> I think this would be for the Spark dev team. I have added to the email.
>>>
>>> HTH
>>>
>>> Dr Mich Talebzadeh,
>>> Architect | Data Science | Financial Crime | Forensic Analysis | GDPR
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, 31 Jan 2025 at 14:28, Frank Bertsch <fb...@netflix.com.invalid>
>>> wrote:
>>>
>>>> Hi All -
>>>>
>>>> I'm working heavily in Spark SQL lately. Specifically, I've been trying
>>>> to understand if SQL UDFs, similar to the databricks offering
>>>> <https://www.databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html>,
>>>> is being tracked as a feature request within Apache Spark, and if there's
>>>> work in progress / a timeline for availability.
>>>>
>>>> I've looked through the Issue Tracker
>>>> <http://issues.apache.org/jira/projects/SPARK/issues>, but wasn't able
>>>> to find anything. Thanks in advance for any help!
>>>>
>>>> -Frank
>>>>
>>>
Re: [Spark SQL]: Are SQL User-Defined Functions on the Roadmap?
Posted by Frank Bertsch <fb...@netflix.com.INVALID>.
Amazing, looking forward to trying it out, thank you Allison!
-Frank
On Tue, Feb 11, 2025 at 11:16 PM Allison Wang <al...@apache.org>
wrote:
> Hi Frank,
>
> I am actively working on SPARK-46057
> <https://issues.apache.org/jira/browse/SPARK-46057>. You should be able
> to try it out once the Spark 4.0 RC is out.
>
> Thanks,
> Allison
>
> On Wed, Feb 5, 2025 at 5:37 PM Reynold Xin <rx...@databricks.com.invalid>
> wrote:
>
>> There's already one here
>> https://issues.apache.org/jira/browse/SPARK-46057
>>
>>
>> On Wed, Feb 5, 2025 at 5:16 PM Soumasish <so...@gmail.com> wrote:
>>
>>> Here I create one, https://issues.apache.org/jira/browse/SPARK-51102
>>>
>>> Best Regards
>>> Soumasish Goswami
>>> in: www.linkedin.com/in/soumasish
>>> # (415) 530-0405
>>>
>>> -
>>>
>>>
>>>
>>> On Wed, Feb 5, 2025 at 4:49 PM Frank Bertsch
>>> <fb...@netflix.com.invalid> wrote:
>>>
>>>> Thank you Mich.
>>>>
>>>> Hi Folks, any lead on this? Just a pointer to a Jira ticket or email
>>>> discussion would be great!
>>>>
>>>> -Frank
>>>>
>>>> On Fri, Jan 31, 2025 at 10:06 AM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> Hi Frank,
>>>>>
>>>>> I think this would be for the Spark dev team. I have added to the
>>>>> email.
>>>>>
>>>>> HTH
>>>>>
>>>>> Dr Mich Talebzadeh,
>>>>> Architect | Data Science | Financial Crime | Forensic Analysis | GDPR
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, 31 Jan 2025 at 14:28, Frank Bertsch
>>>>> <fb...@netflix.com.invalid> wrote:
>>>>>
>>>>>> Hi All -
>>>>>>
>>>>>> I'm working heavily in Spark SQL lately. Specifically, I've been
>>>>>> trying to understand if SQL UDFs, similar to the databricks offering
>>>>>> <https://www.databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html>,
>>>>>> is being tracked as a feature request within Apache Spark, and if there's
>>>>>> work in progress / a timeline for availability.
>>>>>>
>>>>>> I've looked through the Issue Tracker
>>>>>> <http://issues.apache.org/jira/projects/SPARK/issues>, but wasn't
>>>>>> able to find anything. Thanks in advance for any help!
>>>>>>
>>>>>> -Frank
>>>>>>
>>>>>
Re: [Spark SQL]: Are SQL User-Defined Functions on the Roadmap?
Posted by Allison Wang <al...@apache.org>.
Hi Frank,
I am actively working on SPARK-46057
<https://issues.apache.org/jira/browse/SPARK-46057>. You should be able to
try it out once the Spark 4.0 RC is out.
Thanks,
Allison
On Wed, Feb 5, 2025 at 5:37 PM Reynold Xin <rx...@databricks.com.invalid>
wrote:
> There's already one here https://issues.apache.org/jira/browse/SPARK-46057
>
>
> On Wed, Feb 5, 2025 at 5:16 PM Soumasish <so...@gmail.com> wrote:
>
>> Here I create one, https://issues.apache.org/jira/browse/SPARK-51102
>>
>> Best Regards
>> Soumasish Goswami
>> in: www.linkedin.com/in/soumasish
>> # (415) 530-0405
>>
>> -
>>
>>
>>
>> On Wed, Feb 5, 2025 at 4:49 PM Frank Bertsch <fb...@netflix.com.invalid>
>> wrote:
>>
>>> Thank you Mich.
>>>
>>> Hi Folks, any lead on this? Just a pointer to a Jira ticket or email
>>> discussion would be great!
>>>
>>> -Frank
>>>
>>> On Fri, Jan 31, 2025 at 10:06 AM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> Hi Frank,
>>>>
>>>> I think this would be for the Spark dev team. I have added to the email.
>>>>
>>>> HTH
>>>>
>>>> Dr Mich Talebzadeh,
>>>> Architect | Data Science | Financial Crime | Forensic Analysis | GDPR
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, 31 Jan 2025 at 14:28, Frank Bertsch
>>>> <fb...@netflix.com.invalid> wrote:
>>>>
>>>>> Hi All -
>>>>>
>>>>> I'm working heavily in Spark SQL lately. Specifically, I've been
>>>>> trying to understand if SQL UDFs, similar to the databricks offering
>>>>> <https://www.databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html>,
>>>>> is being tracked as a feature request within Apache Spark, and if there's
>>>>> work in progress / a timeline for availability.
>>>>>
>>>>> I've looked through the Issue Tracker
>>>>> <http://issues.apache.org/jira/projects/SPARK/issues>, but wasn't
>>>>> able to find anything. Thanks in advance for any help!
>>>>>
>>>>> -Frank
>>>>>
>>>>
Re: [Spark SQL]: Are SQL User-Defined Functions on the Roadmap?
Posted by Allison Wang <al...@apache.org>.
Hi Frank,
I am actively working on SPARK-46057
<https://issues.apache.org/jira/browse/SPARK-46057>. You should be able to
try it out once the Spark 4.0 RC is out.
Thanks,
Allison
On Wed, Feb 5, 2025 at 5:37 PM Reynold Xin <rx...@databricks.com.invalid>
wrote:
> There's already one here https://issues.apache.org/jira/browse/SPARK-46057
>
>
> On Wed, Feb 5, 2025 at 5:16 PM Soumasish <so...@gmail.com> wrote:
>
>> Here I create one, https://issues.apache.org/jira/browse/SPARK-51102
>>
>> Best Regards
>> Soumasish Goswami
>> in: www.linkedin.com/in/soumasish
>> # (415) 530-0405
>>
>> -
>>
>>
>>
>> On Wed, Feb 5, 2025 at 4:49 PM Frank Bertsch <fb...@netflix.com.invalid>
>> wrote:
>>
>>> Thank you Mich.
>>>
>>> Hi Folks, any lead on this? Just a pointer to a Jira ticket or email
>>> discussion would be great!
>>>
>>> -Frank
>>>
>>> On Fri, Jan 31, 2025 at 10:06 AM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> Hi Frank,
>>>>
>>>> I think this would be for the Spark dev team. I have added to the email.
>>>>
>>>> HTH
>>>>
>>>> Dr Mich Talebzadeh,
>>>> Architect | Data Science | Financial Crime | Forensic Analysis | GDPR
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, 31 Jan 2025 at 14:28, Frank Bertsch
>>>> <fb...@netflix.com.invalid> wrote:
>>>>
>>>>> Hi All -
>>>>>
>>>>> I'm working heavily in Spark SQL lately. Specifically, I've been
>>>>> trying to understand if SQL UDFs, similar to the databricks offering
>>>>> <https://www.databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html>,
>>>>> is being tracked as a feature request within Apache Spark, and if there's
>>>>> work in progress / a timeline for availability.
>>>>>
>>>>> I've looked through the Issue Tracker
>>>>> <http://issues.apache.org/jira/projects/SPARK/issues>, but wasn't
>>>>> able to find anything. Thanks in advance for any help!
>>>>>
>>>>> -Frank
>>>>>
>>>>