You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Raj ks <ra...@gmail.com> on 2022/10/21 16:11:59 UTC
Prometheus with spark
Hi Team,
We wanted to query Prometheus data with spark. Any suggestions will
be appreciated
Searched for documents but did not got any prompt one
RE: CVE-2022-42889
Posted by "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexisrisk.com.INVALID>.
Thanks again Sean!
From: Sean Owen <sr...@gmail.com>
Sent: Thursday, October 27, 2022 11:56 AM
To: Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com>
Cc: dev@spark.apache.org
Subject: Re: CVE-2022-42889
You don't often get email from srowen@gmail.com<ma...@gmail.com>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
*** External email: use caution ***
Right. It seems there is only one direct use of that part of commons-text, and it is not applied to user-supplied inputs (reads and substitutes into error message templates).
At a glance I do not see how it would affect Spark; it's not impossible that it does. In any event, commons-text is being updated anyway in branch 3.2 and later, so this will be updated in maintained branches eventually. It missed the 3.3.1 release, but my message is, it's also not even clear it matters to Spark.
I don't think this would become a Spark CVE; it affects commons-text. Sometimes CVEs note other affected software products when they are widely-used and very directly affected. But typically they would not list every single downstream user, let alone generate separate CVEs, and in any event here I do not see an argument that it affects Spark anyway.
On Thu, Oct 27, 2022 at 10:08 AM Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com>> wrote:
Thanks Sean,
I assume Spark's not affected because it either doesn't reference the affected API(s) or because it does not unsafely utilize user input through the vulnerable API(s), but is there an official statement about this from Spark?
We weren't able to find references to 2022-42889 here: https://spark.apache.org/security.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspark.apache.org%2Fsecurity.html&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C68d3f984b2414e34d77f08dab833d63f%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024830038882908%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Tr5LXzg7O%2F6kTMW9FOdqENU2IYAWMEJ3Aesjr1vPTtY%3D&reserved=0> (likely because Spark determined it is not affected?)
From: Sean Owen <sr...@gmail.com>>
Sent: Thursday, October 27, 2022 10:27 AM
To: Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com.invalid>>
Cc: dev@spark.apache.org<ma...@spark.apache.org>
Subject: Re: CVE-2022-42889
You don't often get email from srowen@gmail.com<ma...@gmail.com>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
*** External email: use caution ***
Probably a few months between maintenance releases.
It does not appear to affect Spark, however.
On Thu, Oct 27, 2022 at 9:24 AM Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com.invalid>> wrote:
Hello,
This issue (SPARK-40801)<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSPARK-40801&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C68d3f984b2414e34d77f08dab833d63f%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024830038882908%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=T28PKRz5aR3fX0f%2B3PINAqvAFiU5lSwyFBHj3jqKets%3D&reserved=0> which addresses CVE-2022-42889 doesn't seem to have been included in the latest release (3.3.1<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspark.apache.org%2Freleases%2Fspark-release-3-3-1.html&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C68d3f984b2414e34d77f08dab833d63f%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024830038882908%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QbQu8azxv7IgS64%2FG66GZyfPVNPv9rz7aOzRx3mTPaI%3D&reserved=0>).
Is there a way to estimate a timeline for the first release which includes that change (likely 3.3.2)? Much appreciation!
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
RE: CVE-2022-42889
Posted by "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexisrisk.com.INVALID>.
Thanks Steve, you're 100% correct, we're reacting to downstream customers being alerted by scanners to the presence of the "vulnerable" commons-text dependency.
We're looking for reliable information to convey downstream. Thanks again.
From: Steve Loughran <st...@cloudera.com>
Sent: Thursday, October 27, 2022 12:37 PM
To: Sean Owen <sr...@gmail.com>
Cc: Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com>; dev@spark.apache.org
Subject: Re: CVE-2022-42889
You don't often get email from stevel@cloudera.com<ma...@cloudera.com>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
*** External email: use caution ***
the api doesn't get used in the hadoop libraries; not sure about other dependencies.
probably makes sense to say on the jira that there's no need to panic here; I've had to start doing that as some of the security scanners appear to overreact
https://issues.apache.org/jira/browse/HDFS-16766<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHDFS-16766&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7Cab1cc34d9c3e403c8bf708dab8398a77%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024854534303468%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5OY0urII%2BlnXFmePhcKbKhQVbu5p%2BFo0oWXeORvL0mU%3D&reserved=0>
On Thu, 27 Oct 2022 at 16:56, Sean Owen <sr...@gmail.com>> wrote:
Right. It seems there is only one direct use of that part of commons-text, and it is not applied to user-supplied inputs (reads and substitutes into error message templates).
At a glance I do not see how it would affect Spark; it's not impossible that it does. In any event, commons-text is being updated anyway in branch 3.2 and later, so this will be updated in maintained branches eventually. It missed the 3.3.1 release, but my message is, it's also not even clear it matters to Spark.
I don't think this would become a Spark CVE; it affects commons-text. Sometimes CVEs note other affected software products when they are widely-used and very directly affected. But typically they would not list every single downstream user, let alone generate separate CVEs, and in any event here I do not see an argument that it affects Spark anyway.
On Thu, Oct 27, 2022 at 10:08 AM Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com>> wrote:
Thanks Sean,
I assume Spark's not affected because it either doesn't reference the affected API(s) or because it does not unsafely utilize user input through the vulnerable API(s), but is there an official statement about this from Spark?
We weren't able to find references to 2022-42889 here: https://spark.apache.org/security.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspark.apache.org%2Fsecurity.html&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7Cab1cc34d9c3e403c8bf708dab8398a77%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024854534303468%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ruLjGWPKmyUC6uGE9ONiOjgLYN6Tcg2cEWqzDQwP2Iw%3D&reserved=0> (likely because Spark determined it is not affected?)
From: Sean Owen <sr...@gmail.com>>
Sent: Thursday, October 27, 2022 10:27 AM
To: Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com.invalid>>
Cc: dev@spark.apache.org<ma...@spark.apache.org>
Subject: Re: CVE-2022-42889
You don't often get email from srowen@gmail.com<ma...@gmail.com>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
*** External email: use caution ***
Probably a few months between maintenance releases.
It does not appear to affect Spark, however.
On Thu, Oct 27, 2022 at 9:24 AM Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com.invalid>> wrote:
Hello,
This issue (SPARK-40801)<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSPARK-40801&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7Cab1cc34d9c3e403c8bf708dab8398a77%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024854534303468%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=trU3KpsjUMc1mCMUadFKcloRFKi3HjgrZv27zs0BJH4%3D&reserved=0> which addresses CVE-2022-42889 doesn't seem to have been included in the latest release (3.3.1<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspark.apache.org%2Freleases%2Fspark-release-3-3-1.html&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7Cab1cc34d9c3e403c8bf708dab8398a77%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024854534303468%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4SAXOZGPmTX0ScceE9MY2TyOqP76BnheKMGrPuzDYxs%3D&reserved=0>).
Is there a way to estimate a timeline for the first release which includes that change (likely 3.3.2)? Much appreciation!
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
Re: CVE-2022-42889
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
the api doesn't get used in the hadoop libraries; not sure about other
dependencies.
probably makes sense to say on the jira that there's no need to panic here;
I've had to start doing that as some of the security scanners appear to
overreact
https://issues.apache.org/jira/browse/HDFS-16766
On Thu, 27 Oct 2022 at 16:56, Sean Owen <sr...@gmail.com> wrote:
> Right. It seems there is only one direct use of that part of commons-text,
> and it is not applied to user-supplied inputs (reads and substitutes into
> error message templates).
> At a glance I do not see how it would affect Spark; it's not impossible
> that it does. In any event, commons-text is being updated anyway in branch
> 3.2 and later, so this will be updated in maintained branches eventually.
> It missed the 3.3.1 release, but my message is, it's also not even clear it
> matters to Spark.
>
> I don't think this would become a Spark CVE; it affects commons-text.
> Sometimes CVEs note other affected software products when they are
> widely-used and very directly affected. But typically they would not list
> every single downstream user, let alone generate separate CVEs, and in any
> event here I do not see an argument that it affects Spark anyway.
>
> On Thu, Oct 27, 2022 at 10:08 AM Pastrana, Rodrigo (RIS-BCT) <
> Rodrigo.Pastrana@lexisnexisrisk.com> wrote:
>
>> Thanks Sean,
>>
>> I assume Spark’s not affected because it either doesn’t reference the
>> affected API(s) or because it does not unsafely utilize user input through
>> the vulnerable API(s), but is there an official statement about this from
>> Spark?
>>
>> We weren’t able to find references to 2022-42889 here:
>> https://spark.apache.org/security.html (likely because Spark determined
>> it is not affected?)
>>
>>
>>
>> *From:* Sean Owen <sr...@gmail.com>
>> *Sent:* Thursday, October 27, 2022 10:27 AM
>> *To:* Pastrana, Rodrigo (RIS-BCT)
>> <Ro...@lexisnexisrisk.com.invalid>
>> *Cc:* dev@spark.apache.org
>> *Subject:* Re: CVE-2022-42889
>>
>>
>>
>> You don't often get email from srowen@gmail.com. Learn why this is
>> important <https://aka.ms/LearnAboutSenderIdentification>
>>
>> **** External email: use caution ****
>>
>>
>>
>> Probably a few months between maintenance releases.
>>
>> It does not appear to affect Spark, however.
>>
>>
>>
>> On Thu, Oct 27, 2022 at 9:24 AM Pastrana, Rodrigo (RIS-BCT) <
>> Rodrigo.Pastrana@lexisnexisrisk.com.invalid> wrote:
>>
>> Hello,
>>
>> This issue (SPARK-40801)
>> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSPARK-40801&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C507dc12538bf44d2646d08dab8276a76%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024776687375556%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=wZV1KpRw248DOPuWkJ2qjDNK9DwN4zFIgL6z2g0MOkw%3D&reserved=0>
>> which addresses CVE-2022-42889 doesn’t seem to have been included in the
>> latest release (3.3.1
>> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspark.apache.org%2Freleases%2Fspark-release-3-3-1.html&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C507dc12538bf44d2646d08dab8276a76%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024776687375556%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aJXVwPl36j83CFFM%2F1jKDhSIm7mCNwRozMpXCt8dvDQ%3D&reserved=0>
>> ).
>>
>> Is there a way to estimate a timeline for the first release which
>> includes that change (likely 3.3.2)? Much appreciation!
>>
>>
>> ------------------------------
>>
>> The information contained in this e-mail message is intended only for the
>> personal and confidential use of the recipient(s) named above. This message
>> may be an attorney-client communication and/or work product and as such is
>> privileged and confidential. If the reader of this message is not the
>> intended recipient or an agent responsible for delivering it to the
>> intended recipient, you are hereby notified that you have received this
>> document in error and that any review, dissemination, distribution, or
>> copying of this message is strictly prohibited. If you have received this
>> communication in error, please notify us immediately by e-mail, and delete
>> the original message.
>>
>>
>> ------------------------------
>> The information contained in this e-mail message is intended only for the
>> personal and confidential use of the recipient(s) named above. This message
>> may be an attorney-client communication and/or work product and as such is
>> privileged and confidential. If the reader of this message is not the
>> intended recipient or an agent responsible for delivering it to the
>> intended recipient, you are hereby notified that you have received this
>> document in error and that any review, dissemination, distribution, or
>> copying of this message is strictly prohibited. If you have received this
>> communication in error, please notify us immediately by e-mail, and delete
>> the original message.
>>
>
Re: CVE-2022-42889
Posted by Sean Owen <sr...@gmail.com>.
Right. It seems there is only one direct use of that part of commons-text,
and it is not applied to user-supplied inputs (reads and substitutes into
error message templates).
At a glance I do not see how it would affect Spark; it's not impossible
that it does. In any event, commons-text is being updated anyway in branch
3.2 and later, so this will be updated in maintained branches eventually.
It missed the 3.3.1 release, but my message is, it's also not even clear it
matters to Spark.
I don't think this would become a Spark CVE; it affects commons-text.
Sometimes CVEs note other affected software products when they are
widely-used and very directly affected. But typically they would not list
every single downstream user, let alone generate separate CVEs, and in any
event here I do not see an argument that it affects Spark anyway.
On Thu, Oct 27, 2022 at 10:08 AM Pastrana, Rodrigo (RIS-BCT) <
Rodrigo.Pastrana@lexisnexisrisk.com> wrote:
> Thanks Sean,
>
> I assume Spark’s not affected because it either doesn’t reference the
> affected API(s) or because it does not unsafely utilize user input through
> the vulnerable API(s), but is there an official statement about this from
> Spark?
>
> We weren’t able to find references to 2022-42889 here:
> https://spark.apache.org/security.html (likely because Spark determined
> it is not affected?)
>
>
>
> *From:* Sean Owen <sr...@gmail.com>
> *Sent:* Thursday, October 27, 2022 10:27 AM
> *To:* Pastrana, Rodrigo (RIS-BCT)
> <Ro...@lexisnexisrisk.com.invalid>
> *Cc:* dev@spark.apache.org
> *Subject:* Re: CVE-2022-42889
>
>
>
> You don't often get email from srowen@gmail.com. Learn why this is
> important <https://aka.ms/LearnAboutSenderIdentification>
>
> **** External email: use caution ****
>
>
>
> Probably a few months between maintenance releases.
>
> It does not appear to affect Spark, however.
>
>
>
> On Thu, Oct 27, 2022 at 9:24 AM Pastrana, Rodrigo (RIS-BCT) <
> Rodrigo.Pastrana@lexisnexisrisk.com.invalid> wrote:
>
> Hello,
>
> This issue (SPARK-40801)
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSPARK-40801&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C507dc12538bf44d2646d08dab8276a76%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024776687375556%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=wZV1KpRw248DOPuWkJ2qjDNK9DwN4zFIgL6z2g0MOkw%3D&reserved=0>
> which addresses CVE-2022-42889 doesn’t seem to have been included in the
> latest release (3.3.1
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspark.apache.org%2Freleases%2Fspark-release-3-3-1.html&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C507dc12538bf44d2646d08dab8276a76%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024776687375556%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aJXVwPl36j83CFFM%2F1jKDhSIm7mCNwRozMpXCt8dvDQ%3D&reserved=0>
> ).
>
> Is there a way to estimate a timeline for the first release which includes
> that change (likely 3.3.2)? Much appreciation!
>
>
> ------------------------------
>
> The information contained in this e-mail message is intended only for the
> personal and confidential use of the recipient(s) named above. This message
> may be an attorney-client communication and/or work product and as such is
> privileged and confidential. If the reader of this message is not the
> intended recipient or an agent responsible for delivering it to the
> intended recipient, you are hereby notified that you have received this
> document in error and that any review, dissemination, distribution, or
> copying of this message is strictly prohibited. If you have received this
> communication in error, please notify us immediately by e-mail, and delete
> the original message.
>
>
> ------------------------------
> The information contained in this e-mail message is intended only for the
> personal and confidential use of the recipient(s) named above. This message
> may be an attorney-client communication and/or work product and as such is
> privileged and confidential. If the reader of this message is not the
> intended recipient or an agent responsible for delivering it to the
> intended recipient, you are hereby notified that you have received this
> document in error and that any review, dissemination, distribution, or
> copying of this message is strictly prohibited. If you have received this
> communication in error, please notify us immediately by e-mail, and delete
> the original message.
>
RE: CVE-2022-42889
Posted by "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexisrisk.com.INVALID>.
Thanks Sean,
I assume Spark's not affected because it either doesn't reference the affected API(s) or because it does not unsafely utilize user input through the vulnerable API(s), but is there an official statement about this from Spark?
We weren't able to find references to 2022-42889 here: https://spark.apache.org/security.html (likely because Spark determined it is not affected?)
From: Sean Owen <sr...@gmail.com>
Sent: Thursday, October 27, 2022 10:27 AM
To: Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com.invalid>
Cc: dev@spark.apache.org
Subject: Re: CVE-2022-42889
You don't often get email from srowen@gmail.com<ma...@gmail.com>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
*** External email: use caution ***
Probably a few months between maintenance releases.
It does not appear to affect Spark, however.
On Thu, Oct 27, 2022 at 9:24 AM Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com.invalid>> wrote:
Hello,
This issue (SPARK-40801)<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSPARK-40801&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C507dc12538bf44d2646d08dab8276a76%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024776687375556%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=wZV1KpRw248DOPuWkJ2qjDNK9DwN4zFIgL6z2g0MOkw%3D&reserved=0> which addresses CVE-2022-42889 doesn't seem to have been included in the latest release (3.3.1<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspark.apache.org%2Freleases%2Fspark-release-3-3-1.html&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C507dc12538bf44d2646d08dab8276a76%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638024776687375556%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aJXVwPl36j83CFFM%2F1jKDhSIm7mCNwRozMpXCt8dvDQ%3D&reserved=0>).
Is there a way to estimate a timeline for the first release which includes that change (likely 3.3.2)? Much appreciation!
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
Re: CVE-2022-42889
Posted by Sean Owen <sr...@gmail.com>.
Probably a few months between maintenance releases.
It does not appear to affect Spark, however.
On Thu, Oct 27, 2022 at 9:24 AM Pastrana, Rodrigo (RIS-BCT)
<Ro...@lexisnexisrisk.com.invalid> wrote:
> Hello,
>
> This issue (SPARK-40801)
> <https://issues.apache.org/jira/browse/SPARK-40801> which addresses
> CVE-2022-42889 doesn’t seem to have been included in the latest release (
> 3.3.1 <https://spark.apache.org/releases/spark-release-3-3-1.html>).
>
> Is there a way to estimate a timeline for the first release which includes
> that change (likely 3.3.2)? Much appreciation!
>
> ------------------------------
> The information contained in this e-mail message is intended only for the
> personal and confidential use of the recipient(s) named above. This message
> may be an attorney-client communication and/or work product and as such is
> privileged and confidential. If the reader of this message is not the
> intended recipient or an agent responsible for delivering it to the
> intended recipient, you are hereby notified that you have received this
> document in error and that any review, dissemination, distribution, or
> copying of this message is strictly prohibited. If you have received this
> communication in error, please notify us immediately by e-mail, and delete
> the original message.
>
CVE-2022-42889
Posted by "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexisrisk.com.INVALID>.
Hello,
This issue (SPARK-40801)<https://issues.apache.org/jira/browse/SPARK-40801> which addresses CVE-2022-42889 doesn't seem to have been included in the latest release (3.3.1<https://spark.apache.org/releases/spark-release-3-3-1.html>).
Is there a way to estimate a timeline for the first release which includes that change (likely 3.3.2)? Much appreciation!
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
RE: 3.3.1 Release
Posted by "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexisrisk.com.INVALID>.
Great! Thank you!
From: Dongjoon Hyun <do...@gmail.com>
Sent: Tuesday, October 25, 2022 6:08 PM
To: Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com.invalid>
Cc: dev@spark.apache.org
Subject: Re: 3.3.1 Release
You don't often get email from dongjoon.hyun@gmail.com<ma...@gmail.com>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
*** External email: use caution ***
It's released Today, Pastrana.
https://downloads.apache.org/spark/spark-3.3.1/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fspark%2Fspark-3.3.1%2F&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C5a76cf82cb0b4665c12908dab6d57032%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638023325285004866%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=ms9DLcg6fJ%2BlOlypC%2FLKwCuxBKH9tsdEzeaiDN1Tqag%3D&reserved=0>
https://spark.apache.org/news/spark-3-3-1-released.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspark.apache.org%2Fnews%2Fspark-3-3-1-released.html&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C5a76cf82cb0b4665c12908dab6d57032%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638023325285161098%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=q41Ul%2FofxXBtTJDCm4cdNEzFU97oMd%2FaDmRRE1MptnU%3D&reserved=0>
https://spark.apache.org/releases/spark-release-3-3-1.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspark.apache.org%2Freleases%2Fspark-release-3-3-1.html&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C5a76cf82cb0b4665c12908dab6d57032%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638023325285161098%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=6yaDmHmCpARIxUSJaSypaOCfSHei%2F0Pg0TqrIqvER44%3D&reserved=0>
https://spark.apache.org/docs/3.3.1/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspark.apache.org%2Fdocs%2F3.3.1%2F&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C5a76cf82cb0b4665c12908dab6d57032%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638023325285161098%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=edMyEZuxPwN5AedRYvFmf5J7zN%2Fl0Nd26bXmzxEc3jA%3D&reserved=0>
https://pypi.org/project/pyspark/3.3.1/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpypi.org%2Fproject%2Fpyspark%2F3.3.1%2F&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C5a76cf82cb0b4665c12908dab6d57032%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638023325285161098%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=Zyejj0BJxLLss21htqm5TRqqfJX00DhUca2XWXGhb1M%3D&reserved=0>
I guess the release manager will announce it officially after finalizing by uploading to DockerHub.
https://hub.docker.com/r/apache/spark/tags<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhub.docker.com%2Fr%2Fapache%2Fspark%2Ftags&data=05%7C01%7CRodrigo.Pastrana%40lexisnexisrisk.com%7C5a76cf82cb0b4665c12908dab6d57032%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638023325285161098%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=fXHRgLDd8zqOxzEcw54G7oOqR6yX8qYqRSVL43X3LFU%3D&reserved=0>
Dongjoon.
On Tue, Oct 25, 2022 at 1:14 PM Pastrana, Rodrigo (RIS-BCT) <Ro...@lexisnexisrisk.com.invalid>> wrote:
Thanks to all involved with the 3.3.1 release. Is there a target date for the official release? Thanks!
[VOTE][RESULT] Release Spark 3.3.1 (RC4)
The vote passes with 11 +1s (6 binding +1s).
Thanks to all who helped with the release!
(* = binding)
+1:
- Sean Owen (*)
- Yang,Jie
- Dongjoon Hyun (*)
- L. C. Hsieh (*)
- Gengliang Wang (*)
- Thomas graves (*)
- Chao Sun
- Wenchen Fan (*)
- Yikun Jiang
- Cheng Pan
- Yuming Wang
+0: None
-1: None
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
Re: 3.3.1 Release
Posted by Dongjoon Hyun <do...@gmail.com>.
It's released Today, Pastrana.
https://downloads.apache.org/spark/spark-3.3.1/
https://spark.apache.org/news/spark-3-3-1-released.html
https://spark.apache.org/releases/spark-release-3-3-1.html
https://spark.apache.org/docs/3.3.1/
https://pypi.org/project/pyspark/3.3.1/
I guess the release manager will announce it officially after finalizing by
uploading to DockerHub.
https://hub.docker.com/r/apache/spark/tags
Dongjoon.
On Tue, Oct 25, 2022 at 1:14 PM Pastrana, Rodrigo (RIS-BCT)
<Ro...@lexisnexisrisk.com.invalid> wrote:
> Thanks to all involved with the 3.3.1 release. Is there a target date for
> the official release? Thanks!
>
>
>
> *[VOTE][RESULT] Release Spark 3.3.1 (RC4)*
>
> The vote passes with 11 +1s (6 binding +1s).
>
> Thanks to all who helped with the release!
>
>
>
> (* = binding)
>
> +1:
>
> - Sean Owen (*)
>
> - Yang,Jie
>
> - Dongjoon Hyun (*)
>
> - L. C. Hsieh (*)
>
> - Gengliang Wang (*)
>
> - Thomas graves (*)
>
> - Chao Sun
>
> - Wenchen Fan (*)
>
> - Yikun Jiang
>
> - Cheng Pan
>
> - Yuming Wang
>
>
>
> +0: None
>
>
>
> -1: None
>
>
>
> ------------------------------
> The information contained in this e-mail message is intended only for the
> personal and confidential use of the recipient(s) named above. This message
> may be an attorney-client communication and/or work product and as such is
> privileged and confidential. If the reader of this message is not the
> intended recipient or an agent responsible for delivering it to the
> intended recipient, you are hereby notified that you have received this
> document in error and that any review, dissemination, distribution, or
> copying of this message is strictly prohibited. If you have received this
> communication in error, please notify us immediately by e-mail, and delete
> the original message.
>
3.3.1 Release
Posted by "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexisrisk.com.INVALID>.
Thanks to all involved with the 3.3.1 release. Is there a target date for the official release? Thanks!
[VOTE][RESULT] Release Spark 3.3.1 (RC4)
The vote passes with 11 +1s (6 binding +1s).
Thanks to all who helped with the release!
(* = binding)
+1:
- Sean Owen (*)
- Yang,Jie
- Dongjoon Hyun (*)
- L. C. Hsieh (*)
- Gengliang Wang (*)
- Thomas graves (*)
- Chao Sun
- Wenchen Fan (*)
- Yikun Jiang
- Cheng Pan
- Yuming Wang
+0: None
-1: None
________________________________
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
Re: Prometheus with spark
Posted by Denny Lee <de...@gmail.com>.
Hi Raja,
A little atypical way to respond to your question - please check out the
most recent Spark AMA where we discuss this:
https://www.linkedin.com/posts/apachespark_apachespark-ama-committers-activity-6989052811397279744-jpWH?utm_source=share&utm_medium=member_ios
HTH!
Denny
On Tue, Oct 25, 2022 at 09:16 Raja bhupati <ra...@gmail.com>
wrote:
> We have use case where we would like process Prometheus metrics data with
> spark
>
> On Tue, Oct 25, 2022, 19:49 Jacek Laskowski <ja...@japila.pl> wrote:
>
>> Hi Raj,
>>
>> Do you want to do the following?
>>
>> spark.read.format("prometheus").load...
>>
>> I haven't heard of such a data source / format before.
>>
>> What would you like it for?
>>
>> Pozdrawiam,
>> Jacek Laskowski
>> ----
>> https://about.me/JacekLaskowski
>> "The Internals Of" Online Books <https://books.japila.pl/>
>> Follow me on https://twitter.com/jaceklaskowski
>>
>> <https://twitter.com/jaceklaskowski>
>>
>>
>> On Fri, Oct 21, 2022 at 6:12 PM Raj ks <ra...@gmail.com> wrote:
>>
>>> Hi Team,
>>>
>>>
>>> We wanted to query Prometheus data with spark. Any suggestions will
>>> be appreciated
>>>
>>> Searched for documents but did not got any prompt one
>>>
>>
Re: Prometheus with spark
Posted by Raja bhupati <ra...@gmail.com>.
We have use case where we would like process Prometheus metrics data with
spark
On Tue, Oct 25, 2022, 19:49 Jacek Laskowski <ja...@japila.pl> wrote:
> Hi Raj,
>
> Do you want to do the following?
>
> spark.read.format("prometheus").load...
>
> I haven't heard of such a data source / format before.
>
> What would you like it for?
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski
> "The Internals Of" Online Books <https://books.japila.pl/>
> Follow me on https://twitter.com/jaceklaskowski
>
> <https://twitter.com/jaceklaskowski>
>
>
> On Fri, Oct 21, 2022 at 6:12 PM Raj ks <ra...@gmail.com> wrote:
>
>> Hi Team,
>>
>>
>> We wanted to query Prometheus data with spark. Any suggestions will
>> be appreciated
>>
>> Searched for documents but did not got any prompt one
>>
>
Re: Prometheus with spark
Posted by Jacek Laskowski <ja...@japila.pl>.
Hi Raj,
Do you want to do the following?
spark.read.format("prometheus").load...
I haven't heard of such a data source / format before.
What would you like it for?
Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
"The Internals Of" Online Books <https://books.japila.pl/>
Follow me on https://twitter.com/jaceklaskowski
<https://twitter.com/jaceklaskowski>
On Fri, Oct 21, 2022 at 6:12 PM Raj ks <ra...@gmail.com> wrote:
> Hi Team,
>
>
> We wanted to query Prometheus data with spark. Any suggestions will
> be appreciated
>
> Searched for documents but did not got any prompt one
>