You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by "L. C. Hsieh" <vi...@gmail.com> on 2023/02/13 22:52:59 UTC
[VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Hi all,
I'd like to start the vote for SPIP: Lazy Materialization for Parquet
Read Performance Improvement.
The high summary of the SPIP is that it proposes an improvement to the
Parquet reader with lazy materialization which only materializes (i.e.
decompress, de-code, etc...) necessary values. For Spark-SQL filter
operations, evaluating the filters first and lazily materializing only
the used values can save computation wastes and improve the read
performance.
References:
JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
SPIP doc https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
Discussion thread
https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
Please vote on the SPIP for the next 72 hours:
[ ] +1: Accept the proposal as an official SPIP
[ ] +0
[ ] -1: I don’t think this is a good idea because …
Thank you!
Liang-Chi Hsieh
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by "L. C. Hsieh" <vi...@gmail.com>.
Based on SPIP doc
(https://spark.apache.org/improvement-proposals.html), the vote passes
if at least 3 +1 votes from PMC members and no -1 votes from PMC
members.
Also, the vote should be open for at least 72 hours.
On Thu, Feb 16, 2023 at 10:34 AM Mich Talebzadeh
<mi...@gmail.com> wrote:
>
> How many votes are needed for the approval state?
>
> Thanks
>
>
>
> view my Linkedin profile
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
>
>
>
>
>
> On Thu, 16 Feb 2023 at 18:19, kazuyuki tanimura <kt...@apple.com> wrote:
>>
>> +1 for myself
>>
>> On Feb 14, 2023, at 10:42 AM, DB Tsai <db...@dbtsai.com> wrote:
>>
>> +1
>>
>> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
>>
>> On Feb 14, 2023, at 8:29 AM, Guo Weijie <we...@gmail.com> wrote:
>>
>> +1
>>
>> Yuming Wang <wg...@gmail.com> 于2023年2月14日周二 15:58写道：
>>>
>>> +1
>>>
>>> On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo <pr...@gmail.com> wrote:
>>>>
>>>> +1
>>>>
>>>> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh <vi...@gmail.com> wrote:
>>>>>
>>>>> +1
>>>>>
>>>>> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <mi...@gmail.com> wrote:
>>>>>>
>>>>>> +1 for me
>>>>>>
>>>>>>
>>>>>> view my Linkedin profile
>>>>>>
>>>>>>
>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>>
>>>>>>
>>>>>>
>>>>>> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, 13 Feb 2023 at 23:18, huaxin gao <hu...@gmail.com> wrote:
>>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun <do...@apache.org> wrote:
>>>>>>>>
>>>>>>>> +1
>>>>>>>>
>>>>>>>> Dongjoon
>>>>>>>>
>>>>>>>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>>>>>>>> > Hi all,
>>>>>>>> >
>>>>>>>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>>>>>>>> > Read Performance Improvement.
>>>>>>>> >
>>>>>>>> > The high summary of the SPIP is that it proposes an improvement to the
>>>>>>>> > Parquet reader with lazy materialization which only materializes (i.e.
>>>>>>>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>>>>>>>> > operations, evaluating the filters first and lazily materializing only
>>>>>>>> > the used values can save computation wastes and improve the read
>>>>>>>> > performance.
>>>>>>>> >
>>>>>>>> > References:
>>>>>>>> >
>>>>>>>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>>>>>>>> > SPIP doc https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>>>>>>>> > Discussion thread
>>>>>>>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>>>>>>>> >
>>>>>>>> > Please vote on the SPIP for the next 72 hours:
>>>>>>>> >
>>>>>>>> > [ ] +1: Accept the proposal as an official SPIP
>>>>>>>> > [ ] +0
>>>>>>>> > [ ] -1: I don’t think this is a good idea because …
>>>>>>>> >
>>>>>>>> > Thank you!
>>>>>>>> >
>>>>>>>> > Liang-Chi Hsieh
>>>>>>>> >
>>>>>>>> > ---------------------------------------------------------------------
>>>>>>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>>> >
>>>>>>>> >
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>>>
>>
>>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by Mich Talebzadeh <mi...@gmail.com>.
How many votes are needed for the approval state?
Thanks
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Thu, 16 Feb 2023 at 18:19, kazuyuki tanimura <kt...@apple.com> wrote:
> +1 for myself
>
> On Feb 14, 2023, at 10:42 AM, DB Tsai <db...@dbtsai.com> wrote:
>
> +1
>
> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
>
> On Feb 14, 2023, at 8:29 AM, Guo Weijie <we...@gmail.com> wrote:
>
> +1
>
> Yuming Wang <wg...@gmail.com> 于2023年2月14日周二 15:58写道：
>
>> +1
>>
>> On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo <pr...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh <vi...@gmail.com> wrote:
>>>
>>>> +1
>>>>
>>>> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <
>>>> mich.talebzadeh@gmail.com> wrote:
>>>>
>>>>> +1 for me
>>>>>
>>>>>
>>>>> view my Linkedin profile
>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>>> any loss, damage or destruction of data or any other property which may
>>>>> arise from relying on this email's technical content is explicitly
>>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>>> arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 13 Feb 2023 at 23:18, huaxin gao <hu...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun <do...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> Dongjoon
>>>>>>>
>>>>>>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>>>>>>> > Hi all,
>>>>>>> >
>>>>>>> > I'd like to start the vote for SPIP: Lazy Materialization for
>>>>>>> Parquet
>>>>>>> > Read Performance Improvement.
>>>>>>> >
>>>>>>> > The high summary of the SPIP is that it proposes an improvement to
>>>>>>> the
>>>>>>> > Parquet reader with lazy materialization which only materializes
>>>>>>> (i.e.
>>>>>>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>>>>>>> > operations, evaluating the filters first and lazily materializing
>>>>>>> only
>>>>>>> > the used values can save computation wastes and improve the read
>>>>>>> > performance.
>>>>>>> >
>>>>>>> > References:
>>>>>>> >
>>>>>>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>>>>>>> > SPIP doc
>>>>>>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>>>>>>> > Discussion thread
>>>>>>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>>>>>>> >
>>>>>>> > Please vote on the SPIP for the next 72 hours:
>>>>>>> >
>>>>>>> > [ ] +1: Accept the proposal as an official SPIP
>>>>>>> > [ ] +0
>>>>>>> > [ ] -1: I don’t think this is a good idea because …
>>>>>>> >
>>>>>>> > Thank you!
>>>>>>> >
>>>>>>> > Liang-Chi Hsieh
>>>>>>> >
>>>>>>> >
>>>>>>> ---------------------------------------------------------------------
>>>>>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>>
>>>>>>>
>
>
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by kazuyuki tanimura <kt...@apple.com.INVALID>.
+1 for myself
> On Feb 14, 2023, at 10:42 AM, DB Tsai <db...@dbtsai.com> wrote:
>
> +1
>
> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
>
>> On Feb 14, 2023, at 8:29 AM, Guo Weijie <we...@gmail.com> wrote:
>>
>> +1
>>
>> Yuming Wang <wgyumg@gmail.com <ma...@gmail.com>> 于2023年2月14日周二 15:58写道：
>> +1
>>
>> On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo <prem.remit@gmail.com <ma...@gmail.com>> wrote:
>> +1
>>
>> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh <viirya@gmail.com <ma...@gmail.com>> wrote:
>> +1
>>
>> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <mich.talebzadeh@gmail.com <ma...@gmail.com>> wrote:
>> +1 for me
>>
>>
>> view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh <https://en.everybodywiki.com/Mich_Talebzadeh>
>>
>> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
>>
>>
>>
>> On Mon, 13 Feb 2023 at 23:18, huaxin gao <huaxin.gao11@gmail.com <ma...@gmail.com>> wrote:
>> +1
>>
>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun <dongjoon@apache.org <ma...@apache.org>> wrote:
>> +1
>>
>> Dongjoon
>>
>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>> > Hi all,
>> >
>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>> > Read Performance Improvement.
>> >
>> > The high summary of the SPIP is that it proposes an improvement to the
>> > Parquet reader with lazy materialization which only materializes (i.e.
>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>> > operations, evaluating the filters first and lazily materializing only
>> > the used values can save computation wastes and improve the read
>> > performance.
>> >
>> > References:
>> >
>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256 <https://issues.apache.org/jira/browse/SPARK-42256>
>> > SPIP doc https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME <https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME>
>> > Discussion thread
>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6 <https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6>
>> >
>> > Please vote on the SPIP for the next 72 hours:
>> >
>> > [ ] +1: Accept the proposal as an official SPIP
>> > [ ] +0
>> > [ ] -1: I don’t think this is a good idea because …
>> >
>> > Thank you!
>> >
>> > Liang-Chi Hsieh
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org <ma...@spark.apache.org>
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org <ma...@spark.apache.org>
>>
>
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by DB Tsai <db...@dbtsai.com>.
+1
DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
> On Feb 14, 2023, at 8:29 AM, Guo Weijie <we...@gmail.com> wrote:
>
> +1
>
> Yuming Wang <wgyumg@gmail.com <ma...@gmail.com>> 于2023年2月14日周二 15:58写道：
>> +1
>>
>> On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo <prem.remit@gmail.com <ma...@gmail.com>> wrote:
>>> +1
>>>
>>> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh <viirya@gmail.com <ma...@gmail.com>> wrote:
>>>> +1
>>>>
>>>> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <mich.talebzadeh@gmail.com <ma...@gmail.com>> wrote:
>>>>> +1 for me
>>>>>
>>>>>
>>>>> view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>>
>>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>>
>>>>>
>>>>> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
>>>>>
>>>>>
>>>>>
>>>>> On Mon, 13 Feb 2023 at 23:18, huaxin gao <huaxin.gao11@gmail.com <ma...@gmail.com>> wrote:
>>>>>> +1
>>>>>>
>>>>>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun <dongjoon@apache.org <ma...@apache.org>> wrote:
>>>>>>> +1
>>>>>>>
>>>>>>> Dongjoon
>>>>>>>
>>>>>>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>>>>>>> > Hi all,
>>>>>>> >
>>>>>>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>>>>>>> > Read Performance Improvement.
>>>>>>> >
>>>>>>> > The high summary of the SPIP is that it proposes an improvement to the
>>>>>>> > Parquet reader with lazy materialization which only materializes (i.e.
>>>>>>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>>>>>>> > operations, evaluating the filters first and lazily materializing only
>>>>>>> > the used values can save computation wastes and improve the read
>>>>>>> > performance.
>>>>>>> >
>>>>>>> > References:
>>>>>>> >
>>>>>>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>>>>>>> > SPIP doc https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>>>>>>> > Discussion thread
>>>>>>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>>>>>>> >
>>>>>>> > Please vote on the SPIP for the next 72 hours:
>>>>>>> >
>>>>>>> > [ ] +1: Accept the proposal as an official SPIP
>>>>>>> > [ ] +0
>>>>>>> > [ ] -1: I don’t think this is a good idea because …
>>>>>>> >
>>>>>>> > Thank you!
>>>>>>> >
>>>>>>> > Liang-Chi Hsieh
>>>>>>> >
>>>>>>> > ---------------------------------------------------------------------
>>>>>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org <ma...@spark.apache.org>
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org <ma...@spark.apache.org>
>>>>>>>
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by Guo Weijie <we...@gmail.com>.
+1
Yuming Wang <wg...@gmail.com> 于2023年2月14日周二 15:58写道：
> +1
>
> On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo <pr...@gmail.com> wrote:
>
>> +1
>>
>> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh <vi...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>> +1 for me
>>>>
>>>>
>>>>
>>>> view my Linkedin profile
>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>>
>>>>
>>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>>
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, 13 Feb 2023 at 23:18, huaxin gao <hu...@gmail.com>
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun <do...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Dongjoon
>>>>>>
>>>>>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>>>>>> > Hi all,
>>>>>> >
>>>>>> > I'd like to start the vote for SPIP: Lazy Materialization for
>>>>>> Parquet
>>>>>> > Read Performance Improvement.
>>>>>> >
>>>>>> > The high summary of the SPIP is that it proposes an improvement to
>>>>>> the
>>>>>> > Parquet reader with lazy materialization which only materializes
>>>>>> (i.e.
>>>>>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>>>>>> > operations, evaluating the filters first and lazily materializing
>>>>>> only
>>>>>> > the used values can save computation wastes and improve the read
>>>>>> > performance.
>>>>>> >
>>>>>> > References:
>>>>>> >
>>>>>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>>>>>> > SPIP doc
>>>>>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>>>>>> > Discussion thread
>>>>>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>>>>>> >
>>>>>> > Please vote on the SPIP for the next 72 hours:
>>>>>> >
>>>>>> > [ ] +1: Accept the proposal as an official SPIP
>>>>>> > [ ] +0
>>>>>> > [ ] -1: I don’t think this is a good idea because …
>>>>>> >
>>>>>> > Thank you!
>>>>>> >
>>>>>> > Liang-Chi Hsieh
>>>>>> >
>>>>>> >
>>>>>> ---------------------------------------------------------------------
>>>>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>> >
>>>>>> >
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>
>>>>>>
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by Yuming Wang <wg...@gmail.com>.
+1
On Tue, Feb 14, 2023 at 11:27 AM Prem Sahoo <pr...@gmail.com> wrote:
> +1
>
> On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh <vi...@gmail.com> wrote:
>
>> +1
>>
>> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <
>> mich.talebzadeh@gmail.com> wrote:
>>
>>> +1 for me
>>>
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 13 Feb 2023 at 23:18, huaxin gao <hu...@gmail.com> wrote:
>>>
>>>> +1
>>>>
>>>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun <do...@apache.org>
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Dongjoon
>>>>>
>>>>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>>>>> > Hi all,
>>>>> >
>>>>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>>>>> > Read Performance Improvement.
>>>>> >
>>>>> > The high summary of the SPIP is that it proposes an improvement to
>>>>> the
>>>>> > Parquet reader with lazy materialization which only materializes
>>>>> (i.e.
>>>>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>>>>> > operations, evaluating the filters first and lazily materializing
>>>>> only
>>>>> > the used values can save computation wastes and improve the read
>>>>> > performance.
>>>>> >
>>>>> > References:
>>>>> >
>>>>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>>>>> > SPIP doc
>>>>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>>>>> > Discussion thread
>>>>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>>>>> >
>>>>> > Please vote on the SPIP for the next 72 hours:
>>>>> >
>>>>> > [ ] +1: Accept the proposal as an official SPIP
>>>>> > [ ] +0
>>>>> > [ ] -1: I don’t think this is a good idea because …
>>>>> >
>>>>> > Thank you!
>>>>> >
>>>>> > Liang-Chi Hsieh
>>>>> >
>>>>> > ---------------------------------------------------------------------
>>>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>> >
>>>>> >
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>
>>>>>
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by Prem Sahoo <pr...@gmail.com>.
+1
On Mon, Feb 13, 2023 at 8:13 PM L. C. Hsieh <vi...@gmail.com> wrote:
> +1
>
> On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
>> +1 for me
>>
>>
>>
>> view my Linkedin profile
>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>
>>
>> https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 13 Feb 2023 at 23:18, huaxin gao <hu...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun <do...@apache.org>
>>> wrote:
>>>
>>>> +1
>>>>
>>>> Dongjoon
>>>>
>>>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>>>> > Hi all,
>>>> >
>>>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>>>> > Read Performance Improvement.
>>>> >
>>>> > The high summary of the SPIP is that it proposes an improvement to the
>>>> > Parquet reader with lazy materialization which only materializes (i.e.
>>>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>>>> > operations, evaluating the filters first and lazily materializing only
>>>> > the used values can save computation wastes and improve the read
>>>> > performance.
>>>> >
>>>> > References:
>>>> >
>>>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>>>> > SPIP doc
>>>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>>>> > Discussion thread
>>>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>>>> >
>>>> > Please vote on the SPIP for the next 72 hours:
>>>> >
>>>> > [ ] +1: Accept the proposal as an official SPIP
>>>> > [ ] +0
>>>> > [ ] -1: I don’t think this is a good idea because …
>>>> >
>>>> > Thank you!
>>>> >
>>>> > Liang-Chi Hsieh
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>> >
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>
>>>>
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by "L. C. Hsieh" <vi...@gmail.com>.
+1
On Mon, Feb 13, 2023 at 3:49 PM Mich Talebzadeh <mi...@gmail.com>
wrote:
> +1 for me
>
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 13 Feb 2023 at 23:18, huaxin gao <hu...@gmail.com> wrote:
>
>> +1
>>
>> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun <do...@apache.org>
>> wrote:
>>
>>> +1
>>>
>>> Dongjoon
>>>
>>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>>> > Hi all,
>>> >
>>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>>> > Read Performance Improvement.
>>> >
>>> > The high summary of the SPIP is that it proposes an improvement to the
>>> > Parquet reader with lazy materialization which only materializes (i.e.
>>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>>> > operations, evaluating the filters first and lazily materializing only
>>> > the used values can save computation wastes and improve the read
>>> > performance.
>>> >
>>> > References:
>>> >
>>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>>> > SPIP doc
>>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>>> > Discussion thread
>>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>>> >
>>> > Please vote on the SPIP for the next 72 hours:
>>> >
>>> > [ ] +1: Accept the proposal as an official SPIP
>>> > [ ] +0
>>> > [ ] -1: I don’t think this is a good idea because …
>>> >
>>> > Thank you!
>>> >
>>> > Liang-Chi Hsieh
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>>
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by Mich Talebzadeh <mi...@gmail.com>.
+1 for me
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Mon, 13 Feb 2023 at 23:18, huaxin gao <hu...@gmail.com> wrote:
> +1
>
> On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun <do...@apache.org> wrote:
>
>> +1
>>
>> Dongjoon
>>
>> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
>> > Hi all,
>> >
>> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
>> > Read Performance Improvement.
>> >
>> > The high summary of the SPIP is that it proposes an improvement to the
>> > Parquet reader with lazy materialization which only materializes (i.e.
>> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
>> > operations, evaluating the filters first and lazily materializing only
>> > the used values can save computation wastes and improve the read
>> > performance.
>> >
>> > References:
>> >
>> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
>> > SPIP doc
>> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
>> > Discussion thread
>> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>> >
>> > Please vote on the SPIP for the next 72 hours:
>> >
>> > [ ] +1: Accept the proposal as an official SPIP
>> > [ ] +0
>> > [ ] -1: I don’t think this is a good idea because …
>> >
>> > Thank you!
>> >
>> > Liang-Chi Hsieh
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by huaxin gao <hu...@gmail.com>.
+1
On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun <do...@apache.org> wrote:
> +1
>
> Dongjoon
>
> On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
> > Hi all,
> >
> > I'd like to start the vote for SPIP: Lazy Materialization for Parquet
> > Read Performance Improvement.
> >
> > The high summary of the SPIP is that it proposes an improvement to the
> > Parquet reader with lazy materialization which only materializes (i.e.
> > decompress, de-code, etc...) necessary values. For Spark-SQL filter
> > operations, evaluating the filters first and lazily materializing only
> > the used values can save computation wastes and improve the read
> > performance.
> >
> > References:
> >
> > JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
> > SPIP doc
> https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
> > Discussion thread
> > https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
> >
> > Please vote on the SPIP for the next 72 hours:
> >
> > [ ] +1: Accept the proposal as an official SPIP
> > [ ] +0
> > [ ] -1: I don’t think this is a good idea because …
> >
> > Thank you!
> >
> > Liang-Chi Hsieh
> >
> > ---------------------------------------------------------------------
> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement
Posted by Dongjoon Hyun <do...@apache.org>.
+1
Dongjoon
On 2023/02/13 22:52:59 "L. C. Hsieh" wrote:
> Hi all,
>
> I'd like to start the vote for SPIP: Lazy Materialization for Parquet
> Read Performance Improvement.
>
> The high summary of the SPIP is that it proposes an improvement to the
> Parquet reader with lazy materialization which only materializes (i.e.
> decompress, de-code, etc...) necessary values. For Spark-SQL filter
> operations, evaluating the filters first and lazily materializing only
> the used values can save computation wastes and improve the read
> performance.
>
> References:
>
> JIRA ticket https://issues.apache.org/jira/browse/SPARK-42256
> SPIP doc https://docs.google.com/document/d/1Kr3y2fVZUbQXGH0y8AvdCAeWC49QJjpczapiaDvFzME
> Discussion thread
> https://lists.apache.org/thread/5yf2ylqhcv94y03m7gp3mgf3q0fp6gw6
>
> Please vote on the SPIP for the next 72 hours:
>
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
>
> Thank you!
>
> Liang-Chi Hsieh
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org