You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Fokko Driesprong <fo...@apache.org> on 2025/02/05 16:18:47 UTC
[DISCUSS] Deprecation of parquet-pig
Hi everyone,
I would like to discuss the deprecation/removal of parquet-pig.
The last Pig release <https://pig.apache.org/releases.html> is from 2017.
And there hasn't been any meaningful contribution
<https://github.com/apache/parquet-java/commits/master/parquet-pig> to the
module in years. It has been removed
<https://lists.apache.org/thread/mw0v24nj9h7b0wyrbw7gn0ldd4m3c8kw> from
Iceberg last year because of being dormant. I think it would be good to
also remove this from Parquet when we're moving to newer versions of the
JDK.
Please share your thoughts and concerns!
Kind regards,
Fokko
Re: [DISCUSS] Deprecation of parquet-pig
Posted by Aaron Niskode-Dossett via dev <de...@parquet.apache.org>.
I missed this for the 1.17.0 release, but this PR will complete the
deprecation of parquet-pig usage from parquet-thrift.
https://github.com/apache/parquet-java/pull/3377/changes
On Thu, Feb 13, 2025 at 1:10 PM Fokko Driesprong <fo...@apache.org> wrote:
> Thanks for the responses so far, moving this to the top of your mailbox :)
>
> Thanks, Aaron, for pointing out the dependency. We could depend on the
> latest
> version that ships pig
> <
> https://github.com/apache/parquet-java/pull/3153/files#diff-a33497891f710706cd432a035219b25f83721d5b3ea8c198aafb5521f7cc5999L126
> >
> and
> go through the normal deprecation cycle in iceberg-thrift. Please share
> your thoughts on the PR.
>
> Kind regards,
> Fokko
>
>
> Op do 6 feb 2025 om 22:25 schreef Aaron Niskode-Dossett
> <an...@etsy.com.invalid>:
>
> > Sorry, I meant to say that *parquet-thrift* has a dependency on
> > *parquet-pig*
> >
> > On Thu, Feb 6, 2025 at 3:21 PM Aaron Niskode-Dossett <
> > aniskodedossett@etsy.com> wrote:
> >
> > > I'll just note that apache-thrift has a dependency on apache-pig and
> > > supports some pig functionality on its own. Removing all of that would
> be
> > > welcome.
> > >
> > > On Thu, Feb 6, 2025 at 3:09 PM Aaron Niskode-Dossett <
> > > aniskodedossett@etsy.com> wrote:
> > >
> > >> Yes, please do!
> > >>
> > >> On Wed, Feb 5, 2025 at 12:39 PM rdblue@gmail.com <rd...@gmail.com>
> > >> wrote:
> > >>
> > >>> +1
> > >>>
> > >>> People that need support can still use older versions, so I don't
> think
> > >>> that this would be a significant problem for anyone.
> > >>>
> > >>> On Wed, Feb 5, 2025 at 8:20 AM Fokko Driesprong <fo...@apache.org>
> > >>> wrote:
> > >>>
> > >>> > Hi everyone,
> > >>> >
> > >>> > I would like to discuss the deprecation/removal of parquet-pig.
> > >>> >
> > >>> > The last Pig release <https://pig.apache.org/releases.html> is
> from
> > >>> 2017.
> > >>> > And there hasn't been any meaningful contribution
> > >>> > <https://github.com/apache/parquet-java/commits/master/parquet-pig
> >
> > >>> to the
> > >>> > module in years. It has been removed
> > >>> > <https://lists.apache.org/thread/mw0v24nj9h7b0wyrbw7gn0ldd4m3c8kw>
> > >>> from
> > >>> > Iceberg last year because of being dormant. I think it would be
> good
> > to
> > >>> > also remove this from Parquet when we're moving to newer versions
> of
> > >>> the
> > >>> > JDK.
> > >>> >
> > >>> > Please share your thoughts and concerns!
> > >>> >
> > >>> > Kind regards,
> > >>> > Fokko
> > >>> >
> > >>>
> > >>
> > >>
> > >> --
> > >> Aaron Niskode-Dossett, Data Engineering -- Etsy
> > >>
> > >
> > >
> > > --
> > > Aaron Niskode-Dossett, Data Engineering -- Etsy
> > >
> >
> >
> > --
> > Aaron Niskode-Dossett, Data Engineering -- Etsy
> >
>
--
Aaron Niskode-Dossett, Data Engineering -- Etsy
Re: [DISCUSS] Deprecation of parquet-pig
Posted by Fokko Driesprong <fo...@apache.org>.
Thanks for the responses so far, moving this to the top of your mailbox :)
Thanks, Aaron, for pointing out the dependency. We could depend on the latest
version that ships pig
<https://github.com/apache/parquet-java/pull/3153/files#diff-a33497891f710706cd432a035219b25f83721d5b3ea8c198aafb5521f7cc5999L126>
and
go through the normal deprecation cycle in iceberg-thrift. Please share
your thoughts on the PR.
Kind regards,
Fokko
Op do 6 feb 2025 om 22:25 schreef Aaron Niskode-Dossett
<an...@etsy.com.invalid>:
> Sorry, I meant to say that *parquet-thrift* has a dependency on
> *parquet-pig*
>
> On Thu, Feb 6, 2025 at 3:21 PM Aaron Niskode-Dossett <
> aniskodedossett@etsy.com> wrote:
>
> > I'll just note that apache-thrift has a dependency on apache-pig and
> > supports some pig functionality on its own. Removing all of that would be
> > welcome.
> >
> > On Thu, Feb 6, 2025 at 3:09 PM Aaron Niskode-Dossett <
> > aniskodedossett@etsy.com> wrote:
> >
> >> Yes, please do!
> >>
> >> On Wed, Feb 5, 2025 at 12:39 PM rdblue@gmail.com <rd...@gmail.com>
> >> wrote:
> >>
> >>> +1
> >>>
> >>> People that need support can still use older versions, so I don't think
> >>> that this would be a significant problem for anyone.
> >>>
> >>> On Wed, Feb 5, 2025 at 8:20 AM Fokko Driesprong <fo...@apache.org>
> >>> wrote:
> >>>
> >>> > Hi everyone,
> >>> >
> >>> > I would like to discuss the deprecation/removal of parquet-pig.
> >>> >
> >>> > The last Pig release <https://pig.apache.org/releases.html> is from
> >>> 2017.
> >>> > And there hasn't been any meaningful contribution
> >>> > <https://github.com/apache/parquet-java/commits/master/parquet-pig>
> >>> to the
> >>> > module in years. It has been removed
> >>> > <https://lists.apache.org/thread/mw0v24nj9h7b0wyrbw7gn0ldd4m3c8kw>
> >>> from
> >>> > Iceberg last year because of being dormant. I think it would be good
> to
> >>> > also remove this from Parquet when we're moving to newer versions of
> >>> the
> >>> > JDK.
> >>> >
> >>> > Please share your thoughts and concerns!
> >>> >
> >>> > Kind regards,
> >>> > Fokko
> >>> >
> >>>
> >>
> >>
> >> --
> >> Aaron Niskode-Dossett, Data Engineering -- Etsy
> >>
> >
> >
> > --
> > Aaron Niskode-Dossett, Data Engineering -- Etsy
> >
>
>
> --
> Aaron Niskode-Dossett, Data Engineering -- Etsy
>
Re: [DISCUSS] Deprecation of parquet-pig
Posted by Aaron Niskode-Dossett <an...@etsy.com.INVALID>.
Sorry, I meant to say that *parquet-thrift* has a dependency on
*parquet-pig*
On Thu, Feb 6, 2025 at 3:21 PM Aaron Niskode-Dossett <
aniskodedossett@etsy.com> wrote:
> I'll just note that apache-thrift has a dependency on apache-pig and
> supports some pig functionality on its own. Removing all of that would be
> welcome.
>
> On Thu, Feb 6, 2025 at 3:09 PM Aaron Niskode-Dossett <
> aniskodedossett@etsy.com> wrote:
>
>> Yes, please do!
>>
>> On Wed, Feb 5, 2025 at 12:39 PM rdblue@gmail.com <rd...@gmail.com>
>> wrote:
>>
>>> +1
>>>
>>> People that need support can still use older versions, so I don't think
>>> that this would be a significant problem for anyone.
>>>
>>> On Wed, Feb 5, 2025 at 8:20 AM Fokko Driesprong <fo...@apache.org>
>>> wrote:
>>>
>>> > Hi everyone,
>>> >
>>> > I would like to discuss the deprecation/removal of parquet-pig.
>>> >
>>> > The last Pig release <https://pig.apache.org/releases.html> is from
>>> 2017.
>>> > And there hasn't been any meaningful contribution
>>> > <https://github.com/apache/parquet-java/commits/master/parquet-pig>
>>> to the
>>> > module in years. It has been removed
>>> > <https://lists.apache.org/thread/mw0v24nj9h7b0wyrbw7gn0ldd4m3c8kw>
>>> from
>>> > Iceberg last year because of being dormant. I think it would be good to
>>> > also remove this from Parquet when we're moving to newer versions of
>>> the
>>> > JDK.
>>> >
>>> > Please share your thoughts and concerns!
>>> >
>>> > Kind regards,
>>> > Fokko
>>> >
>>>
>>
>>
>> --
>> Aaron Niskode-Dossett, Data Engineering -- Etsy
>>
>
>
> --
> Aaron Niskode-Dossett, Data Engineering -- Etsy
>
--
Aaron Niskode-Dossett, Data Engineering -- Etsy
Re: [DISCUSS] Deprecation of parquet-pig
Posted by Aaron Niskode-Dossett <an...@etsy.com.INVALID>.
I'll just note that apache-thrift has a dependency on apache-pig and
supports some pig functionality on its own. Removing all of that would be
welcome.
On Thu, Feb 6, 2025 at 3:09 PM Aaron Niskode-Dossett <
aniskodedossett@etsy.com> wrote:
> Yes, please do!
>
> On Wed, Feb 5, 2025 at 12:39 PM rdblue@gmail.com <rd...@gmail.com> wrote:
>
>> +1
>>
>> People that need support can still use older versions, so I don't think
>> that this would be a significant problem for anyone.
>>
>> On Wed, Feb 5, 2025 at 8:20 AM Fokko Driesprong <fo...@apache.org> wrote:
>>
>> > Hi everyone,
>> >
>> > I would like to discuss the deprecation/removal of parquet-pig.
>> >
>> > The last Pig release <https://pig.apache.org/releases.html> is from
>> 2017.
>> > And there hasn't been any meaningful contribution
>> > <https://github.com/apache/parquet-java/commits/master/parquet-pig> to
>> the
>> > module in years. It has been removed
>> > <https://lists.apache.org/thread/mw0v24nj9h7b0wyrbw7gn0ldd4m3c8kw> from
>> > Iceberg last year because of being dormant. I think it would be good to
>> > also remove this from Parquet when we're moving to newer versions of the
>> > JDK.
>> >
>> > Please share your thoughts and concerns!
>> >
>> > Kind regards,
>> > Fokko
>> >
>>
>
>
> --
> Aaron Niskode-Dossett, Data Engineering -- Etsy
>
--
Aaron Niskode-Dossett, Data Engineering -- Etsy
Re: [DISCUSS] Deprecation of parquet-pig
Posted by Aaron Niskode-Dossett <an...@etsy.com.INVALID>.
Yes, please do!
On Wed, Feb 5, 2025 at 12:39 PM rdblue@gmail.com <rd...@gmail.com> wrote:
> +1
>
> People that need support can still use older versions, so I don't think
> that this would be a significant problem for anyone.
>
> On Wed, Feb 5, 2025 at 8:20 AM Fokko Driesprong <fo...@apache.org> wrote:
>
> > Hi everyone,
> >
> > I would like to discuss the deprecation/removal of parquet-pig.
> >
> > The last Pig release <https://pig.apache.org/releases.html> is from
> 2017.
> > And there hasn't been any meaningful contribution
> > <https://github.com/apache/parquet-java/commits/master/parquet-pig> to
> the
> > module in years. It has been removed
> > <https://lists.apache.org/thread/mw0v24nj9h7b0wyrbw7gn0ldd4m3c8kw> from
> > Iceberg last year because of being dormant. I think it would be good to
> > also remove this from Parquet when we're moving to newer versions of the
> > JDK.
> >
> > Please share your thoughts and concerns!
> >
> > Kind regards,
> > Fokko
> >
>
--
Aaron Niskode-Dossett, Data Engineering -- Etsy
Re: [DISCUSS] Deprecation of parquet-pig
Posted by "rdblue@gmail.com" <rd...@gmail.com>.
+1
People that need support can still use older versions, so I don't think
that this would be a significant problem for anyone.
On Wed, Feb 5, 2025 at 8:20 AM Fokko Driesprong <fo...@apache.org> wrote:
> Hi everyone,
>
> I would like to discuss the deprecation/removal of parquet-pig.
>
> The last Pig release <https://pig.apache.org/releases.html> is from 2017.
> And there hasn't been any meaningful contribution
> <https://github.com/apache/parquet-java/commits/master/parquet-pig> to the
> module in years. It has been removed
> <https://lists.apache.org/thread/mw0v24nj9h7b0wyrbw7gn0ldd4m3c8kw> from
> Iceberg last year because of being dormant. I think it would be good to
> also remove this from Parquet when we're moving to newer versions of the
> JDK.
>
> Please share your thoughts and concerns!
>
> Kind regards,
> Fokko
>