You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Mass Dosage <ma...@gmail.com> on 2020/02/11 11:32:11 UTC
Re: HIVE-21508 and Hive 2.3.7 question
+1.
At Expedia Group we are big users of Hive and are also experiencing issues
with not being able to use Hive 2.3.x on Java >8 which is starting to
seriously impact some of our applications which require Java 11. We worked
on HIVE-21508 in order to get it merged into the various branches and have
been asking for a Hive 2.3.7 release for months with no replies to our
questions on this mailing list.
Could someone from the Hive community please answer and let us know if
there is the possibility of a Hive 2.3.7 release? I've seen at least two
other requests for this on the list over the past few months.
If not we will be forced to fork the current 2.3 branch and release our own
version of Hive 2.3.7 to Maven Central (with a different group id) so that
we can use it (it sounds like this would be useful to others out there
too). We'd really rather not do this but I don't see any other solutions.
Thanks,
Adrian
--
Adrian Woodhead
Principal Engineer
Expedia Group - 407 St John Street, London, EC1V 4EX
On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gu...@gmail.com> wrote:
> Hi Hive dev team,
>
> As informed earlier, I, Yuming and many people from spark dev have made
> huge efforts
> to let Spark use official Hive release. Thanks Alan and all Hive dev for
> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
>
> Few months ago, an unexpected problem was found. Spark throws
> ClassCastException when
> initializing HiveMetaStoreClient.
> Please see SPARK-29245 <https://issues.apache.org/jira/browse/SPARK-29245>
> for
> more details. This has fixed by HIVE-21508
> <https://issues.apache.org/jira/browse/HIVE-21508>.
> We postponed the Hive release request to Spark code freeze schedule to
> avoid multiple requests.
>
> Spark is going to freeze code 31st January (tomorrow), and I currently
> foresee the RC starts around March. So, this will be hopefully the last
> request for Hive release for Spark 3.0.
>
> I was wondering if we could release Hive 2.3.7 soon so Spark can uses it.
>
> Thanks.
>
Re: HIVE-21508 and Hive 2.3.7 question
Posted by Yuming Wang <wg...@gmail.com>.
Awesome, thanks for the update!
On Tue, Mar 31, 2020 at 5:20 PM Mass Dosage <ma...@gmail.com> wrote:
> Hey all,
>
> We've made some progress on this and are getting closer to a 2.3.7
> release. Alan has identified 2 tests failing on the 2.3 branch that are
> fixed in newer versions of Hive so he is proposing to backport the fixes
> for them. The ticket for that is
> https://issues.apache.org/jira/browse/HIVE-23086 if you want to watch it
> and vote it up. Hopefully we can get that merged soon and then we'll be
> good to go.
>
> Thanks,
>
> Adrian
>
> On Sun, 8 Mar 2020 at 02:41, Hyukjin Kwon <gu...@gmail.com> wrote:
>
>> Thank you so much, Alan and all.
>>
>> 2020년 3월 8일 (일) 오전 10:36, Yuming Wang <wg...@gmail.com>님이 작성:
>>
>>> Great, thank you Alan and Adrian.
>>>
>>> On Sun, Mar 8, 2020 at 8:13 AM Alan Gates <al...@gmail.com> wrote:
>>>
>>>> I'm working with Adrian on getting a 2.3.7 release out. That will pick
>>>> up everything that is already on the 2.3 branch.
>>>>
>>>> Alan.
>>>>
>>>> On Sat, Mar 7, 2020 at 6:02 AM Yuming Wang <wg...@gmail.com> wrote:
>>>>
>>>>> Hi Alan and Owen,
>>>>>
>>>>> Is there any plans to release Hive 2.3.7 or Hive 2.4.0? It may be the
>>>>> only one that supports Java 11. Hive 3.x can not support it because of
>>>>> HIVE-22097 <https://issues.apache.org/jira/browse/HIVE-22097>.
>>>>>
>>>>> On Tue, Feb 11, 2020 at 7:32 PM Mass Dosage <ma...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> +1.
>>>>>>
>>>>>> At Expedia Group we are big users of Hive and are also experiencing
>>>>>> issues with not being able to use Hive 2.3.x on Java >8 which is starting
>>>>>> to seriously impact some of our applications which require Java 11. We
>>>>>> worked on HIVE-21508 in order to get it merged into the various branches
>>>>>> and have been asking for a Hive 2.3.7 release for months with no replies to
>>>>>> our questions on this mailing list.
>>>>>>
>>>>>> Could someone from the Hive community please answer and let us know
>>>>>> if there is the possibility of a Hive 2.3.7 release? I've seen at least two
>>>>>> other requests for this on the list over the past few months.
>>>>>>
>>>>>> If not we will be forced to fork the current 2.3 branch and release
>>>>>> our own version of Hive 2.3.7 to Maven Central (with a different group id)
>>>>>> so that we can use it (it sounds like this would be useful to others out
>>>>>> there too). We'd really rather not do this but I don't see any other
>>>>>> solutions.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Adrian
>>>>>> --
>>>>>> Adrian Woodhead
>>>>>> Principal Engineer
>>>>>> Expedia Group - 407 St John Street, London, EC1V 4EX
>>>>>>
>>>>>>
>>>>>> On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gu...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Hive dev team,
>>>>>>>
>>>>>>> As informed earlier, I, Yuming and many people from spark dev have
>>>>>>> made
>>>>>>> huge efforts
>>>>>>> to let Spark use official Hive release. Thanks Alan and all Hive dev
>>>>>>> for
>>>>>>> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
>>>>>>>
>>>>>>> Few months ago, an unexpected problem was found. Spark throws
>>>>>>> ClassCastException when
>>>>>>> initializing HiveMetaStoreClient.
>>>>>>> Please see SPARK-29245 <
>>>>>>> https://issues.apache.org/jira/browse/SPARK-29245> for
>>>>>>> more details. This has fixed by HIVE-21508
>>>>>>> <https://issues.apache.org/jira/browse/HIVE-21508>.
>>>>>>> We postponed the Hive release request to Spark code freeze schedule
>>>>>>> to
>>>>>>> avoid multiple requests.
>>>>>>>
>>>>>>> Spark is going to freeze code 31st January (tomorrow), and I
>>>>>>> currently
>>>>>>> foresee the RC starts around March. So, this will be hopefully the
>>>>>>> last
>>>>>>> request for Hive release for Spark 3.0.
>>>>>>>
>>>>>>> I was wondering if we could release Hive 2.3.7 soon so Spark can
>>>>>>> uses it.
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>
Re: HIVE-21508 and Hive 2.3.7 question
Posted by David Mollitor <da...@gmail.com>.
Cool. Thanks for the consideration.
On Tue, Apr 7, 2020 at 2:41 PM Alan Gates <al...@gmail.com> wrote:
> I agree with Adrian that HIVE-21469 is beyond the scope of 2.3.7 for now.
> If you want to backport it to the 2 branch I think it makes sense to put it
> there in anticipation of a 2.4 release.
>
> Alan.
>
> On Wed, Apr 1, 2020 at 4:40 AM Mass Dosage <ma...@gmail.com> wrote:
>
> > I think, given that we're so close to potentially cutting a 2.3.7 release
> > (see Alan's separate post to the mailing list) that we shouldn't add
> > anything else at this this stage. This could potentially be of interest
> for
> > a 2.3.8 or 2.4.0 release if the rest of the Hive community agrees.
> >
> > Thanks,
> >
> > Adrian
> >
> > On Tue, 31 Mar 2020 at 13:24, David Mollitor <da...@gmail.com> wrote:
> >
> >> Hello Team,
> >>
> >> Just to throw one more thing in there, awhile ago I put a good chunk of
> >> time into shoring up the ZK Lock Manager because I worked with a lot of
> >> folks on locking issues. HDP/CLDR moved away from ZK and is using a
> RDBMS
> >> and therefore never paid it much mind. Any interest in rolling it into
> >> Hive
> >> 2?
> >>
> >> HIVE-21469
> >>
> >> On Tue, Mar 31, 2020, 5:20 AM Mass Dosage <ma...@gmail.com> wrote:
> >>
> >> > Hey all,
> >> >
> >> > We've made some progress on this and are getting closer to a 2.3.7
> >> release.
> >> > Alan has identified 2 tests failing on the 2.3 branch that are fixed
> in
> >> > newer versions of Hive so he is proposing to backport the fixes for
> >> them.
> >> > The ticket for that is
> https://issues.apache.org/jira/browse/HIVE-23086
> >> if
> >> > you want to watch it and vote it up. Hopefully we can get that merged
> >> soon
> >> > and then we'll be good to go.
> >> >
> >> > Thanks,
> >> >
> >> > Adrian
> >> >
> >> > On Sun, 8 Mar 2020 at 02:41, Hyukjin Kwon <gu...@gmail.com>
> wrote:
> >> >
> >> > > Thank you so much, Alan and all.
> >> > >
> >> > > 2020년 3월 8일 (일) 오전 10:36, Yuming Wang <wg...@gmail.com>님이 작성:
> >> > >
> >> > >> Great, thank you Alan and Adrian.
> >> > >>
> >> > >> On Sun, Mar 8, 2020 at 8:13 AM Alan Gates <al...@gmail.com>
> >> wrote:
> >> > >>
> >> > >>> I'm working with Adrian on getting a 2.3.7 release out. That will
> >> pick
> >> > >>> up everything that is already on the 2.3 branch.
> >> > >>>
> >> > >>> Alan.
> >> > >>>
> >> > >>> On Sat, Mar 7, 2020 at 6:02 AM Yuming Wang <wg...@gmail.com>
> >> wrote:
> >> > >>>
> >> > >>>> Hi Alan and Owen,
> >> > >>>>
> >> > >>>> Is there any plans to release Hive 2.3.7 or Hive 2.4.0? It may be
> >> the
> >> > >>>> only one that supports Java 11. Hive 3.x can not support it
> >> because of
> >> > >>>> HIVE-22097 <https://issues.apache.org/jira/browse/HIVE-22097>.
> >> > >>>>
> >> > >>>> On Tue, Feb 11, 2020 at 7:32 PM Mass Dosage <
> massdosage@gmail.com>
> >> > >>>> wrote:
> >> > >>>>
> >> > >>>>> +1.
> >> > >>>>>
> >> > >>>>> At Expedia Group we are big users of Hive and are also
> >> experiencing
> >> > >>>>> issues with not being able to use Hive 2.3.x on Java >8 which is
> >> > starting
> >> > >>>>> to seriously impact some of our applications which require Java
> >> 11.
> >> > We
> >> > >>>>> worked on HIVE-21508 in order to get it merged into the various
> >> > branches
> >> > >>>>> and have been asking for a Hive 2.3.7 release for months with no
> >> > replies to
> >> > >>>>> our questions on this mailing list.
> >> > >>>>>
> >> > >>>>> Could someone from the Hive community please answer and let us
> >> know
> >> > if
> >> > >>>>> there is the possibility of a Hive 2.3.7 release? I've seen at
> >> least
> >> > two
> >> > >>>>> other requests for this on the list over the past few months.
> >> > >>>>>
> >> > >>>>> If not we will be forced to fork the current 2.3 branch and
> >> release
> >> > >>>>> our own version of Hive 2.3.7 to Maven Central (with a different
> >> > group id)
> >> > >>>>> so that we can use it (it sounds like this would be useful to
> >> others
> >> > out
> >> > >>>>> there too). We'd really rather not do this but I don't see any
> >> other
> >> > >>>>> solutions.
> >> > >>>>>
> >> > >>>>> Thanks,
> >> > >>>>>
> >> > >>>>> Adrian
> >> > >>>>> --
> >> > >>>>> Adrian Woodhead
> >> > >>>>> Principal Engineer
> >> > >>>>> Expedia Group - 407 St John Street, London, EC1V 4EX
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gurwls223@gmail.com
> >
> >> > >>>>> wrote:
> >> > >>>>>
> >> > >>>>>> Hi Hive dev team,
> >> > >>>>>>
> >> > >>>>>> As informed earlier, I, Yuming and many people from spark dev
> >> have
> >> > >>>>>> made
> >> > >>>>>> huge efforts
> >> > >>>>>> to let Spark use official Hive release. Thanks Alan and all
> Hive
> >> dev
> >> > >>>>>> for
> >> > >>>>>> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
> >> > >>>>>>
> >> > >>>>>> Few months ago, an unexpected problem was found. Spark throws
> >> > >>>>>> ClassCastException when
> >> > >>>>>> initializing HiveMetaStoreClient.
> >> > >>>>>> Please see SPARK-29245 <
> >> > >>>>>> https://issues.apache.org/jira/browse/SPARK-29245> for
> >> > >>>>>> more details. This has fixed by HIVE-21508
> >> > >>>>>> <https://issues.apache.org/jira/browse/HIVE-21508>.
> >> > >>>>>> We postponed the Hive release request to Spark code freeze
> >> schedule
> >> > to
> >> > >>>>>> avoid multiple requests.
> >> > >>>>>>
> >> > >>>>>> Spark is going to freeze code 31st January (tomorrow), and I
> >> > currently
> >> > >>>>>> foresee the RC starts around March. So, this will be hopefully
> >> the
> >> > >>>>>> last
> >> > >>>>>> request for Hive release for Spark 3.0.
> >> > >>>>>>
> >> > >>>>>> I was wondering if we could release Hive 2.3.7 soon so Spark
> can
> >> > uses
> >> > >>>>>> it.
> >> > >>>>>>
> >> > >>>>>> Thanks.
> >> > >>>>>>
> >> > >>>>>
> >> >
> >>
> >
>
Re: HIVE-21508 and Hive 2.3.7 question
Posted by Alan Gates <al...@gmail.com>.
I agree with Adrian that HIVE-21469 is beyond the scope of 2.3.7 for now.
If you want to backport it to the 2 branch I think it makes sense to put it
there in anticipation of a 2.4 release.
Alan.
On Wed, Apr 1, 2020 at 4:40 AM Mass Dosage <ma...@gmail.com> wrote:
> I think, given that we're so close to potentially cutting a 2.3.7 release
> (see Alan's separate post to the mailing list) that we shouldn't add
> anything else at this this stage. This could potentially be of interest for
> a 2.3.8 or 2.4.0 release if the rest of the Hive community agrees.
>
> Thanks,
>
> Adrian
>
> On Tue, 31 Mar 2020 at 13:24, David Mollitor <da...@gmail.com> wrote:
>
>> Hello Team,
>>
>> Just to throw one more thing in there, awhile ago I put a good chunk of
>> time into shoring up the ZK Lock Manager because I worked with a lot of
>> folks on locking issues. HDP/CLDR moved away from ZK and is using a RDBMS
>> and therefore never paid it much mind. Any interest in rolling it into
>> Hive
>> 2?
>>
>> HIVE-21469
>>
>> On Tue, Mar 31, 2020, 5:20 AM Mass Dosage <ma...@gmail.com> wrote:
>>
>> > Hey all,
>> >
>> > We've made some progress on this and are getting closer to a 2.3.7
>> release.
>> > Alan has identified 2 tests failing on the 2.3 branch that are fixed in
>> > newer versions of Hive so he is proposing to backport the fixes for
>> them.
>> > The ticket for that is https://issues.apache.org/jira/browse/HIVE-23086
>> if
>> > you want to watch it and vote it up. Hopefully we can get that merged
>> soon
>> > and then we'll be good to go.
>> >
>> > Thanks,
>> >
>> > Adrian
>> >
>> > On Sun, 8 Mar 2020 at 02:41, Hyukjin Kwon <gu...@gmail.com> wrote:
>> >
>> > > Thank you so much, Alan and all.
>> > >
>> > > 2020년 3월 8일 (일) 오전 10:36, Yuming Wang <wg...@gmail.com>님이 작성:
>> > >
>> > >> Great, thank you Alan and Adrian.
>> > >>
>> > >> On Sun, Mar 8, 2020 at 8:13 AM Alan Gates <al...@gmail.com>
>> wrote:
>> > >>
>> > >>> I'm working with Adrian on getting a 2.3.7 release out. That will
>> pick
>> > >>> up everything that is already on the 2.3 branch.
>> > >>>
>> > >>> Alan.
>> > >>>
>> > >>> On Sat, Mar 7, 2020 at 6:02 AM Yuming Wang <wg...@gmail.com>
>> wrote:
>> > >>>
>> > >>>> Hi Alan and Owen,
>> > >>>>
>> > >>>> Is there any plans to release Hive 2.3.7 or Hive 2.4.0? It may be
>> the
>> > >>>> only one that supports Java 11. Hive 3.x can not support it
>> because of
>> > >>>> HIVE-22097 <https://issues.apache.org/jira/browse/HIVE-22097>.
>> > >>>>
>> > >>>> On Tue, Feb 11, 2020 at 7:32 PM Mass Dosage <ma...@gmail.com>
>> > >>>> wrote:
>> > >>>>
>> > >>>>> +1.
>> > >>>>>
>> > >>>>> At Expedia Group we are big users of Hive and are also
>> experiencing
>> > >>>>> issues with not being able to use Hive 2.3.x on Java >8 which is
>> > starting
>> > >>>>> to seriously impact some of our applications which require Java
>> 11.
>> > We
>> > >>>>> worked on HIVE-21508 in order to get it merged into the various
>> > branches
>> > >>>>> and have been asking for a Hive 2.3.7 release for months with no
>> > replies to
>> > >>>>> our questions on this mailing list.
>> > >>>>>
>> > >>>>> Could someone from the Hive community please answer and let us
>> know
>> > if
>> > >>>>> there is the possibility of a Hive 2.3.7 release? I've seen at
>> least
>> > two
>> > >>>>> other requests for this on the list over the past few months.
>> > >>>>>
>> > >>>>> If not we will be forced to fork the current 2.3 branch and
>> release
>> > >>>>> our own version of Hive 2.3.7 to Maven Central (with a different
>> > group id)
>> > >>>>> so that we can use it (it sounds like this would be useful to
>> others
>> > out
>> > >>>>> there too). We'd really rather not do this but I don't see any
>> other
>> > >>>>> solutions.
>> > >>>>>
>> > >>>>> Thanks,
>> > >>>>>
>> > >>>>> Adrian
>> > >>>>> --
>> > >>>>> Adrian Woodhead
>> > >>>>> Principal Engineer
>> > >>>>> Expedia Group - 407 St John Street, London, EC1V 4EX
>> > >>>>>
>> > >>>>>
>> > >>>>> On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gu...@gmail.com>
>> > >>>>> wrote:
>> > >>>>>
>> > >>>>>> Hi Hive dev team,
>> > >>>>>>
>> > >>>>>> As informed earlier, I, Yuming and many people from spark dev
>> have
>> > >>>>>> made
>> > >>>>>> huge efforts
>> > >>>>>> to let Spark use official Hive release. Thanks Alan and all Hive
>> dev
>> > >>>>>> for
>> > >>>>>> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
>> > >>>>>>
>> > >>>>>> Few months ago, an unexpected problem was found. Spark throws
>> > >>>>>> ClassCastException when
>> > >>>>>> initializing HiveMetaStoreClient.
>> > >>>>>> Please see SPARK-29245 <
>> > >>>>>> https://issues.apache.org/jira/browse/SPARK-29245> for
>> > >>>>>> more details. This has fixed by HIVE-21508
>> > >>>>>> <https://issues.apache.org/jira/browse/HIVE-21508>.
>> > >>>>>> We postponed the Hive release request to Spark code freeze
>> schedule
>> > to
>> > >>>>>> avoid multiple requests.
>> > >>>>>>
>> > >>>>>> Spark is going to freeze code 31st January (tomorrow), and I
>> > currently
>> > >>>>>> foresee the RC starts around March. So, this will be hopefully
>> the
>> > >>>>>> last
>> > >>>>>> request for Hive release for Spark 3.0.
>> > >>>>>>
>> > >>>>>> I was wondering if we could release Hive 2.3.7 soon so Spark can
>> > uses
>> > >>>>>> it.
>> > >>>>>>
>> > >>>>>> Thanks.
>> > >>>>>>
>> > >>>>>
>> >
>>
>
Re: HIVE-21508 and Hive 2.3.7 question
Posted by Mass Dosage <ma...@gmail.com>.
I think, given that we're so close to potentially cutting a 2.3.7 release
(see Alan's separate post to the mailing list) that we shouldn't add
anything else at this this stage. This could potentially be of interest for
a 2.3.8 or 2.4.0 release if the rest of the Hive community agrees.
Thanks,
Adrian
On Tue, 31 Mar 2020 at 13:24, David Mollitor <da...@gmail.com> wrote:
> Hello Team,
>
> Just to throw one more thing in there, awhile ago I put a good chunk of
> time into shoring up the ZK Lock Manager because I worked with a lot of
> folks on locking issues. HDP/CLDR moved away from ZK and is using a RDBMS
> and therefore never paid it much mind. Any interest in rolling it into Hive
> 2?
>
> HIVE-21469
>
> On Tue, Mar 31, 2020, 5:20 AM Mass Dosage <ma...@gmail.com> wrote:
>
> > Hey all,
> >
> > We've made some progress on this and are getting closer to a 2.3.7
> release.
> > Alan has identified 2 tests failing on the 2.3 branch that are fixed in
> > newer versions of Hive so he is proposing to backport the fixes for them.
> > The ticket for that is https://issues.apache.org/jira/browse/HIVE-23086
> if
> > you want to watch it and vote it up. Hopefully we can get that merged
> soon
> > and then we'll be good to go.
> >
> > Thanks,
> >
> > Adrian
> >
> > On Sun, 8 Mar 2020 at 02:41, Hyukjin Kwon <gu...@gmail.com> wrote:
> >
> > > Thank you so much, Alan and all.
> > >
> > > 2020년 3월 8일 (일) 오전 10:36, Yuming Wang <wg...@gmail.com>님이 작성:
> > >
> > >> Great, thank you Alan and Adrian.
> > >>
> > >> On Sun, Mar 8, 2020 at 8:13 AM Alan Gates <al...@gmail.com>
> wrote:
> > >>
> > >>> I'm working with Adrian on getting a 2.3.7 release out. That will
> pick
> > >>> up everything that is already on the 2.3 branch.
> > >>>
> > >>> Alan.
> > >>>
> > >>> On Sat, Mar 7, 2020 at 6:02 AM Yuming Wang <wg...@gmail.com> wrote:
> > >>>
> > >>>> Hi Alan and Owen,
> > >>>>
> > >>>> Is there any plans to release Hive 2.3.7 or Hive 2.4.0? It may be
> the
> > >>>> only one that supports Java 11. Hive 3.x can not support it because
> of
> > >>>> HIVE-22097 <https://issues.apache.org/jira/browse/HIVE-22097>.
> > >>>>
> > >>>> On Tue, Feb 11, 2020 at 7:32 PM Mass Dosage <ma...@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>> +1.
> > >>>>>
> > >>>>> At Expedia Group we are big users of Hive and are also
> experiencing
> > >>>>> issues with not being able to use Hive 2.3.x on Java >8 which is
> > starting
> > >>>>> to seriously impact some of our applications which require Java 11.
> > We
> > >>>>> worked on HIVE-21508 in order to get it merged into the various
> > branches
> > >>>>> and have been asking for a Hive 2.3.7 release for months with no
> > replies to
> > >>>>> our questions on this mailing list.
> > >>>>>
> > >>>>> Could someone from the Hive community please answer and let us know
> > if
> > >>>>> there is the possibility of a Hive 2.3.7 release? I've seen at
> least
> > two
> > >>>>> other requests for this on the list over the past few months.
> > >>>>>
> > >>>>> If not we will be forced to fork the current 2.3 branch and release
> > >>>>> our own version of Hive 2.3.7 to Maven Central (with a different
> > group id)
> > >>>>> so that we can use it (it sounds like this would be useful to
> others
> > out
> > >>>>> there too). We'd really rather not do this but I don't see any
> other
> > >>>>> solutions.
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>> Adrian
> > >>>>> --
> > >>>>> Adrian Woodhead
> > >>>>> Principal Engineer
> > >>>>> Expedia Group - 407 St John Street, London, EC1V 4EX
> > >>>>>
> > >>>>>
> > >>>>> On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gu...@gmail.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi Hive dev team,
> > >>>>>>
> > >>>>>> As informed earlier, I, Yuming and many people from spark dev have
> > >>>>>> made
> > >>>>>> huge efforts
> > >>>>>> to let Spark use official Hive release. Thanks Alan and all Hive
> dev
> > >>>>>> for
> > >>>>>> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
> > >>>>>>
> > >>>>>> Few months ago, an unexpected problem was found. Spark throws
> > >>>>>> ClassCastException when
> > >>>>>> initializing HiveMetaStoreClient.
> > >>>>>> Please see SPARK-29245 <
> > >>>>>> https://issues.apache.org/jira/browse/SPARK-29245> for
> > >>>>>> more details. This has fixed by HIVE-21508
> > >>>>>> <https://issues.apache.org/jira/browse/HIVE-21508>.
> > >>>>>> We postponed the Hive release request to Spark code freeze
> schedule
> > to
> > >>>>>> avoid multiple requests.
> > >>>>>>
> > >>>>>> Spark is going to freeze code 31st January (tomorrow), and I
> > currently
> > >>>>>> foresee the RC starts around March. So, this will be hopefully the
> > >>>>>> last
> > >>>>>> request for Hive release for Spark 3.0.
> > >>>>>>
> > >>>>>> I was wondering if we could release Hive 2.3.7 soon so Spark can
> > uses
> > >>>>>> it.
> > >>>>>>
> > >>>>>> Thanks.
> > >>>>>>
> > >>>>>
> >
>
Re: HIVE-21508 and Hive 2.3.7 question
Posted by David Mollitor <da...@gmail.com>.
Hello Team,
Just to throw one more thing in there, awhile ago I put a good chunk of
time into shoring up the ZK Lock Manager because I worked with a lot of
folks on locking issues. HDP/CLDR moved away from ZK and is using a RDBMS
and therefore never paid it much mind. Any interest in rolling it into Hive
2?
HIVE-21469
On Tue, Mar 31, 2020, 5:20 AM Mass Dosage <ma...@gmail.com> wrote:
> Hey all,
>
> We've made some progress on this and are getting closer to a 2.3.7 release.
> Alan has identified 2 tests failing on the 2.3 branch that are fixed in
> newer versions of Hive so he is proposing to backport the fixes for them.
> The ticket for that is https://issues.apache.org/jira/browse/HIVE-23086 if
> you want to watch it and vote it up. Hopefully we can get that merged soon
> and then we'll be good to go.
>
> Thanks,
>
> Adrian
>
> On Sun, 8 Mar 2020 at 02:41, Hyukjin Kwon <gu...@gmail.com> wrote:
>
> > Thank you so much, Alan and all.
> >
> > 2020년 3월 8일 (일) 오전 10:36, Yuming Wang <wg...@gmail.com>님이 작성:
> >
> >> Great, thank you Alan and Adrian.
> >>
> >> On Sun, Mar 8, 2020 at 8:13 AM Alan Gates <al...@gmail.com> wrote:
> >>
> >>> I'm working with Adrian on getting a 2.3.7 release out. That will pick
> >>> up everything that is already on the 2.3 branch.
> >>>
> >>> Alan.
> >>>
> >>> On Sat, Mar 7, 2020 at 6:02 AM Yuming Wang <wg...@gmail.com> wrote:
> >>>
> >>>> Hi Alan and Owen,
> >>>>
> >>>> Is there any plans to release Hive 2.3.7 or Hive 2.4.0? It may be the
> >>>> only one that supports Java 11. Hive 3.x can not support it because of
> >>>> HIVE-22097 <https://issues.apache.org/jira/browse/HIVE-22097>.
> >>>>
> >>>> On Tue, Feb 11, 2020 at 7:32 PM Mass Dosage <ma...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> +1.
> >>>>>
> >>>>> At Expedia Group we are big users of Hive and are also experiencing
> >>>>> issues with not being able to use Hive 2.3.x on Java >8 which is
> starting
> >>>>> to seriously impact some of our applications which require Java 11.
> We
> >>>>> worked on HIVE-21508 in order to get it merged into the various
> branches
> >>>>> and have been asking for a Hive 2.3.7 release for months with no
> replies to
> >>>>> our questions on this mailing list.
> >>>>>
> >>>>> Could someone from the Hive community please answer and let us know
> if
> >>>>> there is the possibility of a Hive 2.3.7 release? I've seen at least
> two
> >>>>> other requests for this on the list over the past few months.
> >>>>>
> >>>>> If not we will be forced to fork the current 2.3 branch and release
> >>>>> our own version of Hive 2.3.7 to Maven Central (with a different
> group id)
> >>>>> so that we can use it (it sounds like this would be useful to others
> out
> >>>>> there too). We'd really rather not do this but I don't see any other
> >>>>> solutions.
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Adrian
> >>>>> --
> >>>>> Adrian Woodhead
> >>>>> Principal Engineer
> >>>>> Expedia Group - 407 St John Street, London, EC1V 4EX
> >>>>>
> >>>>>
> >>>>> On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gu...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Hive dev team,
> >>>>>>
> >>>>>> As informed earlier, I, Yuming and many people from spark dev have
> >>>>>> made
> >>>>>> huge efforts
> >>>>>> to let Spark use official Hive release. Thanks Alan and all Hive dev
> >>>>>> for
> >>>>>> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
> >>>>>>
> >>>>>> Few months ago, an unexpected problem was found. Spark throws
> >>>>>> ClassCastException when
> >>>>>> initializing HiveMetaStoreClient.
> >>>>>> Please see SPARK-29245 <
> >>>>>> https://issues.apache.org/jira/browse/SPARK-29245> for
> >>>>>> more details. This has fixed by HIVE-21508
> >>>>>> <https://issues.apache.org/jira/browse/HIVE-21508>.
> >>>>>> We postponed the Hive release request to Spark code freeze schedule
> to
> >>>>>> avoid multiple requests.
> >>>>>>
> >>>>>> Spark is going to freeze code 31st January (tomorrow), and I
> currently
> >>>>>> foresee the RC starts around March. So, this will be hopefully the
> >>>>>> last
> >>>>>> request for Hive release for Spark 3.0.
> >>>>>>
> >>>>>> I was wondering if we could release Hive 2.3.7 soon so Spark can
> uses
> >>>>>> it.
> >>>>>>
> >>>>>> Thanks.
> >>>>>>
> >>>>>
>
Re: HIVE-21508 and Hive 2.3.7 question
Posted by Mass Dosage <ma...@gmail.com>.
Hey all,
We've made some progress on this and are getting closer to a 2.3.7 release.
Alan has identified 2 tests failing on the 2.3 branch that are fixed in
newer versions of Hive so he is proposing to backport the fixes for them.
The ticket for that is https://issues.apache.org/jira/browse/HIVE-23086 if
you want to watch it and vote it up. Hopefully we can get that merged soon
and then we'll be good to go.
Thanks,
Adrian
On Sun, 8 Mar 2020 at 02:41, Hyukjin Kwon <gu...@gmail.com> wrote:
> Thank you so much, Alan and all.
>
> 2020년 3월 8일 (일) 오전 10:36, Yuming Wang <wg...@gmail.com>님이 작성:
>
>> Great, thank you Alan and Adrian.
>>
>> On Sun, Mar 8, 2020 at 8:13 AM Alan Gates <al...@gmail.com> wrote:
>>
>>> I'm working with Adrian on getting a 2.3.7 release out. That will pick
>>> up everything that is already on the 2.3 branch.
>>>
>>> Alan.
>>>
>>> On Sat, Mar 7, 2020 at 6:02 AM Yuming Wang <wg...@gmail.com> wrote:
>>>
>>>> Hi Alan and Owen,
>>>>
>>>> Is there any plans to release Hive 2.3.7 or Hive 2.4.0? It may be the
>>>> only one that supports Java 11. Hive 3.x can not support it because of
>>>> HIVE-22097 <https://issues.apache.org/jira/browse/HIVE-22097>.
>>>>
>>>> On Tue, Feb 11, 2020 at 7:32 PM Mass Dosage <ma...@gmail.com>
>>>> wrote:
>>>>
>>>>> +1.
>>>>>
>>>>> At Expedia Group we are big users of Hive and are also experiencing
>>>>> issues with not being able to use Hive 2.3.x on Java >8 which is starting
>>>>> to seriously impact some of our applications which require Java 11. We
>>>>> worked on HIVE-21508 in order to get it merged into the various branches
>>>>> and have been asking for a Hive 2.3.7 release for months with no replies to
>>>>> our questions on this mailing list.
>>>>>
>>>>> Could someone from the Hive community please answer and let us know if
>>>>> there is the possibility of a Hive 2.3.7 release? I've seen at least two
>>>>> other requests for this on the list over the past few months.
>>>>>
>>>>> If not we will be forced to fork the current 2.3 branch and release
>>>>> our own version of Hive 2.3.7 to Maven Central (with a different group id)
>>>>> so that we can use it (it sounds like this would be useful to others out
>>>>> there too). We'd really rather not do this but I don't see any other
>>>>> solutions.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Adrian
>>>>> --
>>>>> Adrian Woodhead
>>>>> Principal Engineer
>>>>> Expedia Group - 407 St John Street, London, EC1V 4EX
>>>>>
>>>>>
>>>>> On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gu...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Hive dev team,
>>>>>>
>>>>>> As informed earlier, I, Yuming and many people from spark dev have
>>>>>> made
>>>>>> huge efforts
>>>>>> to let Spark use official Hive release. Thanks Alan and all Hive dev
>>>>>> for
>>>>>> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
>>>>>>
>>>>>> Few months ago, an unexpected problem was found. Spark throws
>>>>>> ClassCastException when
>>>>>> initializing HiveMetaStoreClient.
>>>>>> Please see SPARK-29245 <
>>>>>> https://issues.apache.org/jira/browse/SPARK-29245> for
>>>>>> more details. This has fixed by HIVE-21508
>>>>>> <https://issues.apache.org/jira/browse/HIVE-21508>.
>>>>>> We postponed the Hive release request to Spark code freeze schedule to
>>>>>> avoid multiple requests.
>>>>>>
>>>>>> Spark is going to freeze code 31st January (tomorrow), and I currently
>>>>>> foresee the RC starts around March. So, this will be hopefully the
>>>>>> last
>>>>>> request for Hive release for Spark 3.0.
>>>>>>
>>>>>> I was wondering if we could release Hive 2.3.7 soon so Spark can uses
>>>>>> it.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>
Re: HIVE-21508 and Hive 2.3.7 question
Posted by Hyukjin Kwon <gu...@gmail.com>.
Thank you so much, Alan and all.
2020년 3월 8일 (일) 오전 10:36, Yuming Wang <wg...@gmail.com>님이 작성:
> Great, thank you Alan and Adrian.
>
> On Sun, Mar 8, 2020 at 8:13 AM Alan Gates <al...@gmail.com> wrote:
>
>> I'm working with Adrian on getting a 2.3.7 release out. That will pick
>> up everything that is already on the 2.3 branch.
>>
>> Alan.
>>
>> On Sat, Mar 7, 2020 at 6:02 AM Yuming Wang <wg...@gmail.com> wrote:
>>
>>> Hi Alan and Owen,
>>>
>>> Is there any plans to release Hive 2.3.7 or Hive 2.4.0? It may be the
>>> only one that supports Java 11. Hive 3.x can not support it because of
>>> HIVE-22097 <https://issues.apache.org/jira/browse/HIVE-22097>.
>>>
>>> On Tue, Feb 11, 2020 at 7:32 PM Mass Dosage <ma...@gmail.com>
>>> wrote:
>>>
>>>> +1.
>>>>
>>>> At Expedia Group we are big users of Hive and are also experiencing
>>>> issues with not being able to use Hive 2.3.x on Java >8 which is starting
>>>> to seriously impact some of our applications which require Java 11. We
>>>> worked on HIVE-21508 in order to get it merged into the various branches
>>>> and have been asking for a Hive 2.3.7 release for months with no replies to
>>>> our questions on this mailing list.
>>>>
>>>> Could someone from the Hive community please answer and let us know if
>>>> there is the possibility of a Hive 2.3.7 release? I've seen at least two
>>>> other requests for this on the list over the past few months.
>>>>
>>>> If not we will be forced to fork the current 2.3 branch and release our
>>>> own version of Hive 2.3.7 to Maven Central (with a different group id) so
>>>> that we can use it (it sounds like this would be useful to others out there
>>>> too). We'd really rather not do this but I don't see any other solutions.
>>>>
>>>> Thanks,
>>>>
>>>> Adrian
>>>> --
>>>> Adrian Woodhead
>>>> Principal Engineer
>>>> Expedia Group - 407 St John Street, London, EC1V 4EX
>>>>
>>>>
>>>> On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gu...@gmail.com> wrote:
>>>>
>>>>> Hi Hive dev team,
>>>>>
>>>>> As informed earlier, I, Yuming and many people from spark dev have made
>>>>> huge efforts
>>>>> to let Spark use official Hive release. Thanks Alan and all Hive dev
>>>>> for
>>>>> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
>>>>>
>>>>> Few months ago, an unexpected problem was found. Spark throws
>>>>> ClassCastException when
>>>>> initializing HiveMetaStoreClient.
>>>>> Please see SPARK-29245 <
>>>>> https://issues.apache.org/jira/browse/SPARK-29245> for
>>>>> more details. This has fixed by HIVE-21508
>>>>> <https://issues.apache.org/jira/browse/HIVE-21508>.
>>>>> We postponed the Hive release request to Spark code freeze schedule to
>>>>> avoid multiple requests.
>>>>>
>>>>> Spark is going to freeze code 31st January (tomorrow), and I currently
>>>>> foresee the RC starts around March. So, this will be hopefully the last
>>>>> request for Hive release for Spark 3.0.
>>>>>
>>>>> I was wondering if we could release Hive 2.3.7 soon so Spark can uses
>>>>> it.
>>>>>
>>>>> Thanks.
>>>>>
>>>>
Re: HIVE-21508 and Hive 2.3.7 question
Posted by Yuming Wang <wg...@gmail.com>.
Great, thank you Alan and Adrian.
On Sun, Mar 8, 2020 at 8:13 AM Alan Gates <al...@gmail.com> wrote:
> I'm working with Adrian on getting a 2.3.7 release out. That will pick up
> everything that is already on the 2.3 branch.
>
> Alan.
>
> On Sat, Mar 7, 2020 at 6:02 AM Yuming Wang <wg...@gmail.com> wrote:
>
>> Hi Alan and Owen,
>>
>> Is there any plans to release Hive 2.3.7 or Hive 2.4.0? It may be the
>> only one that supports Java 11. Hive 3.x can not support it because of
>> HIVE-22097 <https://issues.apache.org/jira/browse/HIVE-22097>.
>>
>> On Tue, Feb 11, 2020 at 7:32 PM Mass Dosage <ma...@gmail.com> wrote:
>>
>>> +1.
>>>
>>> At Expedia Group we are big users of Hive and are also experiencing
>>> issues with not being able to use Hive 2.3.x on Java >8 which is starting
>>> to seriously impact some of our applications which require Java 11. We
>>> worked on HIVE-21508 in order to get it merged into the various branches
>>> and have been asking for a Hive 2.3.7 release for months with no replies to
>>> our questions on this mailing list.
>>>
>>> Could someone from the Hive community please answer and let us know if
>>> there is the possibility of a Hive 2.3.7 release? I've seen at least two
>>> other requests for this on the list over the past few months.
>>>
>>> If not we will be forced to fork the current 2.3 branch and release our
>>> own version of Hive 2.3.7 to Maven Central (with a different group id) so
>>> that we can use it (it sounds like this would be useful to others out there
>>> too). We'd really rather not do this but I don't see any other solutions.
>>>
>>> Thanks,
>>>
>>> Adrian
>>> --
>>> Adrian Woodhead
>>> Principal Engineer
>>> Expedia Group - 407 St John Street, London, EC1V 4EX
>>>
>>>
>>> On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gu...@gmail.com> wrote:
>>>
>>>> Hi Hive dev team,
>>>>
>>>> As informed earlier, I, Yuming and many people from spark dev have made
>>>> huge efforts
>>>> to let Spark use official Hive release. Thanks Alan and all Hive dev for
>>>> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
>>>>
>>>> Few months ago, an unexpected problem was found. Spark throws
>>>> ClassCastException when
>>>> initializing HiveMetaStoreClient.
>>>> Please see SPARK-29245 <
>>>> https://issues.apache.org/jira/browse/SPARK-29245> for
>>>> more details. This has fixed by HIVE-21508
>>>> <https://issues.apache.org/jira/browse/HIVE-21508>.
>>>> We postponed the Hive release request to Spark code freeze schedule to
>>>> avoid multiple requests.
>>>>
>>>> Spark is going to freeze code 31st January (tomorrow), and I currently
>>>> foresee the RC starts around March. So, this will be hopefully the last
>>>> request for Hive release for Spark 3.0.
>>>>
>>>> I was wondering if we could release Hive 2.3.7 soon so Spark can uses
>>>> it.
>>>>
>>>> Thanks.
>>>>
>>>
Re: HIVE-21508 and Hive 2.3.7 question
Posted by Alan Gates <al...@gmail.com>.
I'm working with Adrian on getting a 2.3.7 release out. That will pick up
everything that is already on the 2.3 branch.
Alan.
On Sat, Mar 7, 2020 at 6:02 AM Yuming Wang <wg...@gmail.com> wrote:
> Hi Alan and Owen,
>
> Is there any plans to release Hive 2.3.7 or Hive 2.4.0? It may be the only
> one that supports Java 11. Hive 3.x can not support it because of
> HIVE-22097 <https://issues.apache.org/jira/browse/HIVE-22097>.
>
> On Tue, Feb 11, 2020 at 7:32 PM Mass Dosage <ma...@gmail.com> wrote:
>
>> +1.
>>
>> At Expedia Group we are big users of Hive and are also experiencing
>> issues with not being able to use Hive 2.3.x on Java >8 which is starting
>> to seriously impact some of our applications which require Java 11. We
>> worked on HIVE-21508 in order to get it merged into the various branches
>> and have been asking for a Hive 2.3.7 release for months with no replies to
>> our questions on this mailing list.
>>
>> Could someone from the Hive community please answer and let us know if
>> there is the possibility of a Hive 2.3.7 release? I've seen at least two
>> other requests for this on the list over the past few months.
>>
>> If not we will be forced to fork the current 2.3 branch and release our
>> own version of Hive 2.3.7 to Maven Central (with a different group id) so
>> that we can use it (it sounds like this would be useful to others out there
>> too). We'd really rather not do this but I don't see any other solutions.
>>
>> Thanks,
>>
>> Adrian
>> --
>> Adrian Woodhead
>> Principal Engineer
>> Expedia Group - 407 St John Street, London, EC1V 4EX
>>
>>
>> On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gu...@gmail.com> wrote:
>>
>>> Hi Hive dev team,
>>>
>>> As informed earlier, I, Yuming and many people from spark dev have made
>>> huge efforts
>>> to let Spark use official Hive release. Thanks Alan and all Hive dev for
>>> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
>>>
>>> Few months ago, an unexpected problem was found. Spark throws
>>> ClassCastException when
>>> initializing HiveMetaStoreClient.
>>> Please see SPARK-29245 <
>>> https://issues.apache.org/jira/browse/SPARK-29245> for
>>> more details. This has fixed by HIVE-21508
>>> <https://issues.apache.org/jira/browse/HIVE-21508>.
>>> We postponed the Hive release request to Spark code freeze schedule to
>>> avoid multiple requests.
>>>
>>> Spark is going to freeze code 31st January (tomorrow), and I currently
>>> foresee the RC starts around March. So, this will be hopefully the last
>>> request for Hive release for Spark 3.0.
>>>
>>> I was wondering if we could release Hive 2.3.7 soon so Spark can uses it.
>>>
>>> Thanks.
>>>
>>
Re: HIVE-21508 and Hive 2.3.7 question
Posted by Yuming Wang <wg...@gmail.com>.
Hi Alan and Owen,
Is there any plans to release Hive 2.3.7 or Hive 2.4.0? It may be the only
one that supports Java 11. Hive 3.x can not support it because of HIVE-22097
<https://issues.apache.org/jira/browse/HIVE-22097>.
On Tue, Feb 11, 2020 at 7:32 PM Mass Dosage <ma...@gmail.com> wrote:
> +1.
>
> At Expedia Group we are big users of Hive and are also experiencing
> issues with not being able to use Hive 2.3.x on Java >8 which is starting
> to seriously impact some of our applications which require Java 11. We
> worked on HIVE-21508 in order to get it merged into the various branches
> and have been asking for a Hive 2.3.7 release for months with no replies to
> our questions on this mailing list.
>
> Could someone from the Hive community please answer and let us know if
> there is the possibility of a Hive 2.3.7 release? I've seen at least two
> other requests for this on the list over the past few months.
>
> If not we will be forced to fork the current 2.3 branch and release our
> own version of Hive 2.3.7 to Maven Central (with a different group id) so
> that we can use it (it sounds like this would be useful to others out there
> too). We'd really rather not do this but I don't see any other solutions.
>
> Thanks,
>
> Adrian
> --
> Adrian Woodhead
> Principal Engineer
> Expedia Group - 407 St John Street, London, EC1V 4EX
>
>
> On Thu, 30 Jan 2020 at 07:34, Hyukjin Kwon <gu...@gmail.com> wrote:
>
>> Hi Hive dev team,
>>
>> As informed earlier, I, Yuming and many people from spark dev have made
>> huge efforts
>> to let Spark use official Hive release. Thanks Alan and all Hive dev for
>> all the efforts for Hive 2.3.6 to make Spark support JDK 11.
>>
>> Few months ago, an unexpected problem was found. Spark throws
>> ClassCastException when
>> initializing HiveMetaStoreClient.
>> Please see SPARK-29245 <https://issues.apache.org/jira/browse/SPARK-29245>
>> for
>> more details. This has fixed by HIVE-21508
>> <https://issues.apache.org/jira/browse/HIVE-21508>.
>> We postponed the Hive release request to Spark code freeze schedule to
>> avoid multiple requests.
>>
>> Spark is going to freeze code 31st January (tomorrow), and I currently
>> foresee the RC starts around March. So, this will be hopefully the last
>> request for Hive release for Spark 3.0.
>>
>> I was wondering if we could release Hive 2.3.7 soon so Spark can uses it.
>>
>> Thanks.
>>
>