You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by Istvan Toth <st...@apache.org> on 2022/08/24 04:45:46 UTC
[DISCUSS] Releasing the next Omid version
Hi!
Most of the planned OMID changes for Phoenix 5.2 have landed.
The only outstanding ticket that I'm aware of is OMID-226 which I also
expect to land soon.
Unless someone has more changes targeted for the next release, I propose
that we release the next Omid version soon after OMID-226.
I also propose bumping the version to 1.1.0, though because of the HBase
1.x compatibility removal, and maven artifact changes we could also argue
for 2.0.0
If there are no other volunteers, I also volunteer to be the RM for the
release.
regards
Istvan
Re: [DISCUSS] Releasing the next Omid version
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
Everything on my list for 1.1 is done now.
Thanks to Andrew, Geoffrey, Richard and Aron for their recent commits.
Unless someone identifies further issues that we have to address for 1.1,
I think that we are ready to release.
Unless someone else volunteers to be the RM, I will start the process when
time permits.
regards
Istvan.
On Mon, Aug 29, 2022 at 12:40 PM Istvan Toth <st...@cloudera.com> wrote:
> https://issues.apache.org/jira/browse/OMID-231 is the ticket.
>
> On Mon, Aug 29, 2022 at 10:28 AM Istvan Toth <st...@cloudera.com> wrote:
>
>> I had a go at this, and it seems to be much easier than I thought.
>>
>> I'm going to open a ticket and PR for this soon.
>>
>> Istvan
>>
>> On Fri, Aug 26, 2022 at 5:15 PM Andrew Purtell <an...@gmail.com>
>> wrote:
>>
>>> I agree with Geoffrey that OMID should align with Phoenix on default
>>> dependency versions but based on this you don’t have an immediate
>>> integration problem. I don’t believe the Configuration API has changed in a
>>> long time.
>>>
>>> The security posture of Hadoop 2 in general is a problem, though. I know
>>> Hadoop released 2.10.2 to address some CVE worthy problems but it is
>>> unclear if 2.10.2 addresses all known issues, unlike 3.3.4. Also as you
>>> know Hadoop 2 has unpatchable dependencies on org.codehaus versions of
>>> Jackson and Jetty, which themselves have high scoring CVEs that will never
>>> be fixed because they are EOL, and other similar issues. Hadoop 3 doesn’t
>>> completely solve such problems but is the only realistic place we can hope
>>> they can be addressed as required. For organizations that implement or
>>> require a top to bottom security audit of their software bill of materials,
>>> it seems possible to avoid some user pain here by aligning build defaults.
>>>
>>> I also see Geoffrey started a discussion on dev@hbase about HBase
>>> producing Hadoop 3 variant builds that could be consumed by downstreamers.
>>> I have responded on that thread today to express my intention to sponsor
>>> that effort, for what it’s worth.
>>>
>>>
>>> > On Aug 25, 2022, at 10:41 PM, Istvan Toth <st...@cloudera.com.invalid>
>>> wrote:
>>> >
>>> > The only non-hbase Hadoop APIs referred in Omid are
>>> > org.apache.hadoop.conf.Configuration, and a few classes from of
>>> > org.apache.hadoop.security
>>> > (the latter is not referenced directly from the coprocessors) .
>>> >
>>> > Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded
>>> in.
>>> >
>>> >
>>> >
>>> >> On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell <
>>> andrew.purtell@gmail.com>
>>> >> wrote:
>>> >>
>>> >> Is any OMID coprocessor code using Hadoop APIs?
>>> >>
>>> >>> On Aug 25, 2022, at 9:41 AM, Istvan Toth <stoty@cloudera.com.invalid
>>> >
>>> >> wrote:
>>> >>>
>>> >>> We dependencyManage every Hadoop component in Phoenix, so going to
>>> >> Hadoop3
>>> >>> in Omid is not strictly necessary.
>>> >>> Hadoop is also added to the TSO libraries in the assembly, but AFAIK
>>> >>> Hadoop2 and Hadoop3 are wire compatible
>>> >>> (and I'm not sure if Omid even calls Hadoop endpoints directly), so
>>> that
>>> >> is
>>> >>> not a show stopper either.
>>> >>>
>>> >>> If we go with the assumption that there is no significant usage of
>>> Omid
>>> >>> outside HBase, then updating to Hadoop3 is a no-brainer, if only to
>>> avoid
>>> >>> juggling
>>> >>> the Hadoop2 and Hadoop3 compiled HBases when developing.
>>> >>> Unfortunately it's quite a bit of work with all the dependency
>>> >> management,
>>> >>> build docs, and updating the CI pipelines to rebuild HBase
>>> >>> like we do in Phoenix.
>>> >>>
>>> >>> Istvan
>>> >>>
>>> >>>
>>> >>>> On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby <gjacoby@apache.org
>>> >
>>> >> wrote:
>>> >>>>
>>> >>>> Thanks for volunteering to be RM for the next Omid release.
>>> >>>>
>>> >>>> I agree we should have a release soon (I think we'll want Phoenix
>>> 5.2 to
>>> >>>> use the new Omid). In addition to OMID-226, I suggest we also
>>> upgrade
>>> >>>> Omid's internal Hadoop version to align with whatever default
>>> Hadoop we
>>> >>>> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
>>> >>>> little) simpler. Right now Omid appears to depend on Hadoop 2.10,
>>> which
>>> >>>> since Phoenix 5 is Hadoop 3-based, seems incorrect.
>>> >>>>
>>> >>>> Geoffrey
>>> >>>>
>>> >>>>> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth <st...@apache.org>
>>> wrote:
>>> >>>>>
>>> >>>>> Hi!
>>> >>>>>
>>> >>>>> Most of the planned OMID changes for Phoenix 5.2 have landed.
>>> >>>>> The only outstanding ticket that I'm aware of is OMID-226 which I
>>> also
>>> >>>>> expect to land soon.
>>> >>>>>
>>> >>>>> Unless someone has more changes targeted for the next release, I
>>> >> propose
>>> >>>>> that we release the next Omid version soon after OMID-226.
>>> >>>>>
>>> >>>>> I also propose bumping the version to 1.1.0, though because of the
>>> >> HBase
>>> >>>>> 1.x compatibility removal, and maven artifact changes we could also
>>> >> argue
>>> >>>>> for 2.0.0
>>> >>>>>
>>> >>>>> If there are no other volunteers, I also volunteer to be the RM
>>> for the
>>> >>>>> release.
>>> >>>>>
>>> >>>>> regards
>>> >>>>> Istvan
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> *István Tóth* | Staff Software Engineer
>>> >>> stoty@cloudera.com <https://www.cloudera.com>
>>> >>> [image: Cloudera] <https://www.cloudera.com/>
>>> >>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>> >>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>> >> Cloudera
>>> >>> on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>> >>> <https://www.cloudera.com/>
>>> >>> ------------------------------
>>> >>
>>> >
>>> >
>>> > --
>>> > *István Tóth* | Staff Software Engineer
>>> > stoty@cloudera.com <https://www.cloudera.com>
>>> > [image: Cloudera] <https://www.cloudera.com/>
>>> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>>> Cloudera
>>> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>> > <https://www.cloudera.com/>
>>> > ------------------------------
>>>
>>
>>
>> --
>> *István Tóth* | Staff Software Engineer
>> stoty@cloudera.com <https://www.cloudera.com>
>> [image: Cloudera] <https://www.cloudera.com/>
>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> <https://www.cloudera.com/>
>> ------------------------------
>>
>
>
> --
> *István Tóth* | Staff Software Engineer
> stoty@cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> ------------------------------
>
--
*István Tóth* | Sr. Staff Software Engineer
*Email*: stoty@cloudera.com
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
------------------------------
Re: [DISCUSS] Releasing the next Omid version
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
https://issues.apache.org/jira/browse/OMID-231 is the ticket.
On Mon, Aug 29, 2022 at 10:28 AM Istvan Toth <st...@cloudera.com> wrote:
> I had a go at this, and it seems to be much easier than I thought.
>
> I'm going to open a ticket and PR for this soon.
>
> Istvan
>
> On Fri, Aug 26, 2022 at 5:15 PM Andrew Purtell <an...@gmail.com>
> wrote:
>
>> I agree with Geoffrey that OMID should align with Phoenix on default
>> dependency versions but based on this you don’t have an immediate
>> integration problem. I don’t believe the Configuration API has changed in a
>> long time.
>>
>> The security posture of Hadoop 2 in general is a problem, though. I know
>> Hadoop released 2.10.2 to address some CVE worthy problems but it is
>> unclear if 2.10.2 addresses all known issues, unlike 3.3.4. Also as you
>> know Hadoop 2 has unpatchable dependencies on org.codehaus versions of
>> Jackson and Jetty, which themselves have high scoring CVEs that will never
>> be fixed because they are EOL, and other similar issues. Hadoop 3 doesn’t
>> completely solve such problems but is the only realistic place we can hope
>> they can be addressed as required. For organizations that implement or
>> require a top to bottom security audit of their software bill of materials,
>> it seems possible to avoid some user pain here by aligning build defaults.
>>
>> I also see Geoffrey started a discussion on dev@hbase about HBase
>> producing Hadoop 3 variant builds that could be consumed by downstreamers.
>> I have responded on that thread today to express my intention to sponsor
>> that effort, for what it’s worth.
>>
>>
>> > On Aug 25, 2022, at 10:41 PM, Istvan Toth <st...@cloudera.com.invalid>
>> wrote:
>> >
>> > The only non-hbase Hadoop APIs referred in Omid are
>> > org.apache.hadoop.conf.Configuration, and a few classes from of
>> > org.apache.hadoop.security
>> > (the latter is not referenced directly from the coprocessors) .
>> >
>> > Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded
>> in.
>> >
>> >
>> >
>> >> On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell <
>> andrew.purtell@gmail.com>
>> >> wrote:
>> >>
>> >> Is any OMID coprocessor code using Hadoop APIs?
>> >>
>> >>> On Aug 25, 2022, at 9:41 AM, Istvan Toth <st...@cloudera.com.invalid>
>> >> wrote:
>> >>>
>> >>> We dependencyManage every Hadoop component in Phoenix, so going to
>> >> Hadoop3
>> >>> in Omid is not strictly necessary.
>> >>> Hadoop is also added to the TSO libraries in the assembly, but AFAIK
>> >>> Hadoop2 and Hadoop3 are wire compatible
>> >>> (and I'm not sure if Omid even calls Hadoop endpoints directly), so
>> that
>> >> is
>> >>> not a show stopper either.
>> >>>
>> >>> If we go with the assumption that there is no significant usage of
>> Omid
>> >>> outside HBase, then updating to Hadoop3 is a no-brainer, if only to
>> avoid
>> >>> juggling
>> >>> the Hadoop2 and Hadoop3 compiled HBases when developing.
>> >>> Unfortunately it's quite a bit of work with all the dependency
>> >> management,
>> >>> build docs, and updating the CI pipelines to rebuild HBase
>> >>> like we do in Phoenix.
>> >>>
>> >>> Istvan
>> >>>
>> >>>
>> >>>> On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby <gj...@apache.org>
>> >> wrote:
>> >>>>
>> >>>> Thanks for volunteering to be RM for the next Omid release.
>> >>>>
>> >>>> I agree we should have a release soon (I think we'll want Phoenix
>> 5.2 to
>> >>>> use the new Omid). In addition to OMID-226, I suggest we also upgrade
>> >>>> Omid's internal Hadoop version to align with whatever default Hadoop
>> we
>> >>>> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
>> >>>> little) simpler. Right now Omid appears to depend on Hadoop 2.10,
>> which
>> >>>> since Phoenix 5 is Hadoop 3-based, seems incorrect.
>> >>>>
>> >>>> Geoffrey
>> >>>>
>> >>>>> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth <st...@apache.org>
>> wrote:
>> >>>>>
>> >>>>> Hi!
>> >>>>>
>> >>>>> Most of the planned OMID changes for Phoenix 5.2 have landed.
>> >>>>> The only outstanding ticket that I'm aware of is OMID-226 which I
>> also
>> >>>>> expect to land soon.
>> >>>>>
>> >>>>> Unless someone has more changes targeted for the next release, I
>> >> propose
>> >>>>> that we release the next Omid version soon after OMID-226.
>> >>>>>
>> >>>>> I also propose bumping the version to 1.1.0, though because of the
>> >> HBase
>> >>>>> 1.x compatibility removal, and maven artifact changes we could also
>> >> argue
>> >>>>> for 2.0.0
>> >>>>>
>> >>>>> If there are no other volunteers, I also volunteer to be the RM for
>> the
>> >>>>> release.
>> >>>>>
>> >>>>> regards
>> >>>>> Istvan
>> >>>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> *István Tóth* | Staff Software Engineer
>> >>> stoty@cloudera.com <https://www.cloudera.com>
>> >>> [image: Cloudera] <https://www.cloudera.com/>
>> >>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>> >>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> >> Cloudera
>> >>> on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> >>> <https://www.cloudera.com/>
>> >>> ------------------------------
>> >>
>> >
>> >
>> > --
>> > *István Tóth* | Staff Software Engineer
>> > stoty@cloudera.com <https://www.cloudera.com>
>> > [image: Cloudera] <https://www.cloudera.com/>
>> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> Cloudera
>> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
>> > <https://www.cloudera.com/>
>> > ------------------------------
>>
>
>
> --
> *István Tóth* | Staff Software Engineer
> stoty@cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> ------------------------------
>
--
*István Tóth* | Staff Software Engineer
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
Re: [DISCUSS] Releasing the next Omid version
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
I had a go at this, and it seems to be much easier than I thought.
I'm going to open a ticket and PR for this soon.
Istvan
On Fri, Aug 26, 2022 at 5:15 PM Andrew Purtell <an...@gmail.com>
wrote:
> I agree with Geoffrey that OMID should align with Phoenix on default
> dependency versions but based on this you don’t have an immediate
> integration problem. I don’t believe the Configuration API has changed in a
> long time.
>
> The security posture of Hadoop 2 in general is a problem, though. I know
> Hadoop released 2.10.2 to address some CVE worthy problems but it is
> unclear if 2.10.2 addresses all known issues, unlike 3.3.4. Also as you
> know Hadoop 2 has unpatchable dependencies on org.codehaus versions of
> Jackson and Jetty, which themselves have high scoring CVEs that will never
> be fixed because they are EOL, and other similar issues. Hadoop 3 doesn’t
> completely solve such problems but is the only realistic place we can hope
> they can be addressed as required. For organizations that implement or
> require a top to bottom security audit of their software bill of materials,
> it seems possible to avoid some user pain here by aligning build defaults.
>
> I also see Geoffrey started a discussion on dev@hbase about HBase
> producing Hadoop 3 variant builds that could be consumed by downstreamers.
> I have responded on that thread today to express my intention to sponsor
> that effort, for what it’s worth.
>
>
> > On Aug 25, 2022, at 10:41 PM, Istvan Toth <st...@cloudera.com.invalid>
> wrote:
> >
> > The only non-hbase Hadoop APIs referred in Omid are
> > org.apache.hadoop.conf.Configuration, and a few classes from of
> > org.apache.hadoop.security
> > (the latter is not referenced directly from the coprocessors) .
> >
> > Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded in.
> >
> >
> >
> >> On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell <
> andrew.purtell@gmail.com>
> >> wrote:
> >>
> >> Is any OMID coprocessor code using Hadoop APIs?
> >>
> >>> On Aug 25, 2022, at 9:41 AM, Istvan Toth <st...@cloudera.com.invalid>
> >> wrote:
> >>>
> >>> We dependencyManage every Hadoop component in Phoenix, so going to
> >> Hadoop3
> >>> in Omid is not strictly necessary.
> >>> Hadoop is also added to the TSO libraries in the assembly, but AFAIK
> >>> Hadoop2 and Hadoop3 are wire compatible
> >>> (and I'm not sure if Omid even calls Hadoop endpoints directly), so
> that
> >> is
> >>> not a show stopper either.
> >>>
> >>> If we go with the assumption that there is no significant usage of Omid
> >>> outside HBase, then updating to Hadoop3 is a no-brainer, if only to
> avoid
> >>> juggling
> >>> the Hadoop2 and Hadoop3 compiled HBases when developing.
> >>> Unfortunately it's quite a bit of work with all the dependency
> >> management,
> >>> build docs, and updating the CI pipelines to rebuild HBase
> >>> like we do in Phoenix.
> >>>
> >>> Istvan
> >>>
> >>>
> >>>> On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby <gj...@apache.org>
> >> wrote:
> >>>>
> >>>> Thanks for volunteering to be RM for the next Omid release.
> >>>>
> >>>> I agree we should have a release soon (I think we'll want Phoenix 5.2
> to
> >>>> use the new Omid). In addition to OMID-226, I suggest we also upgrade
> >>>> Omid's internal Hadoop version to align with whatever default Hadoop
> we
> >>>> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
> >>>> little) simpler. Right now Omid appears to depend on Hadoop 2.10,
> which
> >>>> since Phoenix 5 is Hadoop 3-based, seems incorrect.
> >>>>
> >>>> Geoffrey
> >>>>
> >>>>> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth <st...@apache.org>
> wrote:
> >>>>>
> >>>>> Hi!
> >>>>>
> >>>>> Most of the planned OMID changes for Phoenix 5.2 have landed.
> >>>>> The only outstanding ticket that I'm aware of is OMID-226 which I
> also
> >>>>> expect to land soon.
> >>>>>
> >>>>> Unless someone has more changes targeted for the next release, I
> >> propose
> >>>>> that we release the next Omid version soon after OMID-226.
> >>>>>
> >>>>> I also propose bumping the version to 1.1.0, though because of the
> >> HBase
> >>>>> 1.x compatibility removal, and maven artifact changes we could also
> >> argue
> >>>>> for 2.0.0
> >>>>>
> >>>>> If there are no other volunteers, I also volunteer to be the RM for
> the
> >>>>> release.
> >>>>>
> >>>>> regards
> >>>>> Istvan
> >>>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> *István Tóth* | Staff Software Engineer
> >>> stoty@cloudera.com <https://www.cloudera.com>
> >>> [image: Cloudera] <https://www.cloudera.com/>
> >>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> >>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> >> Cloudera
> >>> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> >>> <https://www.cloudera.com/>
> >>> ------------------------------
> >>
> >
> >
> > --
> > *István Tóth* | Staff Software Engineer
> > stoty@cloudera.com <https://www.cloudera.com>
> > [image: Cloudera] <https://www.cloudera.com/>
> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera
> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > <https://www.cloudera.com/>
> > ------------------------------
>
--
*István Tóth* | Staff Software Engineer
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
Re: [DISCUSS] Releasing the next Omid version
Posted by Andrew Purtell <an...@gmail.com>.
I agree with Geoffrey that OMID should align with Phoenix on default dependency versions but based on this you don’t have an immediate integration problem. I don’t believe the Configuration API has changed in a long time.
The security posture of Hadoop 2 in general is a problem, though. I know Hadoop released 2.10.2 to address some CVE worthy problems but it is unclear if 2.10.2 addresses all known issues, unlike 3.3.4. Also as you know Hadoop 2 has unpatchable dependencies on org.codehaus versions of Jackson and Jetty, which themselves have high scoring CVEs that will never be fixed because they are EOL, and other similar issues. Hadoop 3 doesn’t completely solve such problems but is the only realistic place we can hope they can be addressed as required. For organizations that implement or require a top to bottom security audit of their software bill of materials, it seems possible to avoid some user pain here by aligning build defaults.
I also see Geoffrey started a discussion on dev@hbase about HBase producing Hadoop 3 variant builds that could be consumed by downstreamers. I have responded on that thread today to express my intention to sponsor that effort, for what it’s worth.
> On Aug 25, 2022, at 10:41 PM, Istvan Toth <st...@cloudera.com.invalid> wrote:
>
> The only non-hbase Hadoop APIs referred in Omid are
> org.apache.hadoop.conf.Configuration, and a few classes from of
> org.apache.hadoop.security
> (the latter is not referenced directly from the coprocessors) .
>
> Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded in.
>
>
>
>> On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell <an...@gmail.com>
>> wrote:
>>
>> Is any OMID coprocessor code using Hadoop APIs?
>>
>>> On Aug 25, 2022, at 9:41 AM, Istvan Toth <st...@cloudera.com.invalid>
>> wrote:
>>>
>>> We dependencyManage every Hadoop component in Phoenix, so going to
>> Hadoop3
>>> in Omid is not strictly necessary.
>>> Hadoop is also added to the TSO libraries in the assembly, but AFAIK
>>> Hadoop2 and Hadoop3 are wire compatible
>>> (and I'm not sure if Omid even calls Hadoop endpoints directly), so that
>> is
>>> not a show stopper either.
>>>
>>> If we go with the assumption that there is no significant usage of Omid
>>> outside HBase, then updating to Hadoop3 is a no-brainer, if only to avoid
>>> juggling
>>> the Hadoop2 and Hadoop3 compiled HBases when developing.
>>> Unfortunately it's quite a bit of work with all the dependency
>> management,
>>> build docs, and updating the CI pipelines to rebuild HBase
>>> like we do in Phoenix.
>>>
>>> Istvan
>>>
>>>
>>>> On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby <gj...@apache.org>
>> wrote:
>>>>
>>>> Thanks for volunteering to be RM for the next Omid release.
>>>>
>>>> I agree we should have a release soon (I think we'll want Phoenix 5.2 to
>>>> use the new Omid). In addition to OMID-226, I suggest we also upgrade
>>>> Omid's internal Hadoop version to align with whatever default Hadoop we
>>>> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
>>>> little) simpler. Right now Omid appears to depend on Hadoop 2.10, which
>>>> since Phoenix 5 is Hadoop 3-based, seems incorrect.
>>>>
>>>> Geoffrey
>>>>
>>>>> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth <st...@apache.org> wrote:
>>>>>
>>>>> Hi!
>>>>>
>>>>> Most of the planned OMID changes for Phoenix 5.2 have landed.
>>>>> The only outstanding ticket that I'm aware of is OMID-226 which I also
>>>>> expect to land soon.
>>>>>
>>>>> Unless someone has more changes targeted for the next release, I
>> propose
>>>>> that we release the next Omid version soon after OMID-226.
>>>>>
>>>>> I also propose bumping the version to 1.1.0, though because of the
>> HBase
>>>>> 1.x compatibility removal, and maven artifact changes we could also
>> argue
>>>>> for 2.0.0
>>>>>
>>>>> If there are no other volunteers, I also volunteer to be the RM for the
>>>>> release.
>>>>>
>>>>> regards
>>>>> Istvan
>>>>>
>>>>
>>>
>>>
>>> --
>>> *István Tóth* | Staff Software Engineer
>>> stoty@cloudera.com <https://www.cloudera.com>
>>> [image: Cloudera] <https://www.cloudera.com/>
>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> Cloudera
>>> on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>> <https://www.cloudera.com/>
>>> ------------------------------
>>
>
>
> --
> *István Tóth* | Staff Software Engineer
> stoty@cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> ------------------------------
Re: [DISCUSS] Releasing the next Omid version
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
The only non-hbase Hadoop APIs referred in Omid are
org.apache.hadoop.conf.Configuration, and a few classes from of
org.apache.hadoop.security
(the latter is not referenced directly from the coprocessors) .
Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded in.
On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell <an...@gmail.com>
wrote:
> Is any OMID coprocessor code using Hadoop APIs?
>
> > On Aug 25, 2022, at 9:41 AM, Istvan Toth <st...@cloudera.com.invalid>
> wrote:
> >
> > We dependencyManage every Hadoop component in Phoenix, so going to
> Hadoop3
> > in Omid is not strictly necessary.
> > Hadoop is also added to the TSO libraries in the assembly, but AFAIK
> > Hadoop2 and Hadoop3 are wire compatible
> > (and I'm not sure if Omid even calls Hadoop endpoints directly), so that
> is
> > not a show stopper either.
> >
> > If we go with the assumption that there is no significant usage of Omid
> > outside HBase, then updating to Hadoop3 is a no-brainer, if only to avoid
> > juggling
> > the Hadoop2 and Hadoop3 compiled HBases when developing.
> > Unfortunately it's quite a bit of work with all the dependency
> management,
> > build docs, and updating the CI pipelines to rebuild HBase
> > like we do in Phoenix.
> >
> > Istvan
> >
> >
> >> On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby <gj...@apache.org>
> wrote:
> >>
> >> Thanks for volunteering to be RM for the next Omid release.
> >>
> >> I agree we should have a release soon (I think we'll want Phoenix 5.2 to
> >> use the new Omid). In addition to OMID-226, I suggest we also upgrade
> >> Omid's internal Hadoop version to align with whatever default Hadoop we
> >> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
> >> little) simpler. Right now Omid appears to depend on Hadoop 2.10, which
> >> since Phoenix 5 is Hadoop 3-based, seems incorrect.
> >>
> >> Geoffrey
> >>
> >>> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth <st...@apache.org> wrote:
> >>>
> >>> Hi!
> >>>
> >>> Most of the planned OMID changes for Phoenix 5.2 have landed.
> >>> The only outstanding ticket that I'm aware of is OMID-226 which I also
> >>> expect to land soon.
> >>>
> >>> Unless someone has more changes targeted for the next release, I
> propose
> >>> that we release the next Omid version soon after OMID-226.
> >>>
> >>> I also propose bumping the version to 1.1.0, though because of the
> HBase
> >>> 1.x compatibility removal, and maven artifact changes we could also
> argue
> >>> for 2.0.0
> >>>
> >>> If there are no other volunteers, I also volunteer to be the RM for the
> >>> release.
> >>>
> >>> regards
> >>> Istvan
> >>>
> >>
> >
> >
> > --
> > *István Tóth* | Staff Software Engineer
> > stoty@cloudera.com <https://www.cloudera.com>
> > [image: Cloudera] <https://www.cloudera.com/>
> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera
> > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > <https://www.cloudera.com/>
> > ------------------------------
>
--
*István Tóth* | Staff Software Engineer
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
Re: [DISCUSS] Releasing the next Omid version
Posted by Andrew Purtell <an...@gmail.com>.
Is any OMID coprocessor code using Hadoop APIs?
> On Aug 25, 2022, at 9:41 AM, Istvan Toth <st...@cloudera.com.invalid> wrote:
>
> We dependencyManage every Hadoop component in Phoenix, so going to Hadoop3
> in Omid is not strictly necessary.
> Hadoop is also added to the TSO libraries in the assembly, but AFAIK
> Hadoop2 and Hadoop3 are wire compatible
> (and I'm not sure if Omid even calls Hadoop endpoints directly), so that is
> not a show stopper either.
>
> If we go with the assumption that there is no significant usage of Omid
> outside HBase, then updating to Hadoop3 is a no-brainer, if only to avoid
> juggling
> the Hadoop2 and Hadoop3 compiled HBases when developing.
> Unfortunately it's quite a bit of work with all the dependency management,
> build docs, and updating the CI pipelines to rebuild HBase
> like we do in Phoenix.
>
> Istvan
>
>
>> On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby <gj...@apache.org> wrote:
>>
>> Thanks for volunteering to be RM for the next Omid release.
>>
>> I agree we should have a release soon (I think we'll want Phoenix 5.2 to
>> use the new Omid). In addition to OMID-226, I suggest we also upgrade
>> Omid's internal Hadoop version to align with whatever default Hadoop we
>> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
>> little) simpler. Right now Omid appears to depend on Hadoop 2.10, which
>> since Phoenix 5 is Hadoop 3-based, seems incorrect.
>>
>> Geoffrey
>>
>>> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth <st...@apache.org> wrote:
>>>
>>> Hi!
>>>
>>> Most of the planned OMID changes for Phoenix 5.2 have landed.
>>> The only outstanding ticket that I'm aware of is OMID-226 which I also
>>> expect to land soon.
>>>
>>> Unless someone has more changes targeted for the next release, I propose
>>> that we release the next Omid version soon after OMID-226.
>>>
>>> I also propose bumping the version to 1.1.0, though because of the HBase
>>> 1.x compatibility removal, and maven artifact changes we could also argue
>>> for 2.0.0
>>>
>>> If there are no other volunteers, I also volunteer to be the RM for the
>>> release.
>>>
>>> regards
>>> Istvan
>>>
>>
>
>
> --
> *István Tóth* | Staff Software Engineer
> stoty@cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> ------------------------------
Re: [DISCUSS] Releasing the next Omid version
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
We dependencyManage every Hadoop component in Phoenix, so going to Hadoop3
in Omid is not strictly necessary.
Hadoop is also added to the TSO libraries in the assembly, but AFAIK
Hadoop2 and Hadoop3 are wire compatible
(and I'm not sure if Omid even calls Hadoop endpoints directly), so that is
not a show stopper either.
If we go with the assumption that there is no significant usage of Omid
outside HBase, then updating to Hadoop3 is a no-brainer, if only to avoid
juggling
the Hadoop2 and Hadoop3 compiled HBases when developing.
Unfortunately it's quite a bit of work with all the dependency management,
build docs, and updating the CI pipelines to rebuild HBase
like we do in Phoenix.
Istvan
On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby <gj...@apache.org> wrote:
> Thanks for volunteering to be RM for the next Omid release.
>
> I agree we should have a release soon (I think we'll want Phoenix 5.2 to
> use the new Omid). In addition to OMID-226, I suggest we also upgrade
> Omid's internal Hadoop version to align with whatever default Hadoop we
> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
> little) simpler. Right now Omid appears to depend on Hadoop 2.10, which
> since Phoenix 5 is Hadoop 3-based, seems incorrect.
>
> Geoffrey
>
> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth <st...@apache.org> wrote:
>
> > Hi!
> >
> > Most of the planned OMID changes for Phoenix 5.2 have landed.
> > The only outstanding ticket that I'm aware of is OMID-226 which I also
> > expect to land soon.
> >
> > Unless someone has more changes targeted for the next release, I propose
> > that we release the next Omid version soon after OMID-226.
> >
> > I also propose bumping the version to 1.1.0, though because of the HBase
> > 1.x compatibility removal, and maven artifact changes we could also argue
> > for 2.0.0
> >
> > If there are no other volunteers, I also volunteer to be the RM for the
> > release.
> >
> > regards
> > Istvan
> >
>
--
*István Tóth* | Staff Software Engineer
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
Re: [DISCUSS] Releasing the next Omid version
Posted by Geoffrey Jacoby <gj...@apache.org>.
Thanks for volunteering to be RM for the next Omid release.
I agree we should have a release soon (I think we'll want Phoenix 5.2 to
use the new Omid). In addition to OMID-226, I suggest we also upgrade
Omid's internal Hadoop version to align with whatever default Hadoop we
choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
little) simpler. Right now Omid appears to depend on Hadoop 2.10, which
since Phoenix 5 is Hadoop 3-based, seems incorrect.
Geoffrey
On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth <st...@apache.org> wrote:
> Hi!
>
> Most of the planned OMID changes for Phoenix 5.2 have landed.
> The only outstanding ticket that I'm aware of is OMID-226 which I also
> expect to land soon.
>
> Unless someone has more changes targeted for the next release, I propose
> that we release the next Omid version soon after OMID-226.
>
> I also propose bumping the version to 1.1.0, though because of the HBase
> 1.x compatibility removal, and maven artifact changes we could also argue
> for 2.0.0
>
> If there are no other volunteers, I also volunteer to be the RM for the
> release.
>
> regards
> Istvan
>