You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Istvan Toth <st...@apache.org> on 2025/03/17 17:48:12 UTC
[DISCUSS] How to handle assemblies for running integration tests ?
In order to get a leaner binary assembly and to reduce HBase's CVE surface
we have removed the test JARs from the normal assemblies in HBase 3 .
However, these assemblies cannot run the HBase integration tests in the
hbase-it module.
There are several options for handling this, but the easiest ones are:
*1. * Adding back a separate assembly similar to the existing 2.x assembly
*1.a * but not publishing it as part of releases
*1.b * and publishing it as part of releases
*2.* Adding a profile to the assembly modules that adds the test
dependencies
Which one would be preferable ?
Would you suggest another option ?
My personal expectation is that only developers will run the integration
tests, who will have no problems with rebuilding the assemblies with the
test profile, so I'm leaning towards option* 2.*
Another question is Chaos Monkey.
I have never used it myself, so I don't know what is required for running
it.
Has anyone tested it with branch-3 or trunk lately ?
Does it require test jars ?
When I was testing hbase-it, IntegrationTestMonkeys failed, but I'm not
sure why.
Istvan
Re: [DISCUSS] How to handle assemblies for running integration tests ?
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
Fixed in HBASE-29192
On Wed, Mar 19, 2025 at 9:07 AM 张铎(Duo Zhang) <pa...@gmail.com> wrote:
> Ah I can not get 'mvn verify' pass for hbase-it on branch-3, an
> obvious problem is HBASE-29198, but even with the fix in place, there
> are still test failures.
>
> So I do not think it is a blocker issue for adding back the testing
> assembly.
>
> Thanks.
>
> 张铎(Duo Zhang) <pa...@gmail.com> 于2025年3月18日周二 22:26写道：
>
> >
> > For me I prefer we just add a profile to hbase-assembly to generate
> > the tarball with test jars, as only developers need to this.
> >
> > For ChaosMonkey, I only used it when running ITBLL against a cluster.
> >
> > Let me try running failsafe locally too.
> >
> > Thanks.
> >
> > Istvan Toth <st...@apache.org> 于2025年3月18日周二 01:48写道：
> > >
> > > In order to get a leaner binary assembly and to reduce HBase's CVE
> surface
> > > we have removed the test JARs from the normal assemblies in HBase 3 .
> > >
> > > However, these assemblies cannot run the HBase integration tests in the
> > > hbase-it module.
> > >
> > > There are several options for handling this, but the easiest ones are:
> > >
> > > *1. * Adding back a separate assembly similar to the existing 2.x
> assembly
> > > *1.a * but not publishing it as part of releases
> > > *1.b * and publishing it as part of releases
> > > *2.* Adding a profile to the assembly modules that adds the test
> > > dependencies
> > >
> > > Which one would be preferable ?
> > > Would you suggest another option ?
> > >
> > > My personal expectation is that only developers will run the
> integration
> > > tests, who will have no problems with rebuilding the assemblies with
> the
> > > test profile, so I'm leaning towards option* 2.*
> > >
> > > Another question is Chaos Monkey.
> > > I have never used it myself, so I don't know what is required for
> running
> > > it.
> > > Has anyone tested it with branch-3 or trunk lately ?
> > > Does it require test jars ?
> > > When I was testing hbase-it, IntegrationTestMonkeys failed, but I'm not
> > > sure why.
> > >
> > > Istvan
>
--
*István Tóth* | Sr. Staff Software Engineer
*Email*: stoty@cloudera.com
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
------------------------------
Re: [DISCUSS] How to handle assemblies for running integration tests ?
Posted by Istvan Toth <st...@cloudera.com.INVALID>.
Fixed in HBASE-29192
On Wed, Mar 19, 2025 at 9:07 AM 张铎(Duo Zhang) <pa...@gmail.com> wrote:
> Ah I can not get 'mvn verify' pass for hbase-it on branch-3, an
> obvious problem is HBASE-29198, but even with the fix in place, there
> are still test failures.
>
> So I do not think it is a blocker issue for adding back the testing
> assembly.
>
> Thanks.
>
> 张铎(Duo Zhang) <pa...@gmail.com> 于2025年3月18日周二 22:26写道：
>
> >
> > For me I prefer we just add a profile to hbase-assembly to generate
> > the tarball with test jars, as only developers need to this.
> >
> > For ChaosMonkey, I only used it when running ITBLL against a cluster.
> >
> > Let me try running failsafe locally too.
> >
> > Thanks.
> >
> > Istvan Toth <st...@apache.org> 于2025年3月18日周二 01:48写道：
> > >
> > > In order to get a leaner binary assembly and to reduce HBase's CVE
> surface
> > > we have removed the test JARs from the normal assemblies in HBase 3 .
> > >
> > > However, these assemblies cannot run the HBase integration tests in the
> > > hbase-it module.
> > >
> > > There are several options for handling this, but the easiest ones are:
> > >
> > > *1. * Adding back a separate assembly similar to the existing 2.x
> assembly
> > > *1.a * but not publishing it as part of releases
> > > *1.b * and publishing it as part of releases
> > > *2.* Adding a profile to the assembly modules that adds the test
> > > dependencies
> > >
> > > Which one would be preferable ?
> > > Would you suggest another option ?
> > >
> > > My personal expectation is that only developers will run the
> integration
> > > tests, who will have no problems with rebuilding the assemblies with
> the
> > > test profile, so I'm leaning towards option* 2.*
> > >
> > > Another question is Chaos Monkey.
> > > I have never used it myself, so I don't know what is required for
> running
> > > it.
> > > Has anyone tested it with branch-3 or trunk lately ?
> > > Does it require test jars ?
> > > When I was testing hbase-it, IntegrationTestMonkeys failed, but I'm not
> > > sure why.
> > >
> > > Istvan
>
--
*István Tóth* | Sr. Staff Software Engineer
*Email*: stoty@cloudera.com
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
------------------------------
Re: [DISCUSS] How to handle assemblies for running integration tests ?
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
Ah I can not get 'mvn verify' pass for hbase-it on branch-3, an
obvious problem is HBASE-29198, but even with the fix in place, there
are still test failures.
So I do not think it is a blocker issue for adding back the testing assembly.
Thanks.
张铎(Duo Zhang) <pa...@gmail.com> 于2025年3月18日周二 22:26写道：
>
> For me I prefer we just add a profile to hbase-assembly to generate
> the tarball with test jars, as only developers need to this.
>
> For ChaosMonkey, I only used it when running ITBLL against a cluster.
>
> Let me try running failsafe locally too.
>
> Thanks.
>
> Istvan Toth <st...@apache.org> 于2025年3月18日周二 01:48写道：
> >
> > In order to get a leaner binary assembly and to reduce HBase's CVE surface
> > we have removed the test JARs from the normal assemblies in HBase 3 .
> >
> > However, these assemblies cannot run the HBase integration tests in the
> > hbase-it module.
> >
> > There are several options for handling this, but the easiest ones are:
> >
> > *1. * Adding back a separate assembly similar to the existing 2.x assembly
> > *1.a * but not publishing it as part of releases
> > *1.b * and publishing it as part of releases
> > *2.* Adding a profile to the assembly modules that adds the test
> > dependencies
> >
> > Which one would be preferable ?
> > Would you suggest another option ?
> >
> > My personal expectation is that only developers will run the integration
> > tests, who will have no problems with rebuilding the assemblies with the
> > test profile, so I'm leaning towards option* 2.*
> >
> > Another question is Chaos Monkey.
> > I have never used it myself, so I don't know what is required for running
> > it.
> > Has anyone tested it with branch-3 or trunk lately ?
> > Does it require test jars ?
> > When I was testing hbase-it, IntegrationTestMonkeys failed, but I'm not
> > sure why.
> >
> > Istvan
Re: [DISCUSS] How to handle assemblies for running integration tests ?
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
Ah I can not get 'mvn verify' pass for hbase-it on branch-3, an
obvious problem is HBASE-29198, but even with the fix in place, there
are still test failures.
So I do not think it is a blocker issue for adding back the testing assembly.
Thanks.
张铎(Duo Zhang) <pa...@gmail.com> 于2025年3月18日周二 22:26写道：
>
> For me I prefer we just add a profile to hbase-assembly to generate
> the tarball with test jars, as only developers need to this.
>
> For ChaosMonkey, I only used it when running ITBLL against a cluster.
>
> Let me try running failsafe locally too.
>
> Thanks.
>
> Istvan Toth <st...@apache.org> 于2025年3月18日周二 01:48写道：
> >
> > In order to get a leaner binary assembly and to reduce HBase's CVE surface
> > we have removed the test JARs from the normal assemblies in HBase 3 .
> >
> > However, these assemblies cannot run the HBase integration tests in the
> > hbase-it module.
> >
> > There are several options for handling this, but the easiest ones are:
> >
> > *1. * Adding back a separate assembly similar to the existing 2.x assembly
> > *1.a * but not publishing it as part of releases
> > *1.b * and publishing it as part of releases
> > *2.* Adding a profile to the assembly modules that adds the test
> > dependencies
> >
> > Which one would be preferable ?
> > Would you suggest another option ?
> >
> > My personal expectation is that only developers will run the integration
> > tests, who will have no problems with rebuilding the assemblies with the
> > test profile, so I'm leaning towards option* 2.*
> >
> > Another question is Chaos Monkey.
> > I have never used it myself, so I don't know what is required for running
> > it.
> > Has anyone tested it with branch-3 or trunk lately ?
> > Does it require test jars ?
> > When I was testing hbase-it, IntegrationTestMonkeys failed, but I'm not
> > sure why.
> >
> > Istvan
Re: [DISCUSS] How to handle assemblies for running integration tests ?
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
For me I prefer we just add a profile to hbase-assembly to generate
the tarball with test jars, as only developers need to this.
For ChaosMonkey, I only used it when running ITBLL against a cluster.
Let me try running failsafe locally too.
Thanks.
Istvan Toth <st...@apache.org> 于2025年3月18日周二 01:48写道：
>
> In order to get a leaner binary assembly and to reduce HBase's CVE surface
> we have removed the test JARs from the normal assemblies in HBase 3 .
>
> However, these assemblies cannot run the HBase integration tests in the
> hbase-it module.
>
> There are several options for handling this, but the easiest ones are:
>
> *1. * Adding back a separate assembly similar to the existing 2.x assembly
> *1.a * but not publishing it as part of releases
> *1.b * and publishing it as part of releases
> *2.* Adding a profile to the assembly modules that adds the test
> dependencies
>
> Which one would be preferable ?
> Would you suggest another option ?
>
> My personal expectation is that only developers will run the integration
> tests, who will have no problems with rebuilding the assemblies with the
> test profile, so I'm leaning towards option* 2.*
>
> Another question is Chaos Monkey.
> I have never used it myself, so I don't know what is required for running
> it.
> Has anyone tested it with branch-3 or trunk lately ?
> Does it require test jars ?
> When I was testing hbase-it, IntegrationTestMonkeys failed, but I'm not
> sure why.
>
> Istvan
Re: [DISCUSS] How to handle assemblies for running integration tests ?
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
For me I prefer we just add a profile to hbase-assembly to generate
the tarball with test jars, as only developers need to this.
For ChaosMonkey, I only used it when running ITBLL against a cluster.
Let me try running failsafe locally too.
Thanks.
Istvan Toth <st...@apache.org> 于2025年3月18日周二 01:48写道：
>
> In order to get a leaner binary assembly and to reduce HBase's CVE surface
> we have removed the test JARs from the normal assemblies in HBase 3 .
>
> However, these assemblies cannot run the HBase integration tests in the
> hbase-it module.
>
> There are several options for handling this, but the easiest ones are:
>
> *1. * Adding back a separate assembly similar to the existing 2.x assembly
> *1.a * but not publishing it as part of releases
> *1.b * and publishing it as part of releases
> *2.* Adding a profile to the assembly modules that adds the test
> dependencies
>
> Which one would be preferable ?
> Would you suggest another option ?
>
> My personal expectation is that only developers will run the integration
> tests, who will have no problems with rebuilding the assemblies with the
> test profile, so I'm leaning towards option* 2.*
>
> Another question is Chaos Monkey.
> I have never used it myself, so I don't know what is required for running
> it.
> Has anyone tested it with branch-3 or trunk lately ?
> Does it require test jars ?
> When I was testing hbase-it, IntegrationTestMonkeys failed, but I'm not
> sure why.
>
> Istvan