Re: [DISCUSS] KIP-826: Define platforms supported

[Ismael Juma <is...@juma.me.uk>]

Regarding 2, relying on CI alone is quite challenging from a debugging
perspective. Committers typically rely on the ability to run locally or via
a VM on a given platform. For platforms that are not particularly popular,
I am not sure if the Apache Kafka committer should incur the burden of
supporting them.

Ismael

On Thu, Mar 31, 2022 at 9:28 AM Mickael Maison <mi...@gmail.com>
wrote:

> Hi Ismael,
>
> Thanks for the feedback!
>
> 1. I agree we need to run system tests. As far as I can tell we don't
> currently run system tests for any platforms in the Apache CI, not
> even for x86_64. System tests require a lot of resources so I'm not
> sure we can run them daily on all platforms in the Apache CI. However,
> I think we should be able to run them in the Apache CI (probably only
> triggered by committers) whenever required, for example when doing a
> release, or when debugging broken tests. We've not enabled integration
> tests on aarch64 and pp64le yet so it's hard to gauge if a single
> machine is enough to handle all builds. But if we also want to run
> system tests regularly, ensuring we have at least 2 machines for each
> platforms seems like a requirement.
>
> 2. Machines are managed by the Apache Infra team. I'd expect the
> process to be similar for all platforms. If Infra can't manage some
> machines effectively and help us debug issues on them, then they
> should not be part of the CI.
>
> Thanks,
> Mickael
>
> On Thu, Mar 31, 2022 at 5:50 PM Mickael Maison <mi...@gmail.com>
> wrote:
> >
> > Hi Divij,
> >
> > Thanks for taking a look at the KIP!
> >
> > 1. I'm not sure we should include the kernel as part of the support
> > statement. Kafka does not interact with the kernel directly and
> > instead relies on the JVM to make the right system calls. So I think
> > it's best to keep that out.
> >
> > 2. That's a good point. The system tests that Ismael mentioned include
> > benchmarks, so if we run them we would get some performance metrics.
> >
> > Thanks,
> > Mickael
> >
> >
> > On Wed, Mar 30, 2022 at 3:10 PM Ismael Juma <is...@juma.me.uk> wrote:
> > >
> > > Thanks for the KIP. I would say there are 2 requirements to fully
> support
> > > platforms that the KIP does not mention:
> > >
> > > 1. Running system tests on that platform nightly.
> > > 2. Committers need a mechanism to be able to debug and fix issues
> affecting
> > > the platforms.
> > >
> > > arm64 seems like the kind of platform where we can achieve both, but
> the
> > > others are more challenging.
> > >
> > > Ismael
> > >
> > > On Wed, Mar 30, 2022 at 4:01 AM Mickael Maison <
> mickael.maison@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I recently started a thread
> > > > (https://lists.apache.org/thread/4ffpspc59cnzqbgjf3l50ykcvyp7vwtv)
> > > > about whether we should consider aarch64 (ARM64) and ppc64le
> (PowerPC)
> > > > as supported platforms because we've recently started using these
> > > > platforms in the Kafka CI.
> > > >
> > > > Unfortunately nobody replied so I guess a better way to start a
> > > > discussion is to make a proposal. I opened KIP-826 to propose clearly
> > > > defining the platforms we support in our docs and also introducing a
> > > > process for adding additional platforms in the future:
> > > >
> > > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-826%3A+Define+platforms+supported
> > > >
> > > > Let me know if you have any feedback.
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
>

Re: [DISCUSS] KIP-826: Define platforms supported

[Ismael Juma <is...@juma.me.uk>]

On Mon, Apr 4, 2022 at 3:15 PM Ismael Juma <is...@juma.me.uk> wrote:

> Regarding 2, relying on CI alone is quite challenging from a debugging
> perspective. Committers typically rely on the ability to run locally or via
> a VM on a given platform. For platforms that are not particularly popular,
> I am not sure if the Apache Kafka committer should incur the burden of
> supporting them.
>

I meant Apache Kafka community vs committer.