You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Chris McFarlen <ch...@mcfarlen.us> on 2023/04/03 17:38:18 UTC
Feedback on removing AIO_MODE_NATIVE from iocore/aio
I would like to remove this disk IO mode from ATS. If nobody is using it, then I'd like to delete it for the 10.x release. If that isn't possible, I think it should be deprecated in favor of io_uring and removed in a future version.
Is anyone using this IO mode currently? If so, are there any arguments against deprecation in favor of io_uring for linux?
Thanks,
Chris
Re: Feedback on removing AIO_MODE_NATIVE from iocore/aio
Posted by Leif Hedstrom <zw...@apache.org>.
> On May 4, 2023, at 2:50 PM, Bryan Call <bc...@apache.org> wrote:
>
> It is an experimental feature that isn’t used to my knowledge. I am ok with removing it for 10.x.
>
> If it was a supported feature then we should do the deprecation in one major version and then remove it in the next major version process.
+1, kill it with fire.
— Leif
>
> -Bryan
>
>> On Apr 24, 2023, at 9:28 AM, Chris McFarlen <ch...@mcfarlen.us> wrote:
>>
>> Thanks to everyone for the feedback. We have decided to deprecate the NATIVE AIO (libaio) implementation and will remove it after 10.x.
>>
>> Chris
>>
>> Sent with Proton Mail secure email.
>>
>> ------- Original Message -------
>> On Wednesday, April 19th, 2023 at 11:17 PM, Zhao Yongming <mi...@gmail.com> wrote:
>>
>>
>>> missed this thread, I think I need to make it clear that linux native
>>> AIO in the stage of very beta after our Weijin's work, and we are sure
>>> that it is not in good performance, and we don't use it anywhere. I
>>> believe we just do it in suggest of Leif or someone else so we can
>>> compare what is our direct io solution.
>>>
>>> I am sure linux native aio is fine to be killed.
>>>
>>> just FYI.
>>>
>>> On Tue, 2023-04-04 at 11:50 +0000, Chris McFarlen wrote:
>>>
>>>>>> On Apr 3, 2023, at 11:38, Chris McFarlen chris@mcfarlen.us wrote:
>>>>>>
>>>>>> I would like to remove this disk IO mode from ATS. If nobody is
>>>>>> using it, then I'd like to delete it for the 10.x release. If
>>>>>> that isn't possible, I think it should be deprecated in favor of
>>>>>> io_uring and removed in a future version.
>>>>>
>>>>> Is this the Linux native AIO? You’re not suggesting removing ATS
>>>>> own AIO right ?
>>>>
>>>> Correct, AIO_MODE_NATIVE is the linux only mode that uses libaio,
>>>> which is superseded by io_uring. AIO_MODE_THREAD is ATS' own
>>>> implementation that uses a thread pool of AIO threads.
>>>> AIO_MODE_IOURING is the new one that uses io_uring. My proposal is
>>>> to remove AIO_MODE_NATIVE(if unused) and make using io_uring or ATS
>>>> thread IO a startup decision based on config and kernel probing
>>>> rather than compile time.
>>>>
>>>> I think this will allow an easier, more palatable transition to
>>>> io_uring. If there are stability problems then it can be switched
>>>> off with config and a restart rather than a code rollback. io_uring
>>>> support is still very kernel-version dependent so stability and
>>>> feature availability will vary based on what version is deployed.
>>>> This should also (hopefully) allow us to test io_uring in CI if
>>>> available without separate build pipelines.
>>>>
>>>>> — Leif
>>>>>
>>>>>> Is anyone using this IO mode currently? If so, are there any
>>>>>> arguments against deprecation in favor of io_uring for linux?
>>>>>>
>>>>>> Thanks,
>>>>>> Chris
>>>
>>>
>>> --
>>> -Miles Zhao, aka Yongming
>>> 029 128 4335
>
Re: Feedback on removing AIO_MODE_NATIVE from iocore/aio
Posted by Bryan Call <bc...@apache.org>.
It is an experimental feature that isn’t used to my knowledge. I am ok with removing it for 10.x.
If it was a supported feature then we should do the deprecation in one major version and then remove it in the next major version process.
-Bryan
> On Apr 24, 2023, at 9:28 AM, Chris McFarlen <ch...@mcfarlen.us> wrote:
>
> Thanks to everyone for the feedback. We have decided to deprecate the NATIVE AIO (libaio) implementation and will remove it after 10.x.
>
> Chris
>
> Sent with Proton Mail secure email.
>
> ------- Original Message -------
> On Wednesday, April 19th, 2023 at 11:17 PM, Zhao Yongming <mi...@gmail.com> wrote:
>
>
>> missed this thread, I think I need to make it clear that linux native
>> AIO in the stage of very beta after our Weijin's work, and we are sure
>> that it is not in good performance, and we don't use it anywhere. I
>> believe we just do it in suggest of Leif or someone else so we can
>> compare what is our direct io solution.
>>
>> I am sure linux native aio is fine to be killed.
>>
>> just FYI.
>>
>> On Tue, 2023-04-04 at 11:50 +0000, Chris McFarlen wrote:
>>
>>>>> On Apr 3, 2023, at 11:38, Chris McFarlen chris@mcfarlen.us wrote:
>>>>>
>>>>> I would like to remove this disk IO mode from ATS. If nobody is
>>>>> using it, then I'd like to delete it for the 10.x release. If
>>>>> that isn't possible, I think it should be deprecated in favor of
>>>>> io_uring and removed in a future version.
>>>>
>>>> Is this the Linux native AIO? You’re not suggesting removing ATS
>>>> own AIO right ?
>>>
>>> Correct, AIO_MODE_NATIVE is the linux only mode that uses libaio,
>>> which is superseded by io_uring. AIO_MODE_THREAD is ATS' own
>>> implementation that uses a thread pool of AIO threads.
>>> AIO_MODE_IOURING is the new one that uses io_uring. My proposal is
>>> to remove AIO_MODE_NATIVE(if unused) and make using io_uring or ATS
>>> thread IO a startup decision based on config and kernel probing
>>> rather than compile time.
>>>
>>> I think this will allow an easier, more palatable transition to
>>> io_uring. If there are stability problems then it can be switched
>>> off with config and a restart rather than a code rollback. io_uring
>>> support is still very kernel-version dependent so stability and
>>> feature availability will vary based on what version is deployed.
>>> This should also (hopefully) allow us to test io_uring in CI if
>>> available without separate build pipelines.
>>>
>>>> — Leif
>>>>
>>>>> Is anyone using this IO mode currently? If so, are there any
>>>>> arguments against deprecation in favor of io_uring for linux?
>>>>>
>>>>> Thanks,
>>>>> Chris
>>
>>
>> --
>> -Miles Zhao, aka Yongming
>> 029 128 4335
Re: Feedback on removing AIO_MODE_NATIVE from iocore/aio
Posted by Chris McFarlen <ch...@mcfarlen.us>.
Thanks to everyone for the feedback. We have decided to deprecate the NATIVE AIO (libaio) implementation and will remove it after 10.x.
Chris
Sent with Proton Mail secure email.
------- Original Message -------
On Wednesday, April 19th, 2023 at 11:17 PM, Zhao Yongming <mi...@gmail.com> wrote:
> missed this thread, I think I need to make it clear that linux native
> AIO in the stage of very beta after our Weijin's work, and we are sure
> that it is not in good performance, and we don't use it anywhere. I
> believe we just do it in suggest of Leif or someone else so we can
> compare what is our direct io solution.
>
> I am sure linux native aio is fine to be killed.
>
> just FYI.
>
> On Tue, 2023-04-04 at 11:50 +0000, Chris McFarlen wrote:
>
> > > > On Apr 3, 2023, at 11:38, Chris McFarlen chris@mcfarlen.us wrote:
> > > >
> > > > I would like to remove this disk IO mode from ATS. If nobody is
> > > > using it, then I'd like to delete it for the 10.x release. If
> > > > that isn't possible, I think it should be deprecated in favor of
> > > > io_uring and removed in a future version.
> > >
> > > Is this the Linux native AIO? You’re not suggesting removing ATS
> > > own AIO right ?
> >
> > Correct, AIO_MODE_NATIVE is the linux only mode that uses libaio,
> > which is superseded by io_uring. AIO_MODE_THREAD is ATS' own
> > implementation that uses a thread pool of AIO threads.
> > AIO_MODE_IOURING is the new one that uses io_uring. My proposal is
> > to remove AIO_MODE_NATIVE(if unused) and make using io_uring or ATS
> > thread IO a startup decision based on config and kernel probing
> > rather than compile time.
> >
> > I think this will allow an easier, more palatable transition to
> > io_uring. If there are stability problems then it can be switched
> > off with config and a restart rather than a code rollback. io_uring
> > support is still very kernel-version dependent so stability and
> > feature availability will vary based on what version is deployed.
> > This should also (hopefully) allow us to test io_uring in CI if
> > available without separate build pipelines.
> >
> > > — Leif
> > >
> > > > Is anyone using this IO mode currently? If so, are there any
> > > > arguments against deprecation in favor of io_uring for linux?
> > > >
> > > > Thanks,
> > > > Chris
>
>
> --
> -Miles Zhao, aka Yongming
> 029 128 4335
Re: Feedback on removing AIO_MODE_NATIVE from iocore/aio
Posted by Zhao Yongming <mi...@gmail.com>.
missed this thread, I think I need to make it clear that linux native
AIO in the stage of very beta after our Weijin's work, and we are sure
that it is not in good performance, and we don't use it anywhere. I
believe we just do it in suggest of Leif or someone else so we can
compare what is our direct io solution.
I am sure linux native aio is fine to be killed.
just FYI.
On Tue, 2023-04-04 at 11:50 +0000, Chris McFarlen wrote:
>
> > > On Apr 3, 2023, at 11:38, Chris McFarlen chris@mcfarlen.us wrote:
> > >
> > > I would like to remove this disk IO mode from ATS. If nobody is
> > > using it, then I'd like to delete it for the 10.x release. If
> > > that isn't possible, I think it should be deprecated in favor of
> > > io_uring and removed in a future version.
> >
> >
> > Is this the Linux native AIO? You’re not suggesting removing ATS
> > own AIO right ?
>
> Correct, AIO_MODE_NATIVE is the linux only mode that uses libaio,
> which is superseded by io_uring. AIO_MODE_THREAD is ATS' own
> implementation that uses a thread pool of AIO threads.
> AIO_MODE_IOURING is the new one that uses io_uring. My proposal is
> to remove AIO_MODE_NATIVE(if unused) and make using io_uring or ATS
> thread IO a startup decision based on config and kernel probing
> rather than compile time.
>
> I think this will allow an easier, more palatable transition to
> io_uring. If there are stability problems then it can be switched
> off with config and a restart rather than a code rollback. io_uring
> support is still very kernel-version dependent so stability and
> feature availability will vary based on what version is deployed.
> This should also (hopefully) allow us to test io_uring in CI if
> available without separate build pipelines.
>
>
> >
> > — Leif
> >
> > > Is anyone using this IO mode currently? If so, are there any
> > > arguments against deprecation in favor of io_uring for linux?
> > >
> > > Thanks,
> > > Chris
--
-Miles Zhao, aka Yongming
029 128 4335
Re: Feedback on removing AIO_MODE_NATIVE from iocore/aio
Posted by Chris McFarlen <ch...@mcfarlen.us>.
> > On Apr 3, 2023, at 11:38, Chris McFarlen chris@mcfarlen.us wrote:
> >
> > I would like to remove this disk IO mode from ATS. If nobody is using it, then I'd like to delete it for the 10.x release. If that isn't possible, I think it should be deprecated in favor of io_uring and removed in a future version.
>
>
> Is this the Linux native AIO? You’re not suggesting removing ATS own AIO right ?
Correct, AIO_MODE_NATIVE is the linux only mode that uses libaio, which is superseded by io_uring. AIO_MODE_THREAD is ATS' own implementation that uses a thread pool of AIO threads. AIO_MODE_IOURING is the new one that uses io_uring. My proposal is to remove AIO_MODE_NATIVE(if unused) and make using io_uring or ATS thread IO a startup decision based on config and kernel probing rather than compile time.
I think this will allow an easier, more palatable transition to io_uring. If there are stability problems then it can be switched off with config and a restart rather than a code rollback. io_uring support is still very kernel-version dependent so stability and feature availability will vary based on what version is deployed. This should also (hopefully) allow us to test io_uring in CI if available without separate build pipelines.
>
> — Leif
>
> > Is anyone using this IO mode currently? If so, are there any arguments against deprecation in favor of io_uring for linux?
> >
> > Thanks,
> > Chris
Re: Feedback on removing AIO_MODE_NATIVE from iocore/aio
Posted by Leif Hedstrom <zw...@apache.org>.
> On Apr 3, 2023, at 11:38, Chris McFarlen <ch...@mcfarlen.us> wrote:
>
> I would like to remove this disk IO mode from ATS. If nobody is using it, then I'd like to delete it for the 10.x release. If that isn't possible, I think it should be deprecated in favor of io_uring and removed in a future version.
Is this the Linux native AIO? You’re not suggesting removing ATS own AIO right ?
— Leif
>
> Is anyone using this IO mode currently? If so, are there any arguments against deprecation in favor of io_uring for linux?
>
> Thanks,
> Chris
RE: Feedback on removing AIO_MODE_NATIVE from iocore/aio
Posted by Alan Carroll <am...@network-geographics.com>.
I agree with deprecating Linux native AIO. I have heard of many problems with it and not once any successful use. I don't think it's the ATS implementation, I think the kernel support has never been all there. And now with io_uring, I doubt it will ever be fixed, rather native AIO will be replaced by io_uring.
-----Original Message-----
From: Chris McFarlen <ch...@mcfarlen.us>
Sent: Monday, April 3, 2023 12:38 PM
To: dev@trafficserver.apache.org
Subject: Feedback on removing AIO_MODE_NATIVE from iocore/aio
I would like to remove this disk IO mode from ATS. If nobody is using it, then I'd like to delete it for the 10.x release. If that isn't possible, I think it should be deprecated in favor of io_uring and removed in a future version.
Is anyone using this IO mode currently? If so, are there any arguments against deprecation in favor of io_uring for linux?
Thanks,
Chris
Re: [E] Feedback on removing AIO_MODE_NATIVE from iocore/aio
Posted by Chris McFarlen <ch...@mcfarlen.us>.
> I believe there's a concern about io_uring in that using it means ATS can
> now crash the kernel, which is harder to debug than a process crash.
Agreed. io_uring is still in development. That's why I'm proposing this runtime selection so we can have a more managed transition to it for those that want to.
>
> On Mon, Apr 3, 2023 at 12:38 PM Chris McFarlen chris@mcfarlen.us wrote:
>
> > I would like to remove this disk IO mode from ATS. If nobody is using it,
> > then I'd like to delete it for the 10.x release. If that isn't possible, I
> > think it should be deprecated in favor of io_uring and removed in a future
> > version.
> >
> > Is anyone using this IO mode currently? If so, are there any arguments
> > against deprecation in favor of io_uring for linux?
> >
> > Thanks,
> > Chris
Re: [E] Feedback on removing AIO_MODE_NATIVE from iocore/aio
Posted by Walt Karas <wk...@yahooinc.com.INVALID>.
I believe there's a concern about io_uring in that using it means ATS can
now crash the kernel, which is harder to debug than a process crash.
On Mon, Apr 3, 2023 at 12:38 PM Chris McFarlen <ch...@mcfarlen.us> wrote:
> I would like to remove this disk IO mode from ATS. If nobody is using it,
> then I'd like to delete it for the 10.x release. If that isn't possible, I
> think it should be deprecated in favor of io_uring and removed in a future
> version.
>
> Is anyone using this IO mode currently? If so, are there any arguments
> against deprecation in favor of io_uring for linux?
>
> Thanks,
> Chris