You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apisix.apache.org by Ling Samuel <li...@gmail.com> on 2023/06/14 08:55:02 UTC
[DISCUSS] Proposal: remove snowflake algorithm in the request-id plugin
*Background*
The snowflake algorithm in the request-id plugin introduces an unnecessary
dependency on etcd. This will significantly affect APISIX performance if
etcd is unavailable.
Snowflake depends on a unique worker_id, which is acquired from etcd. If
etcd fails, each request with request-id plugin that uses the snowflake
algorithm needs to try to retrieve a unique worker_id. This has a
significant impact on performance.
In addition, the data plane may not have access to etcd. This limitation
would make the algorithm completely unusable.
In the request-id scenario, uuid can generate a globally unique id, which
is enough for this case.
So I suggest removing this algorithm from the request-id plugin. There are
some benefits here:
1. reduce code complexity and the maintenance burden
2. remove dependency on etcd on the data plane
3. reduce the potential instability of request-id that may affect user
business
Re: [DISCUSS] Proposal: remove snowflake algorithm in the request-id plugin
Posted by wei liu <mo...@gmail.com>.
I agree with the removal of this plugin.
The uuid algorithm is sufficient for now.
The snowflake algorithm does not provide additional benefits and leads to
possible business disruptions.
If the user insists on using this algorithm, as it is only a plugin, it is
relatively easy to upgrade up from the old version.
On Thu, Jun 15, 2023 at 12:38 AM LI Fabrice <yi...@hotmail.com> wrote:
> Hi,
>
> First of all, that will be a breaking change. In addition, if the user is
> using the old version and then updates to the new version, it is necessary
> to verify and explain the change of request-id plugin. Finally, can you
> give some info of the request-id plugin when etcd is unavailable?
>
> Fabriceli
> https://github.com/Fabriceli
>
> > 2023年6月14日 16:55，Ling Samuel <li...@gmail.com> 写道：
> >
> > *Background*
> >
> > The snowflake algorithm in the request-id plugin introduces an
> unnecessary
> > dependency on etcd. This will significantly affect APISIX performance if
> > etcd is unavailable.
> >
> > Snowflake depends on a unique worker_id, which is acquired from etcd. If
> > etcd fails, each request with request-id plugin that uses the snowflake
> > algorithm needs to try to retrieve a unique worker_id. This has a
> > significant impact on performance.
> >
> > In addition, the data plane may not have access to etcd. This limitation
> > would make the algorithm completely unusable.
> >
> > In the request-id scenario, uuid can generate a globally unique id, which
> > is enough for this case.
> >
> > So I suggest removing this algorithm from the request-id plugin. There
> are
> > some benefits here:
> >
> > 1. reduce code complexity and the maintenance burden
> > 2. remove dependency on etcd on the data plane
> > 3. reduce the potential instability of request-id that may affect user
> > business
>
>
Re: [DISCUSS] Proposal: remove snowflake algorithm in the request-id plugin
Posted by Chao Zhang <zc...@gmail.com>.
OK, I see. +1
Chao Zhang
https://github.com/tokers
On June 20, 2023 at 15:12:27, Ling Samuel (lingsamuelgrace@gmail.com) wrote:
Hi, it is a breaking change indeed.
About the plugin detail, I have explained in the main thread.
> Snowflake depends on a unique worker_id, which is acquired from etcd. If
etcd
fails, each request with request-id plugin that uses the snowflake
algorithm
needs to try to retrieve a unique worker_id. This has a significant impact
on performance.
LI Fabrice <yi...@hotmail.com> 于2023年6月15日周四 00:38写道：
> Hi,
>
> First of all, that will be a breaking change. In addition, if the user is
> using the old version and then updates to the new version, it is
necessary
> to verify and explain the change of request-id plugin. Finally, can you
> give some info of the request-id plugin when etcd is unavailable?
>
> Fabriceli
> https://github.com/Fabriceli
>
> > 2023年6月14日 16:55，Ling Samuel <li...@gmail.com> 写道：
> >
> > *Background*
> >
> > The snowflake algorithm in the request-id plugin introduces an
> unnecessary
> > dependency on etcd. This will significantly affect APISIX performance
if
> > etcd is unavailable.
> >
> > Snowflake depends on a unique worker_id, which is acquired from etcd.
If
> > etcd fails, each request with request-id plugin that uses the snowflake
> > algorithm needs to try to retrieve a unique worker_id. This has a
> > significant impact on performance.
> >
> > In addition, the data plane may not have access to etcd. This
limitation
> > would make the algorithm completely unusable.
> >
> > In the request-id scenario, uuid can generate a globally unique id,
which
> > is enough for this case.
> >
> > So I suggest removing this algorithm from the request-id plugin. There
> are
> > some benefits here:
> >
> > 1. reduce code complexity and the maintenance burden
> > 2. remove dependency on etcd on the data plane
> > 3. reduce the potential instability of request-id that may affect user
> > business
>
>
Re: [DISCUSS] Proposal: remove snowflake algorithm in the request-id plugin
Posted by Ling Samuel <li...@gmail.com>.
Hi, it is a breaking change indeed.
About the plugin detail, I have explained in the main thread.
> Snowflake depends on a unique worker_id, which is acquired from etcd. If etcd
fails, each request with request-id plugin that uses the snowflake algorithm
needs to try to retrieve a unique worker_id. This has a significant impact
on performance.
LI Fabrice <yi...@hotmail.com> 于2023年6月15日周四 00:38写道：
> Hi,
>
> First of all, that will be a breaking change. In addition, if the user is
> using the old version and then updates to the new version, it is necessary
> to verify and explain the change of request-id plugin. Finally, can you
> give some info of the request-id plugin when etcd is unavailable?
>
> Fabriceli
> https://github.com/Fabriceli
>
> > 2023年6月14日 16:55，Ling Samuel <li...@gmail.com> 写道：
> >
> > *Background*
> >
> > The snowflake algorithm in the request-id plugin introduces an
> unnecessary
> > dependency on etcd. This will significantly affect APISIX performance if
> > etcd is unavailable.
> >
> > Snowflake depends on a unique worker_id, which is acquired from etcd. If
> > etcd fails, each request with request-id plugin that uses the snowflake
> > algorithm needs to try to retrieve a unique worker_id. This has a
> > significant impact on performance.
> >
> > In addition, the data plane may not have access to etcd. This limitation
> > would make the algorithm completely unusable.
> >
> > In the request-id scenario, uuid can generate a globally unique id, which
> > is enough for this case.
> >
> > So I suggest removing this algorithm from the request-id plugin. There
> are
> > some benefits here:
> >
> > 1. reduce code complexity and the maintenance burden
> > 2. remove dependency on etcd on the data plane
> > 3. reduce the potential instability of request-id that may affect user
> > business
>
>
Re: [DISCUSS] Proposal: remove snowflake algorithm in the request-id plugin
Posted by LI Fabrice <yi...@hotmail.com>.
Hi,
First of all, that will be a breaking change. In addition, if the user is using the old version and then updates to the new version, it is necessary to verify and explain the change of request-id plugin. Finally, can you give some info of the request-id plugin when etcd is unavailable?
Fabriceli
https://github.com/Fabriceli
> 2023年6月14日 16:55，Ling Samuel <li...@gmail.com> 写道：
>
> *Background*
>
> The snowflake algorithm in the request-id plugin introduces an unnecessary
> dependency on etcd. This will significantly affect APISIX performance if
> etcd is unavailable.
>
> Snowflake depends on a unique worker_id, which is acquired from etcd. If
> etcd fails, each request with request-id plugin that uses the snowflake
> algorithm needs to try to retrieve a unique worker_id. This has a
> significant impact on performance.
>
> In addition, the data plane may not have access to etcd. This limitation
> would make the algorithm completely unusable.
>
> In the request-id scenario, uuid can generate a globally unique id, which
> is enough for this case.
>
> So I suggest removing this algorithm from the request-id plugin. There are
> some benefits here:
>
> 1. reduce code complexity and the maintenance burden
> 2. remove dependency on etcd on the data plane
> 3. reduce the potential instability of request-id that may affect user
> business
Re: [DISCUSS] Proposal: remove snowflake algorithm in the request-id plugin
Posted by Jinhua Luo <lu...@gmail.com>.
I prefer the remove snowflake algorithm because it does more harm than
good. And compared to uuid, its only benefits are faster speed and
incremental id generation, but these two benefits have little use in
practice.
Ling Samuel <li...@gmail.com> 于2023年6月14日周三 16:55写道：
>
> *Background*
>
> The snowflake algorithm in the request-id plugin introduces an unnecessary
> dependency on etcd. This will significantly affect APISIX performance if
> etcd is unavailable.
>
> Snowflake depends on a unique worker_id, which is acquired from etcd. If
> etcd fails, each request with request-id plugin that uses the snowflake
> algorithm needs to try to retrieve a unique worker_id. This has a
> significant impact on performance.
>
> In addition, the data plane may not have access to etcd. This limitation
> would make the algorithm completely unusable.
>
> In the request-id scenario, uuid can generate a globally unique id, which
> is enough for this case.
>
> So I suggest removing this algorithm from the request-id plugin. There are
> some benefits here:
>
> 1. reduce code complexity and the maintenance burden
> 2. remove dependency on etcd on the data plane
> 3. reduce the potential instability of request-id that may affect user
> business
Re: [DISCUSS] Proposal: remove snowflake algorithm in the request-id plugin
Posted by LI Fabrice <yi...@hotmail.com>.
Hi,
First of all, that will be a breaking change. In addition, if the user is using the old version and then updates to the new version, it is necessary to verify and explain the change of request-id plugin. Finally, can you give some info of the request-id plugin when etcd is unavailable?
Fabriceli
https://github.com/Fabriceli
> 2023年6月14日 16:55，Ling Samuel <li...@gmail.com> 写道：
>
> *Background*
>
> The snowflake algorithm in the request-id plugin introduces an unnecessary
> dependency on etcd. This will significantly affect APISIX performance if
> etcd is unavailable.
>
> Snowflake depends on a unique worker_id, which is acquired from etcd. If
> etcd fails, each request with request-id plugin that uses the snowflake
> algorithm needs to try to retrieve a unique worker_id. This has a
> significant impact on performance.
>
> In addition, the data plane may not have access to etcd. This limitation
> would make the algorithm completely unusable.
>
> In the request-id scenario, uuid can generate a globally unique id, which
> is enough for this case.
>
> So I suggest removing this algorithm from the request-id plugin. There are
> some benefits here:
>
> 1. reduce code complexity and the maintenance burden
> 2. remove dependency on etcd on the data plane
> 3. reduce the potential instability of request-id that may affect user
> business
Re: [DISCUSS] Proposal: remove snowflake algorithm in the request-id plugin
Posted by Chao Zhang <zc...@gmail.com>.
Hi,
Does this feature used by users?
Chao Zhang
https://github.com/tokers
On June 14, 2023 at 16:55:45, Ling Samuel (lingsamuelgrace@gmail.com) wrote:
*Background*
The snowflake algorithm in the request-id plugin introduces an unnecessary
dependency on etcd. This will significantly affect APISIX performance if
etcd is unavailable.
Snowflake depends on a unique worker_id, which is acquired from etcd. If
etcd fails, each request with request-id plugin that uses the snowflake
algorithm needs to try to retrieve a unique worker_id. This has a
significant impact on performance.
In addition, the data plane may not have access to etcd. This limitation
would make the algorithm completely unusable.
In the request-id scenario, uuid can generate a globally unique id, which
is enough for this case.
So I suggest removing this algorithm from the request-id plugin. There are
some benefits here:
1. reduce code complexity and the maintenance burden
2. remove dependency on etcd on the data plane
3. reduce the potential instability of request-id that may affect user
business