You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apisix.apache.org by wei jin <kv...@apache.org> on 2020/08/20 01:51:58 UTC

[DISCUSS]Modify the manager-api DB from mysql to sqllite

hi, community
Many users have reported that the current apisix-dashboard is a little
troublesome during deployment. One of the problems is that we rely on
mysql. Users need to maintain additional database instances.
I propose to modify mysql to an embedded database such as sqllite.
considerations for using sqllite:
1.Simplify the complexity of manager-api deployment.
2.We still need a relational database to organize the relationship and
minimize the changes caused by database changes.
3.sqllite is a relatively mature database.

Any suggestions on this proposal? Or is there a better recommendation for
an embedded database?

Thanks!

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by Ming Wen <we...@apache.org>.
sqlite is Public Domain[1], which is may be included within Apache
products[2].
So it's better choose a DB which is Apache 2.0 license or a compatible
license.

[1] https://www.sqlite.org/copyright.html
[2]
https://www.apache.org/legal/resolved.html#handling-public-domain-licensed-works

Thanks,
Ming Wen, Apache APISIX & Apache SkyWalking
Twitter: _WenMing


Sheng Wu <wu...@gmail.com> 于2020年8月20日周四 上午11:54写道:

> There are a lot of DB could be chosen, if you like. Many of them are Apache
> 2.0 or a compatible license.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> vincixu <vi...@apache.org> 于2020年8月20日周四 上午11:42写道:
>
> > >  support search and paging, etcd cannot support such features.
> > back-end search and paging is unnecessary, we can implement it in the
> > front-end, because users will not have too many configurations.
> >
> > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
> >
> > > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:
> > >
> > > > I haven't followed APISIX Dashboard until I saw this email.
> > > > In my opinion, APISIX Dashboard should not rely on storage other than
> > > ETCD
> > > > or even ManagerAPI, which will increase the overall complexity.
> > > >
> > >
> > > `manager api` relies on relational databases, because some data needs
> to
> > > support search and paging, etcd cannot support such features.
> > >
> > > For the `route.name` field, it is necessary to ensure that it is
> unique
> > > among all routes, which cannot be implemented in etcd.
> > >
> > >
> > > > This is really not a good idea. Dashboard should be kept simple and
> > easy
> > > to
> > > > deploy, not another huge application.
> > > >
> > > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:
> > > >
> > > > > HI,
> > > > >
> > > > > Last I looked mysql was also GPL which is not compatible with the
> > > Apache
> > > > > License. I assume this is an optional dependancy?
> > > > >
> > > > > Thanks,
> > > > > Justin
> > > >
> > >
> > >
> > > --
> > >
> > > *MembPhis*
> > > My GitHub: https://github.com/membphis
> > > Apache APISIX: https://github.com/apache/incubator-apisix
> > >
> >
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by Sheng Wu <wu...@gmail.com>.
There are a lot of DB could be chosen, if you like. Many of them are Apache
2.0 or a compatible license.

Sheng Wu 吴晟
Twitter, wusheng1108


vincixu <vi...@apache.org> 于2020年8月20日周四 上午11:42写道:

> >  support search and paging, etcd cannot support such features.
> back-end search and paging is unnecessary, we can implement it in the
> front-end, because users will not have too many configurations.
>
> YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
>
> > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:
> >
> > > I haven't followed APISIX Dashboard until I saw this email.
> > > In my opinion, APISIX Dashboard should not rely on storage other than
> > ETCD
> > > or even ManagerAPI, which will increase the overall complexity.
> > >
> >
> > `manager api` relies on relational databases, because some data needs to
> > support search and paging, etcd cannot support such features.
> >
> > For the `route.name` field, it is necessary to ensure that it is unique
> > among all routes, which cannot be implemented in etcd.
> >
> >
> > > This is really not a good idea. Dashboard should be kept simple and
> easy
> > to
> > > deploy, not another huge application.
> > >
> > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:
> > >
> > > > HI,
> > > >
> > > > Last I looked mysql was also GPL which is not compatible with the
> > Apache
> > > > License. I assume this is an optional dependancy?
> > > >
> > > > Thanks,
> > > > Justin
> > >
> >
> >
> > --
> >
> > *MembPhis*
> > My GitHub: https://github.com/membphis
> > Apache APISIX: https://github.com/apache/incubator-apisix
> >
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by Zhiyuan Ju <ju...@apache.org>.
That's fine to do page sorting on the front end if there doesn't have too
many records.

I'm not sure where we should implement some logic out of the manager-api,
such as Authentication.

Best Regards!
@ Zhiyuan Ju <https://www.shaoyaoju.org/>


vincixu <vi...@apache.org> 于2020年8月20日周四 上午11:42写道:

> >  support search and paging, etcd cannot support such features.
> back-end search and paging is unnecessary, we can implement it in the
> front-end, because users will not have too many configurations.
>
> YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
>
> > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:
> >
> > > I haven't followed APISIX Dashboard until I saw this email.
> > > In my opinion, APISIX Dashboard should not rely on storage other than
> > ETCD
> > > or even ManagerAPI, which will increase the overall complexity.
> > >
> >
> > `manager api` relies on relational databases, because some data needs to
> > support search and paging, etcd cannot support such features.
> >
> > For the `route.name` field, it is necessary to ensure that it is unique
> > among all routes, which cannot be implemented in etcd.
> >
> >
> > > This is really not a good idea. Dashboard should be kept simple and
> easy
> > to
> > > deploy, not another huge application.
> > >
> > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:
> > >
> > > > HI,
> > > >
> > > > Last I looked mysql was also GPL which is not compatible with the
> > Apache
> > > > License. I assume this is an optional dependancy?
> > > >
> > > > Thanks,
> > > > Justin
> > >
> >
> >
> > --
> >
> > *MembPhis*
> > My GitHub: https://github.com/membphis
> > Apache APISIX: https://github.com/apache/incubator-apisix
> >
>

回复: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by 指尖旳〃圊舂 ︶ <24...@qq.com>.
When we deploy, we need too much skill
golang mysql nodejs reactjs nginx openresty lua etcd shell

Can we try to make APIs IX easier to use







------------------&nbsp;原始邮件&nbsp;------------------
发件人: "Ming Wen"<wenming@apache.org&gt;; 
发送时间: 2020年8月20日(星期四) 下午5:33
收件人: "dev"<dev@apisix.apache.org&gt;; 
主题: Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite



That&nbsp;will&nbsp;be&nbsp;great&nbsp;if&nbsp;we&nbsp;can&nbsp;remove&nbsp;RDS.

Thanks,
Ming&nbsp;Wen,&nbsp;Apache&nbsp;APISIX&nbsp;&amp;&nbsp;Apache&nbsp;SkyWalking
Twitter:&nbsp;_WenMing


vincixu&nbsp;<vinci@apache.org&gt;&nbsp;于2020年8月20日周四&nbsp;下午4:30写道:

&gt;&nbsp;Relational&nbsp;database&nbsp;is&nbsp;not&nbsp;the&nbsp;only&nbsp;way.k8s&nbsp;implement&nbsp;all&nbsp;features(RBAC,
&gt;&nbsp;ABAC,&nbsp;cascade)&nbsp;we&nbsp;need&nbsp;but&nbsp;no&nbsp;relational&nbsp;databases.
&gt;&nbsp;Keep&nbsp;configuration&nbsp;in&nbsp;two&nbsp;places,&nbsp;it&nbsp;will&nbsp;bring&nbsp;a&nbsp;hard&nbsp;problem:&nbsp;how&nbsp;to
&gt;&nbsp;keep&nbsp;consistency.A&nbsp;problem&nbsp;that&nbsp;engineers&nbsp;have&nbsp;struggled&nbsp;for&nbsp;many&nbsp;years&nbsp;but
&gt;&nbsp;there&nbsp;has&nbsp;been&nbsp;no&nbsp;easy&nbsp;solution&nbsp;until&nbsp;now,I&nbsp;think&nbsp;it’s&nbsp;not&nbsp;worth
&gt;&nbsp;introducing&nbsp;such&nbsp;a&nbsp;problem&nbsp;to&nbsp;implement&nbsp;the&nbsp;mentioned&nbsp;features.
&gt;&nbsp;(&nbsp;I&nbsp;also&nbsp;have&nbsp;never&nbsp;seen&nbsp;a&nbsp;project&nbsp;using&nbsp;etcd&nbsp;and&nbsp;relational&nbsp;storage&nbsp;at&nbsp;the
&gt;&nbsp;same&nbsp;time&nbsp;to&nbsp;store&nbsp;the&nbsp;same&nbsp;data,&nbsp;)
&gt;
&gt;&nbsp;wei&nbsp;jin&nbsp;<kvn@apache.org&gt;&nbsp;于2020年8月20日周四&nbsp;下午4:04写道:
&gt;
&gt;&nbsp;&gt;&nbsp;We&nbsp;need&nbsp;a&nbsp;relational&nbsp;database&nbsp;to&nbsp;do&nbsp;some&nbsp;function&nbsp;extensions&nbsp;to&nbsp;meet&nbsp;some
&gt;&nbsp;&gt;&nbsp;practical&nbsp;needs,&nbsp;such&nbsp;as&nbsp;authority&nbsp;management.&nbsp;But&nbsp;we&nbsp;also&nbsp;hope&nbsp;to&nbsp;keep
&gt;&nbsp;&gt;&nbsp;apisix&nbsp;sufficiently&nbsp;convergent&nbsp;and&nbsp;focus&nbsp;on&nbsp;a&nbsp;high-performance&nbsp;gateway.
&gt;&nbsp;&gt;&nbsp;APISIX&nbsp;should&nbsp;not&nbsp;be&nbsp;affected&nbsp;by&nbsp;external&nbsp;function&nbsp;extensions.
&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;vincixu&nbsp;<vinci@apache.org&gt;&nbsp;于2020年8月20日周四&nbsp;下午3:53写道:
&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;I&nbsp;don&#39;t&nbsp;think&nbsp;a&nbsp;lot&nbsp;of&nbsp;routes&nbsp;is&nbsp;a&nbsp;good&nbsp;practice.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;But&nbsp;we&nbsp;can&nbsp;discuss&nbsp;how&nbsp;many&nbsp;routes&nbsp;APISIX&nbsp;plans&nbsp;to&nbsp;support.&nbsp;There&nbsp;is&nbsp;no
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;pressure&nbsp;on&nbsp;the&nbsp;front-end&nbsp;sorting&nbsp;of&nbsp;records&nbsp;less&nbsp;than&nbsp;50,000
&gt;&nbsp;records(Our
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;team&nbsp;way).
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;If&nbsp;we&nbsp;plan&nbsp;to&nbsp;support&nbsp;large&nbsp;rotes&nbsp;query,&nbsp;&nbsp;paging&nbsp;and&nbsp;so&nbsp;on,&nbsp;we&nbsp;can&nbsp;load
&gt;&nbsp;&gt;&nbsp;the
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;data&nbsp;into&nbsp;memory,&nbsp;and&nbsp;then&nbsp;simulate&nbsp;query&nbsp;and&nbsp;paging&nbsp;in&nbsp;the&nbsp;backend
&gt;&nbsp;&gt;&nbsp;instead
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;of&nbsp;depend&nbsp;on&nbsp;another&nbsp;storage&nbsp;(the&nbsp;k8s&nbsp;apiserver&nbsp;way),
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;if&nbsp;we&nbsp;really&nbsp;need&nbsp;a&nbsp;storage&nbsp;that&nbsp;is&nbsp;easy&nbsp;to&nbsp;query,&nbsp;why&nbsp;not&nbsp;directly&nbsp;use
&gt;&nbsp;&gt;&nbsp;it
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;as&nbsp;the&nbsp;configuration&nbsp;library&nbsp;of&nbsp;APISIX?&nbsp;Just&nbsp;like&nbsp;tyk&nbsp;did(&nbsp;using&nbsp;mongo
&gt;&nbsp;as
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;configuration&nbsp;storage).
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;It&nbsp;is&nbsp;very&nbsp;dangerous&nbsp;and&nbsp;inelegant&nbsp;to&nbsp;keep&nbsp;duplicate&nbsp;configuration&nbsp;even
&gt;&nbsp;&gt;&nbsp;in
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;different&nbsp;storage&nbsp;media.
&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;YuanSheng&nbsp;Wang&nbsp;<membphis@apache.org&gt;&nbsp;于2020年8月20日周四&nbsp;下午12:30写道:
&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;On&nbsp;Thu,&nbsp;Aug&nbsp;20,&nbsp;2020&nbsp;at&nbsp;11:42&nbsp;AM&nbsp;vincixu&nbsp;<vinci@apache.org&gt;&nbsp;wrote:
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&nbsp;support&nbsp;search&nbsp;and&nbsp;paging,&nbsp;etcd&nbsp;cannot&nbsp;support&nbsp;such&nbsp;features.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;back-end&nbsp;search&nbsp;and&nbsp;paging&nbsp;is&nbsp;unnecessary,&nbsp;we&nbsp;can&nbsp;implement&nbsp;it&nbsp;in
&gt;&nbsp;the
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;front-end,&nbsp;because&nbsp;users&nbsp;will&nbsp;not&nbsp;have&nbsp;too&nbsp;many&nbsp;configurations.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;because&nbsp;users&nbsp;will&nbsp;not&nbsp;have&nbsp;too&nbsp;many&nbsp;configurations
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;Apache&nbsp;APISIX&nbsp;can&nbsp;support&nbsp;a&nbsp;large&nbsp;number&nbsp;of&nbsp;routes,&nbsp;which&nbsp;is&nbsp;an
&gt;&nbsp;&gt;&nbsp;important
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;feature&nbsp;for&nbsp;our&nbsp;users.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;Another&nbsp;case:
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;the&nbsp;`service`&nbsp;or&nbsp;`upstream`&nbsp;object&nbsp;can&nbsp;be&nbsp;bound&nbsp;in&nbsp;`route`,&nbsp;when&nbsp;we
&gt;&nbsp;&gt;&nbsp;need
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;to
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;delete&nbsp;the&nbsp;`service`
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;or&nbsp;`upstream`&nbsp;object,&nbsp;we&nbsp;need&nbsp;to&nbsp;confirm&nbsp;there&nbsp;is&nbsp;no&nbsp;route&nbsp;using&nbsp;it.
&gt;&nbsp;if
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;we
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;are&nbsp;using&nbsp;RDS,&nbsp;its&nbsp;easy&nbsp;for
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;this&nbsp;case.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;If&nbsp;we&nbsp;only&nbsp;use&nbsp;etcd,&nbsp;it&nbsp;is&nbsp;hard&nbsp;to&nbsp;support&nbsp;this&nbsp;case&nbsp;too.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;YuanSheng&nbsp;Wang&nbsp;<membphis@apache.org&gt;&nbsp;于2020年8月20日周四&nbsp;上午11:29写道:
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;On&nbsp;Thu,&nbsp;Aug&nbsp;20,&nbsp;2020&nbsp;at&nbsp;10:57&nbsp;AM&nbsp;vincixu&nbsp;<vinci@apache.org&gt;
&gt;&nbsp;wrote:
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;I&nbsp;haven&#39;t&nbsp;followed&nbsp;APISIX&nbsp;Dashboard&nbsp;until&nbsp;I&nbsp;saw&nbsp;this&nbsp;email.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;In&nbsp;my&nbsp;opinion,&nbsp;APISIX&nbsp;Dashboard&nbsp;should&nbsp;not&nbsp;rely&nbsp;on&nbsp;storage
&gt;&nbsp;other
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;than
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;ETCD
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;or&nbsp;even&nbsp;ManagerAPI,&nbsp;which&nbsp;will&nbsp;increase&nbsp;the&nbsp;overall&nbsp;complexity.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;`manager&nbsp;api`&nbsp;relies&nbsp;on&nbsp;relational&nbsp;databases,&nbsp;because&nbsp;some&nbsp;data
&gt;&nbsp;&gt;&nbsp;needs
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;to
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;support&nbsp;search&nbsp;and&nbsp;paging,&nbsp;etcd&nbsp;cannot&nbsp;support&nbsp;such&nbsp;features.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;For&nbsp;the&nbsp;`route.name`&nbsp;field,&nbsp;it&nbsp;is&nbsp;necessary&nbsp;to&nbsp;ensure&nbsp;that&nbsp;it&nbsp;is
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;unique
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;among&nbsp;all&nbsp;routes,&nbsp;which&nbsp;cannot&nbsp;be&nbsp;implemented&nbsp;in&nbsp;etcd.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;This&nbsp;is&nbsp;really&nbsp;not&nbsp;a&nbsp;good&nbsp;idea.&nbsp;Dashboard&nbsp;should&nbsp;be&nbsp;kept&nbsp;simple
&gt;&nbsp;&gt;&nbsp;and
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;easy
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;to
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;deploy,&nbsp;not&nbsp;another&nbsp;huge&nbsp;application.
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;Justin&nbsp;Mclean&nbsp;<justin@classsoftware.com&gt;&nbsp;于2020年8月20日周四
&gt;&nbsp;&gt;&nbsp;上午10:05写道:
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;HI,
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;Last&nbsp;I&nbsp;looked&nbsp;mysql&nbsp;was&nbsp;also&nbsp;GPL&nbsp;which&nbsp;is&nbsp;not&nbsp;compatible&nbsp;with
&gt;&nbsp;&gt;&nbsp;the
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;Apache
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;License.&nbsp;I&nbsp;assume&nbsp;this&nbsp;is&nbsp;an&nbsp;optional&nbsp;dependancy?
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;Thanks,
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;Justin
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;--
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;*MembPhis*
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;My&nbsp;GitHub:&nbsp;https://github.com/membphis
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;Apache&nbsp;APISIX:&nbsp;https://github.com/apache/incubator-apisix
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;--
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;*MembPhis*
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;My&nbsp;GitHub:&nbsp;https://github.com/membphis
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;Apache&nbsp;APISIX:&nbsp;https://github.com/apache/incubator-apisix
&gt;&nbsp;&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;&nbsp;&gt;
&gt;&nbsp;&gt;
&gt;

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by Ming Wen <we...@apache.org>.
created a github milestone for this discuss:
https://github.com/apache/apisix-dashboard/milestone/3

Thanks,
Ming Wen, Apache APISIX & Apache SkyWalking
Twitter: _WenMing


wei jin <kv...@apache.org> 于2020年8月21日周五 上午11:04写道:

> ok, I agree that the data is persisted in etcd, which can indeed avoid a
> lot of unnecessary trouble.
>
> YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 下午11:22写道:
>
> > On Thu, Aug 20, 2020 at 5:33 PM Ming Wen <we...@apache.org> wrote:
> >
> > > That will be great if we can remove RDS.
> > >
> >
> > yes, I love this way too.
> >
> >
> > >
> > > Thanks,
> > > Ming Wen, Apache APISIX & Apache SkyWalking
> > > Twitter: _WenMing
> > >
> > >
> > > vincixu <vi...@apache.org> 于2020年8月20日周四 下午4:30写道:
> > >
> > > > Relational database is not the only way.k8s implement all
> > features(RBAC,
> > > > ABAC, cascade) we need but no relational databases.
> > > > Keep configuration in two places, it will bring a hard problem: how
> to
> > > > keep consistency.A problem that engineers have struggled for many
> years
> > > but
> > > > there has been no easy solution until now,I think it’s not worth
> > > > introducing such a problem to implement the mentioned features.
> > > > ( I also have never seen a project using etcd and relational storage
> at
> > > the
> > > > same time to store the same data, )
> > > >
> > > > wei jin <kv...@apache.org> 于2020年8月20日周四 下午4:04写道:
> > > >
> > > > > We need a relational database to do some function extensions to
> meet
> > > some
> > > > > practical needs, such as authority management. But we also hope to
> > keep
> > > > > apisix sufficiently convergent and focus on a high-performance
> > gateway.
> > > > > APISIX should not be affected by external function extensions.
> > > > >
> > > > > vincixu <vi...@apache.org> 于2020年8月20日周四 下午3:53写道:
> > > > >
> > > > > > I don't think a lot of routes is a good practice.
> > > > > > But we can discuss how many routes APISIX plans to support. There
> > is
> > > no
> > > > > > pressure on the front-end sorting of records less than 50,000
> > > > records(Our
> > > > > > team way).
> > > > > > If we plan to support large rotes query,  paging and so on, we
> can
> > > load
> > > > > the
> > > > > > data into memory, and then simulate query and paging in the
> backend
> > > > > instead
> > > > > > of depend on another storage (the k8s apiserver way),
> > > > > > if we really need a storage that is easy to query, why not
> directly
> > > use
> > > > > it
> > > > > > as the configuration library of APISIX? Just like tyk did( using
> > > mongo
> > > > as
> > > > > > configuration storage).
> > > > > > It is very dangerous and inelegant to keep duplicate
> configuration
> > > even
> > > > > in
> > > > > > different storage media.
> > > > > >
> > > > > > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 下午12:30写道:
> > > > > >
> > > > > > > On Thu, Aug 20, 2020 at 11:42 AM vincixu <vi...@apache.org>
> > wrote:
> > > > > > >
> > > > > > > > >  support search and paging, etcd cannot support such
> > features.
> > > > > > > > back-end search and paging is unnecessary, we can implement
> it
> > in
> > > > the
> > > > > > > > front-end, because users will not have too many
> configurations.
> > > > > > > >
> > > > > > >
> > > > > > > > because users will not have too many configurations
> > > > > > >
> > > > > > > Apache APISIX can support a large number of routes, which is an
> > > > > important
> > > > > > > feature for our users.
> > > > > > >
> > > > > > > Another case:
> > > > > > >
> > > > > > > the `service` or `upstream` object can be bound in `route`,
> when
> > we
> > > > > need
> > > > > > to
> > > > > > > delete the `service`
> > > > > > > or `upstream` object, we need to confirm there is no route
> using
> > > it.
> > > > if
> > > > > > we
> > > > > > > are using RDS, its easy for
> > > > > > > this case.
> > > > > > >
> > > > > > > If we only use etcd, it is hard to support this case too.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四
> 上午11:29写道:
> > > > > > > >
> > > > > > > > > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vinci@apache.org
> >
> > > > wrote:
> > > > > > > > >
> > > > > > > > > > I haven't followed APISIX Dashboard until I saw this
> email.
> > > > > > > > > > In my opinion, APISIX Dashboard should not rely on
> storage
> > > > other
> > > > > > than
> > > > > > > > > ETCD
> > > > > > > > > > or even ManagerAPI, which will increase the overall
> > > complexity.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > `manager api` relies on relational databases, because some
> > data
> > > > > needs
> > > > > > > to
> > > > > > > > > support search and paging, etcd cannot support such
> features.
> > > > > > > > >
> > > > > > > > > For the `route.name` field, it is necessary to ensure that
> > it
> > > is
> > > > > > > unique
> > > > > > > > > among all routes, which cannot be implemented in etcd.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > This is really not a good idea. Dashboard should be kept
> > > simple
> > > > > and
> > > > > > > > easy
> > > > > > > > > to
> > > > > > > > > > deploy, not another huge application.
> > > > > > > > > >
> > > > > > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四
> > > > > 上午10:05写道:
> > > > > > > > > >
> > > > > > > > > > > HI,
> > > > > > > > > > >
> > > > > > > > > > > Last I looked mysql was also GPL which is not
> compatible
> > > with
> > > > > the
> > > > > > > > > Apache
> > > > > > > > > > > License. I assume this is an optional dependancy?
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Justin
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > *MembPhis*
> > > > > > > > > My GitHub: https://github.com/membphis
> > > > > > > > > Apache APISIX: https://github.com/apache/incubator-apisix
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > *MembPhis*
> > > > > > > My GitHub: https://github.com/membphis
> > > > > > > Apache APISIX: https://github.com/apache/incubator-apisix
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > *MembPhis*
> > My GitHub: https://github.com/membphis
> > Apache APISIX: https://github.com/apache/incubator-apisix
> >
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by wei jin <kv...@apache.org>.
ok, I agree that the data is persisted in etcd, which can indeed avoid a
lot of unnecessary trouble.

YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 下午11:22写道:

> On Thu, Aug 20, 2020 at 5:33 PM Ming Wen <we...@apache.org> wrote:
>
> > That will be great if we can remove RDS.
> >
>
> yes, I love this way too.
>
>
> >
> > Thanks,
> > Ming Wen, Apache APISIX & Apache SkyWalking
> > Twitter: _WenMing
> >
> >
> > vincixu <vi...@apache.org> 于2020年8月20日周四 下午4:30写道:
> >
> > > Relational database is not the only way.k8s implement all
> features(RBAC,
> > > ABAC, cascade) we need but no relational databases.
> > > Keep configuration in two places, it will bring a hard problem: how to
> > > keep consistency.A problem that engineers have struggled for many years
> > but
> > > there has been no easy solution until now,I think it’s not worth
> > > introducing such a problem to implement the mentioned features.
> > > ( I also have never seen a project using etcd and relational storage at
> > the
> > > same time to store the same data, )
> > >
> > > wei jin <kv...@apache.org> 于2020年8月20日周四 下午4:04写道:
> > >
> > > > We need a relational database to do some function extensions to meet
> > some
> > > > practical needs, such as authority management. But we also hope to
> keep
> > > > apisix sufficiently convergent and focus on a high-performance
> gateway.
> > > > APISIX should not be affected by external function extensions.
> > > >
> > > > vincixu <vi...@apache.org> 于2020年8月20日周四 下午3:53写道:
> > > >
> > > > > I don't think a lot of routes is a good practice.
> > > > > But we can discuss how many routes APISIX plans to support. There
> is
> > no
> > > > > pressure on the front-end sorting of records less than 50,000
> > > records(Our
> > > > > team way).
> > > > > If we plan to support large rotes query,  paging and so on, we can
> > load
> > > > the
> > > > > data into memory, and then simulate query and paging in the backend
> > > > instead
> > > > > of depend on another storage (the k8s apiserver way),
> > > > > if we really need a storage that is easy to query, why not directly
> > use
> > > > it
> > > > > as the configuration library of APISIX? Just like tyk did( using
> > mongo
> > > as
> > > > > configuration storage).
> > > > > It is very dangerous and inelegant to keep duplicate configuration
> > even
> > > > in
> > > > > different storage media.
> > > > >
> > > > > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 下午12:30写道:
> > > > >
> > > > > > On Thu, Aug 20, 2020 at 11:42 AM vincixu <vi...@apache.org>
> wrote:
> > > > > >
> > > > > > > >  support search and paging, etcd cannot support such
> features.
> > > > > > > back-end search and paging is unnecessary, we can implement it
> in
> > > the
> > > > > > > front-end, because users will not have too many configurations.
> > > > > > >
> > > > > >
> > > > > > > because users will not have too many configurations
> > > > > >
> > > > > > Apache APISIX can support a large number of routes, which is an
> > > > important
> > > > > > feature for our users.
> > > > > >
> > > > > > Another case:
> > > > > >
> > > > > > the `service` or `upstream` object can be bound in `route`, when
> we
> > > > need
> > > > > to
> > > > > > delete the `service`
> > > > > > or `upstream` object, we need to confirm there is no route using
> > it.
> > > if
> > > > > we
> > > > > > are using RDS, its easy for
> > > > > > this case.
> > > > > >
> > > > > > If we only use etcd, it is hard to support this case too.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
> > > > > > >
> > > > > > > > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org>
> > > wrote:
> > > > > > > >
> > > > > > > > > I haven't followed APISIX Dashboard until I saw this email.
> > > > > > > > > In my opinion, APISIX Dashboard should not rely on storage
> > > other
> > > > > than
> > > > > > > > ETCD
> > > > > > > > > or even ManagerAPI, which will increase the overall
> > complexity.
> > > > > > > > >
> > > > > > > >
> > > > > > > > `manager api` relies on relational databases, because some
> data
> > > > needs
> > > > > > to
> > > > > > > > support search and paging, etcd cannot support such features.
> > > > > > > >
> > > > > > > > For the `route.name` field, it is necessary to ensure that
> it
> > is
> > > > > > unique
> > > > > > > > among all routes, which cannot be implemented in etcd.
> > > > > > > >
> > > > > > > >
> > > > > > > > > This is really not a good idea. Dashboard should be kept
> > simple
> > > > and
> > > > > > > easy
> > > > > > > > to
> > > > > > > > > deploy, not another huge application.
> > > > > > > > >
> > > > > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四
> > > > 上午10:05写道:
> > > > > > > > >
> > > > > > > > > > HI,
> > > > > > > > > >
> > > > > > > > > > Last I looked mysql was also GPL which is not compatible
> > with
> > > > the
> > > > > > > > Apache
> > > > > > > > > > License. I assume this is an optional dependancy?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Justin
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > *MembPhis*
> > > > > > > > My GitHub: https://github.com/membphis
> > > > > > > > Apache APISIX: https://github.com/apache/incubator-apisix
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > *MembPhis*
> > > > > > My GitHub: https://github.com/membphis
> > > > > > Apache APISIX: https://github.com/apache/incubator-apisix
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
> --
>
> *MembPhis*
> My GitHub: https://github.com/membphis
> Apache APISIX: https://github.com/apache/incubator-apisix
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by YuanSheng Wang <me...@apache.org>.
On Thu, Aug 20, 2020 at 5:33 PM Ming Wen <we...@apache.org> wrote:

> That will be great if we can remove RDS.
>

yes, I love this way too.


>
> Thanks,
> Ming Wen, Apache APISIX & Apache SkyWalking
> Twitter: _WenMing
>
>
> vincixu <vi...@apache.org> 于2020年8月20日周四 下午4:30写道:
>
> > Relational database is not the only way.k8s implement all features(RBAC,
> > ABAC, cascade) we need but no relational databases.
> > Keep configuration in two places, it will bring a hard problem: how to
> > keep consistency.A problem that engineers have struggled for many years
> but
> > there has been no easy solution until now,I think it’s not worth
> > introducing such a problem to implement the mentioned features.
> > ( I also have never seen a project using etcd and relational storage at
> the
> > same time to store the same data, )
> >
> > wei jin <kv...@apache.org> 于2020年8月20日周四 下午4:04写道:
> >
> > > We need a relational database to do some function extensions to meet
> some
> > > practical needs, such as authority management. But we also hope to keep
> > > apisix sufficiently convergent and focus on a high-performance gateway.
> > > APISIX should not be affected by external function extensions.
> > >
> > > vincixu <vi...@apache.org> 于2020年8月20日周四 下午3:53写道:
> > >
> > > > I don't think a lot of routes is a good practice.
> > > > But we can discuss how many routes APISIX plans to support. There is
> no
> > > > pressure on the front-end sorting of records less than 50,000
> > records(Our
> > > > team way).
> > > > If we plan to support large rotes query,  paging and so on, we can
> load
> > > the
> > > > data into memory, and then simulate query and paging in the backend
> > > instead
> > > > of depend on another storage (the k8s apiserver way),
> > > > if we really need a storage that is easy to query, why not directly
> use
> > > it
> > > > as the configuration library of APISIX? Just like tyk did( using
> mongo
> > as
> > > > configuration storage).
> > > > It is very dangerous and inelegant to keep duplicate configuration
> even
> > > in
> > > > different storage media.
> > > >
> > > > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 下午12:30写道:
> > > >
> > > > > On Thu, Aug 20, 2020 at 11:42 AM vincixu <vi...@apache.org> wrote:
> > > > >
> > > > > > >  support search and paging, etcd cannot support such features.
> > > > > > back-end search and paging is unnecessary, we can implement it in
> > the
> > > > > > front-end, because users will not have too many configurations.
> > > > > >
> > > > >
> > > > > > because users will not have too many configurations
> > > > >
> > > > > Apache APISIX can support a large number of routes, which is an
> > > important
> > > > > feature for our users.
> > > > >
> > > > > Another case:
> > > > >
> > > > > the `service` or `upstream` object can be bound in `route`, when we
> > > need
> > > > to
> > > > > delete the `service`
> > > > > or `upstream` object, we need to confirm there is no route using
> it.
> > if
> > > > we
> > > > > are using RDS, its easy for
> > > > > this case.
> > > > >
> > > > > If we only use etcd, it is hard to support this case too.
> > > > >
> > > > >
> > > > > >
> > > > > > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
> > > > > >
> > > > > > > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org>
> > wrote:
> > > > > > >
> > > > > > > > I haven't followed APISIX Dashboard until I saw this email.
> > > > > > > > In my opinion, APISIX Dashboard should not rely on storage
> > other
> > > > than
> > > > > > > ETCD
> > > > > > > > or even ManagerAPI, which will increase the overall
> complexity.
> > > > > > > >
> > > > > > >
> > > > > > > `manager api` relies on relational databases, because some data
> > > needs
> > > > > to
> > > > > > > support search and paging, etcd cannot support such features.
> > > > > > >
> > > > > > > For the `route.name` field, it is necessary to ensure that it
> is
> > > > > unique
> > > > > > > among all routes, which cannot be implemented in etcd.
> > > > > > >
> > > > > > >
> > > > > > > > This is really not a good idea. Dashboard should be kept
> simple
> > > and
> > > > > > easy
> > > > > > > to
> > > > > > > > deploy, not another huge application.
> > > > > > > >
> > > > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四
> > > 上午10:05写道:
> > > > > > > >
> > > > > > > > > HI,
> > > > > > > > >
> > > > > > > > > Last I looked mysql was also GPL which is not compatible
> with
> > > the
> > > > > > > Apache
> > > > > > > > > License. I assume this is an optional dependancy?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Justin
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > *MembPhis*
> > > > > > > My GitHub: https://github.com/membphis
> > > > > > > Apache APISIX: https://github.com/apache/incubator-apisix
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > *MembPhis*
> > > > > My GitHub: https://github.com/membphis
> > > > > Apache APISIX: https://github.com/apache/incubator-apisix
> > > > >
> > > >
> > >
> >
>


-- 

*MembPhis*
My GitHub: https://github.com/membphis
Apache APISIX: https://github.com/apache/incubator-apisix

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by Ming Wen <we...@apache.org>.
That will be great if we can remove RDS.

Thanks,
Ming Wen, Apache APISIX & Apache SkyWalking
Twitter: _WenMing


vincixu <vi...@apache.org> 于2020年8月20日周四 下午4:30写道:

> Relational database is not the only way.k8s implement all features(RBAC,
> ABAC, cascade) we need but no relational databases.
> Keep configuration in two places, it will bring a hard problem: how to
> keep consistency.A problem that engineers have struggled for many years but
> there has been no easy solution until now,I think it’s not worth
> introducing such a problem to implement the mentioned features.
> ( I also have never seen a project using etcd and relational storage at the
> same time to store the same data, )
>
> wei jin <kv...@apache.org> 于2020年8月20日周四 下午4:04写道:
>
> > We need a relational database to do some function extensions to meet some
> > practical needs, such as authority management. But we also hope to keep
> > apisix sufficiently convergent and focus on a high-performance gateway.
> > APISIX should not be affected by external function extensions.
> >
> > vincixu <vi...@apache.org> 于2020年8月20日周四 下午3:53写道:
> >
> > > I don't think a lot of routes is a good practice.
> > > But we can discuss how many routes APISIX plans to support. There is no
> > > pressure on the front-end sorting of records less than 50,000
> records(Our
> > > team way).
> > > If we plan to support large rotes query,  paging and so on, we can load
> > the
> > > data into memory, and then simulate query and paging in the backend
> > instead
> > > of depend on another storage (the k8s apiserver way),
> > > if we really need a storage that is easy to query, why not directly use
> > it
> > > as the configuration library of APISIX? Just like tyk did( using mongo
> as
> > > configuration storage).
> > > It is very dangerous and inelegant to keep duplicate configuration even
> > in
> > > different storage media.
> > >
> > > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 下午12:30写道:
> > >
> > > > On Thu, Aug 20, 2020 at 11:42 AM vincixu <vi...@apache.org> wrote:
> > > >
> > > > > >  support search and paging, etcd cannot support such features.
> > > > > back-end search and paging is unnecessary, we can implement it in
> the
> > > > > front-end, because users will not have too many configurations.
> > > > >
> > > >
> > > > > because users will not have too many configurations
> > > >
> > > > Apache APISIX can support a large number of routes, which is an
> > important
> > > > feature for our users.
> > > >
> > > > Another case:
> > > >
> > > > the `service` or `upstream` object can be bound in `route`, when we
> > need
> > > to
> > > > delete the `service`
> > > > or `upstream` object, we need to confirm there is no route using it.
> if
> > > we
> > > > are using RDS, its easy for
> > > > this case.
> > > >
> > > > If we only use etcd, it is hard to support this case too.
> > > >
> > > >
> > > > >
> > > > > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
> > > > >
> > > > > > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org>
> wrote:
> > > > > >
> > > > > > > I haven't followed APISIX Dashboard until I saw this email.
> > > > > > > In my opinion, APISIX Dashboard should not rely on storage
> other
> > > than
> > > > > > ETCD
> > > > > > > or even ManagerAPI, which will increase the overall complexity.
> > > > > > >
> > > > > >
> > > > > > `manager api` relies on relational databases, because some data
> > needs
> > > > to
> > > > > > support search and paging, etcd cannot support such features.
> > > > > >
> > > > > > For the `route.name` field, it is necessary to ensure that it is
> > > > unique
> > > > > > among all routes, which cannot be implemented in etcd.
> > > > > >
> > > > > >
> > > > > > > This is really not a good idea. Dashboard should be kept simple
> > and
> > > > > easy
> > > > > > to
> > > > > > > deploy, not another huge application.
> > > > > > >
> > > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四
> > 上午10:05写道:
> > > > > > >
> > > > > > > > HI,
> > > > > > > >
> > > > > > > > Last I looked mysql was also GPL which is not compatible with
> > the
> > > > > > Apache
> > > > > > > > License. I assume this is an optional dependancy?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Justin
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > *MembPhis*
> > > > > > My GitHub: https://github.com/membphis
> > > > > > Apache APISIX: https://github.com/apache/incubator-apisix
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > *MembPhis*
> > > > My GitHub: https://github.com/membphis
> > > > Apache APISIX: https://github.com/apache/incubator-apisix
> > > >
> > >
> >
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by vincixu <vi...@apache.org>.
Relational database is not the only way.k8s implement all features(RBAC,
ABAC, cascade) we need but no relational databases.
Keep configuration in two places, it will bring a hard problem: how to
keep consistency.A problem that engineers have struggled for many years but
there has been no easy solution until now,I think it’s not worth
introducing such a problem to implement the mentioned features.
( I also have never seen a project using etcd and relational storage at the
same time to store the same data, )

wei jin <kv...@apache.org> 于2020年8月20日周四 下午4:04写道:

> We need a relational database to do some function extensions to meet some
> practical needs, such as authority management. But we also hope to keep
> apisix sufficiently convergent and focus on a high-performance gateway.
> APISIX should not be affected by external function extensions.
>
> vincixu <vi...@apache.org> 于2020年8月20日周四 下午3:53写道:
>
> > I don't think a lot of routes is a good practice.
> > But we can discuss how many routes APISIX plans to support. There is no
> > pressure on the front-end sorting of records less than 50,000 records(Our
> > team way).
> > If we plan to support large rotes query,  paging and so on, we can load
> the
> > data into memory, and then simulate query and paging in the backend
> instead
> > of depend on another storage (the k8s apiserver way),
> > if we really need a storage that is easy to query, why not directly use
> it
> > as the configuration library of APISIX? Just like tyk did( using mongo as
> > configuration storage).
> > It is very dangerous and inelegant to keep duplicate configuration even
> in
> > different storage media.
> >
> > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 下午12:30写道:
> >
> > > On Thu, Aug 20, 2020 at 11:42 AM vincixu <vi...@apache.org> wrote:
> > >
> > > > >  support search and paging, etcd cannot support such features.
> > > > back-end search and paging is unnecessary, we can implement it in the
> > > > front-end, because users will not have too many configurations.
> > > >
> > >
> > > > because users will not have too many configurations
> > >
> > > Apache APISIX can support a large number of routes, which is an
> important
> > > feature for our users.
> > >
> > > Another case:
> > >
> > > the `service` or `upstream` object can be bound in `route`, when we
> need
> > to
> > > delete the `service`
> > > or `upstream` object, we need to confirm there is no route using it. if
> > we
> > > are using RDS, its easy for
> > > this case.
> > >
> > > If we only use etcd, it is hard to support this case too.
> > >
> > >
> > > >
> > > > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
> > > >
> > > > > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:
> > > > >
> > > > > > I haven't followed APISIX Dashboard until I saw this email.
> > > > > > In my opinion, APISIX Dashboard should not rely on storage other
> > than
> > > > > ETCD
> > > > > > or even ManagerAPI, which will increase the overall complexity.
> > > > > >
> > > > >
> > > > > `manager api` relies on relational databases, because some data
> needs
> > > to
> > > > > support search and paging, etcd cannot support such features.
> > > > >
> > > > > For the `route.name` field, it is necessary to ensure that it is
> > > unique
> > > > > among all routes, which cannot be implemented in etcd.
> > > > >
> > > > >
> > > > > > This is really not a good idea. Dashboard should be kept simple
> and
> > > > easy
> > > > > to
> > > > > > deploy, not another huge application.
> > > > > >
> > > > > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四
> 上午10:05写道:
> > > > > >
> > > > > > > HI,
> > > > > > >
> > > > > > > Last I looked mysql was also GPL which is not compatible with
> the
> > > > > Apache
> > > > > > > License. I assume this is an optional dependancy?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Justin
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > *MembPhis*
> > > > > My GitHub: https://github.com/membphis
> > > > > Apache APISIX: https://github.com/apache/incubator-apisix
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > *MembPhis*
> > > My GitHub: https://github.com/membphis
> > > Apache APISIX: https://github.com/apache/incubator-apisix
> > >
> >
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by wei jin <kv...@apache.org>.
We need a relational database to do some function extensions to meet some
practical needs, such as authority management. But we also hope to keep
apisix sufficiently convergent and focus on a high-performance gateway.
APISIX should not be affected by external function extensions.

vincixu <vi...@apache.org> 于2020年8月20日周四 下午3:53写道:

> I don't think a lot of routes is a good practice.
> But we can discuss how many routes APISIX plans to support. There is no
> pressure on the front-end sorting of records less than 50,000 records(Our
> team way).
> If we plan to support large rotes query,  paging and so on, we can load the
> data into memory, and then simulate query and paging in the backend instead
> of depend on another storage (the k8s apiserver way),
> if we really need a storage that is easy to query, why not directly use it
> as the configuration library of APISIX? Just like tyk did( using mongo as
> configuration storage).
> It is very dangerous and inelegant to keep duplicate configuration even in
> different storage media.
>
> YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 下午12:30写道:
>
> > On Thu, Aug 20, 2020 at 11:42 AM vincixu <vi...@apache.org> wrote:
> >
> > > >  support search and paging, etcd cannot support such features.
> > > back-end search and paging is unnecessary, we can implement it in the
> > > front-end, because users will not have too many configurations.
> > >
> >
> > > because users will not have too many configurations
> >
> > Apache APISIX can support a large number of routes, which is an important
> > feature for our users.
> >
> > Another case:
> >
> > the `service` or `upstream` object can be bound in `route`, when we need
> to
> > delete the `service`
> > or `upstream` object, we need to confirm there is no route using it. if
> we
> > are using RDS, its easy for
> > this case.
> >
> > If we only use etcd, it is hard to support this case too.
> >
> >
> > >
> > > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
> > >
> > > > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:
> > > >
> > > > > I haven't followed APISIX Dashboard until I saw this email.
> > > > > In my opinion, APISIX Dashboard should not rely on storage other
> than
> > > > ETCD
> > > > > or even ManagerAPI, which will increase the overall complexity.
> > > > >
> > > >
> > > > `manager api` relies on relational databases, because some data needs
> > to
> > > > support search and paging, etcd cannot support such features.
> > > >
> > > > For the `route.name` field, it is necessary to ensure that it is
> > unique
> > > > among all routes, which cannot be implemented in etcd.
> > > >
> > > >
> > > > > This is really not a good idea. Dashboard should be kept simple and
> > > easy
> > > > to
> > > > > deploy, not another huge application.
> > > > >
> > > > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:
> > > > >
> > > > > > HI,
> > > > > >
> > > > > > Last I looked mysql was also GPL which is not compatible with the
> > > > Apache
> > > > > > License. I assume this is an optional dependancy?
> > > > > >
> > > > > > Thanks,
> > > > > > Justin
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > *MembPhis*
> > > > My GitHub: https://github.com/membphis
> > > > Apache APISIX: https://github.com/apache/incubator-apisix
> > > >
> > >
> >
> >
> > --
> >
> > *MembPhis*
> > My GitHub: https://github.com/membphis
> > Apache APISIX: https://github.com/apache/incubator-apisix
> >
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by vincixu <vi...@apache.org>.
I don't think a lot of routes is a good practice.
But we can discuss how many routes APISIX plans to support. There is no
pressure on the front-end sorting of records less than 50,000 records(Our
team way).
If we plan to support large rotes query,  paging and so on, we can load the
data into memory, and then simulate query and paging in the backend instead
of depend on another storage (the k8s apiserver way),
if we really need a storage that is easy to query, why not directly use it
as the configuration library of APISIX? Just like tyk did( using mongo as
configuration storage).
It is very dangerous and inelegant to keep duplicate configuration even in
different storage media.

YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 下午12:30写道:

> On Thu, Aug 20, 2020 at 11:42 AM vincixu <vi...@apache.org> wrote:
>
> > >  support search and paging, etcd cannot support such features.
> > back-end search and paging is unnecessary, we can implement it in the
> > front-end, because users will not have too many configurations.
> >
>
> > because users will not have too many configurations
>
> Apache APISIX can support a large number of routes, which is an important
> feature for our users.
>
> Another case:
>
> the `service` or `upstream` object can be bound in `route`, when we need to
> delete the `service`
> or `upstream` object, we need to confirm there is no route using it. if we
> are using RDS, its easy for
> this case.
>
> If we only use etcd, it is hard to support this case too.
>
>
> >
> > YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
> >
> > > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:
> > >
> > > > I haven't followed APISIX Dashboard until I saw this email.
> > > > In my opinion, APISIX Dashboard should not rely on storage other than
> > > ETCD
> > > > or even ManagerAPI, which will increase the overall complexity.
> > > >
> > >
> > > `manager api` relies on relational databases, because some data needs
> to
> > > support search and paging, etcd cannot support such features.
> > >
> > > For the `route.name` field, it is necessary to ensure that it is
> unique
> > > among all routes, which cannot be implemented in etcd.
> > >
> > >
> > > > This is really not a good idea. Dashboard should be kept simple and
> > easy
> > > to
> > > > deploy, not another huge application.
> > > >
> > > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:
> > > >
> > > > > HI,
> > > > >
> > > > > Last I looked mysql was also GPL which is not compatible with the
> > > Apache
> > > > > License. I assume this is an optional dependancy?
> > > > >
> > > > > Thanks,
> > > > > Justin
> > > >
> > >
> > >
> > > --
> > >
> > > *MembPhis*
> > > My GitHub: https://github.com/membphis
> > > Apache APISIX: https://github.com/apache/incubator-apisix
> > >
> >
>
>
> --
>
> *MembPhis*
> My GitHub: https://github.com/membphis
> Apache APISIX: https://github.com/apache/incubator-apisix
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by YuanSheng Wang <me...@apache.org>.
On Thu, Aug 20, 2020 at 11:42 AM vincixu <vi...@apache.org> wrote:

> >  support search and paging, etcd cannot support such features.
> back-end search and paging is unnecessary, we can implement it in the
> front-end, because users will not have too many configurations.
>

> because users will not have too many configurations

Apache APISIX can support a large number of routes, which is an important
feature for our users.

Another case:

the `service` or `upstream` object can be bound in `route`, when we need to
delete the `service`
or `upstream` object, we need to confirm there is no route using it. if we
are using RDS, its easy for
this case.

If we only use etcd, it is hard to support this case too.


>
> YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
>
> > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:
> >
> > > I haven't followed APISIX Dashboard until I saw this email.
> > > In my opinion, APISIX Dashboard should not rely on storage other than
> > ETCD
> > > or even ManagerAPI, which will increase the overall complexity.
> > >
> >
> > `manager api` relies on relational databases, because some data needs to
> > support search and paging, etcd cannot support such features.
> >
> > For the `route.name` field, it is necessary to ensure that it is unique
> > among all routes, which cannot be implemented in etcd.
> >
> >
> > > This is really not a good idea. Dashboard should be kept simple and
> easy
> > to
> > > deploy, not another huge application.
> > >
> > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:
> > >
> > > > HI,
> > > >
> > > > Last I looked mysql was also GPL which is not compatible with the
> > Apache
> > > > License. I assume this is an optional dependancy?
> > > >
> > > > Thanks,
> > > > Justin
> > >
> >
> >
> > --
> >
> > *MembPhis*
> > My GitHub: https://github.com/membphis
> > Apache APISIX: https://github.com/apache/incubator-apisix
> >
>


-- 

*MembPhis*
My GitHub: https://github.com/membphis
Apache APISIX: https://github.com/apache/incubator-apisix

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by Ming Wen <we...@apache.org>.
etcd is used for the configuration of Apache APISIX.
If the management data (such as search, tag, status, etc.) is mixed, it
will affect the stability of the gateway.

Thanks,
Ming Wen, Apache APISIX & Apache SkyWalking
Twitter: _WenMing


vincixu <vi...@apache.org> 于2020年8月20日周四 上午11:42写道:

> >  support search and paging, etcd cannot support such features.
> back-end search and paging is unnecessary, we can implement it in the
> front-end, because users will not have too many configurations.
>
> YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:
>
> > On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:
> >
> > > I haven't followed APISIX Dashboard until I saw this email.
> > > In my opinion, APISIX Dashboard should not rely on storage other than
> > ETCD
> > > or even ManagerAPI, which will increase the overall complexity.
> > >
> >
> > `manager api` relies on relational databases, because some data needs to
> > support search and paging, etcd cannot support such features.
> >
> > For the `route.name` field, it is necessary to ensure that it is unique
> > among all routes, which cannot be implemented in etcd.
> >
> >
> > > This is really not a good idea. Dashboard should be kept simple and
> easy
> > to
> > > deploy, not another huge application.
> > >
> > > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:
> > >
> > > > HI,
> > > >
> > > > Last I looked mysql was also GPL which is not compatible with the
> > Apache
> > > > License. I assume this is an optional dependancy?
> > > >
> > > > Thanks,
> > > > Justin
> > >
> >
> >
> > --
> >
> > *MembPhis*
> > My GitHub: https://github.com/membphis
> > Apache APISIX: https://github.com/apache/incubator-apisix
> >
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by vincixu <vi...@apache.org>.
>  support search and paging, etcd cannot support such features.
back-end search and paging is unnecessary, we can implement it in the
front-end, because users will not have too many configurations.

YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:

> On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:
>
> > I haven't followed APISIX Dashboard until I saw this email.
> > In my opinion, APISIX Dashboard should not rely on storage other than
> ETCD
> > or even ManagerAPI, which will increase the overall complexity.
> >
>
> `manager api` relies on relational databases, because some data needs to
> support search and paging, etcd cannot support such features.
>
> For the `route.name` field, it is necessary to ensure that it is unique
> among all routes, which cannot be implemented in etcd.
>
>
> > This is really not a good idea. Dashboard should be kept simple and easy
> to
> > deploy, not another huge application.
> >
> > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:
> >
> > > HI,
> > >
> > > Last I looked mysql was also GPL which is not compatible with the
> Apache
> > > License. I assume this is an optional dependancy?
> > >
> > > Thanks,
> > > Justin
> >
>
>
> --
>
> *MembPhis*
> My GitHub: https://github.com/membphis
> Apache APISIX: https://github.com/apache/incubator-apisix
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by Sheng Wu <wu...@gmail.com>.
You can't depend on MySQL, because of the license.
MySQL could be optional storage only.

Sheng Wu 吴晟
Twitter, wusheng1108


YuanSheng Wang <me...@apache.org> 于2020年8月20日周四 上午11:29写道:

> On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:
>
> > I haven't followed APISIX Dashboard until I saw this email.
> > In my opinion, APISIX Dashboard should not rely on storage other than
> ETCD
> > or even ManagerAPI, which will increase the overall complexity.
> >
>
> `manager api` relies on relational databases, because some data needs to
> support search and paging, etcd cannot support such features.
>
> For the `route.name` field, it is necessary to ensure that it is unique
> among all routes, which cannot be implemented in etcd.
>
>
> > This is really not a good idea. Dashboard should be kept simple and easy
> to
> > deploy, not another huge application.
> >
> > Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:
> >
> > > HI,
> > >
> > > Last I looked mysql was also GPL which is not compatible with the
> Apache
> > > License. I assume this is an optional dependancy?
> > >
> > > Thanks,
> > > Justin
> >
>
>
> --
>
> *MembPhis*
> My GitHub: https://github.com/membphis
> Apache APISIX: https://github.com/apache/incubator-apisix
>

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by YuanSheng Wang <me...@apache.org>.
On Thu, Aug 20, 2020 at 10:57 AM vincixu <vi...@apache.org> wrote:

> I haven't followed APISIX Dashboard until I saw this email.
> In my opinion, APISIX Dashboard should not rely on storage other than ETCD
> or even ManagerAPI, which will increase the overall complexity.
>

`manager api` relies on relational databases, because some data needs to
support search and paging, etcd cannot support such features.

For the `route.name` field, it is necessary to ensure that it is unique
among all routes, which cannot be implemented in etcd.


> This is really not a good idea. Dashboard should be kept simple and easy to
> deploy, not another huge application.
>
> Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:
>
> > HI,
> >
> > Last I looked mysql was also GPL which is not compatible with the Apache
> > License. I assume this is an optional dependancy?
> >
> > Thanks,
> > Justin
>


-- 

*MembPhis*
My GitHub: https://github.com/membphis
Apache APISIX: https://github.com/apache/incubator-apisix

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by vincixu <vi...@apache.org>.
I haven't followed APISIX Dashboard until I saw this email.
In my opinion, APISIX Dashboard should not rely on storage other than ETCD
or even ManagerAPI, which will increase the overall complexity.
This is really not a good idea. Dashboard should be kept simple and easy to
deploy, not another huge application.

Justin Mclean <ju...@classsoftware.com> 于2020年8月20日周四 上午10:05写道:

> HI,
>
> Last I looked mysql was also GPL which is not compatible with the Apache
> License. I assume this is an optional dependancy?
>
> Thanks,
> Justin

Re: [DISCUSS]Modify the manager-api DB from mysql to sqllite

Posted by Justin Mclean <ju...@classsoftware.com>.
HI,

Last I looked mysql was also GPL which is not compatible with the Apache License. I assume this is an optional dependancy?

Thanks,
Justin