[ANNOUNCE] Open source lua-resty-nonblocking-ffi

[Jinhua Luo <lu...@gmail.com>]

lua-resty-nonblocking-ffi provides an efficient and generic API to do
hybrid programming in openresty with mainstream languages (Go, Python,
Java, etc.).

In fact, this library has been used successfully by several commercial
companies for over two years. My customers come from different
domains, including AI, IOT, Edge computing. Now I plan to open source
it for use by the community.

Features:

* simple but extensible interface, supports any C ABI compliant language
* once and for all, no need to write C/Lua codes to do coupling anymore
* high performance, 3~4 times faster than unix domain socket way
* bridges for python/java, once and for all
* any serialization message format you like

Background:

In openresty land, when you turn to implement some logic, especially
to couple with third-party popular frameworks, it's likely to suck in
awkward: make bricks without straw.

1. C is a low-level language, very little ecosystem, no unified and
rich libraries, and almost all modern frameworks do not support C,
instead, they like Java, Python, Go. For example, when you need to do
grpc to access external services, you must depend on C++ lib, which is
huge and cumbersome.

2. Lua is an embedded and minority programming language, which means
all the powers comes from the host. In openresty, it means all
functionalities come from lua-nginx-modules. Like C, or even worse,
you have to reinvent the wheels via cosocket to do modern networking
stuff. A lot of lua-resty-* were born, but they are almost
semi-finished compared to native lib in other languages. For example,
lua-resty-kafka doesn't support consumer groups, lua-resty-postgres
doesn't support notify and prepared statements, etc. Moreover, most of
those authors of lua-resty-* stop development at some stage because
the lua community is so small and less attractive.

To implement a common function in other mainstream languages, you have
to do a lot of adaptive coding back and forward between nginx and
openresty. For example, http2 requires SNI and viable session reuse,
then you have to patch C codes and openresty and/or nginx. Moreover,
when you turn to accomplish a new job, you need to redo all such
things again! Then, a lot of *-nginx-module born and recompile your
nginx! Painful, right?

May I extend the openresty with modern programming languages (Go,
Python, Java, etc.) and reuse their rich ecosystems directly? That
means, we reserved everything from those mainstream languages and let
them work together with Nginx at ease.

Re: [ANNOUNCE] Open source lua-resty-nonblocking-ffi

[Jinhua Luo <lu...@gmail.com>]

Sorry, it's a bit of a misunderstanding.
It's my mistake, and I misuse the concept "ANNOUNCE" and "DISCUSS".
In fact, I just tried to launch a discussion with the APISIX community.

YuanSheng Wang <me...@apache.org> 于2022年9月20日周二 15:45写道：
>
> Jinhua:
>
> Why are you using ANNOUNCE directly? It is not the thing that is discussed
> and agreed upon in the community.
>
> This will confuse many folks in the community.
>
> You cannot announce results on behalf of the Apache APISIX community.
>
> If you want to do something, we should discuss it first. Then we can do it
> if most of the folks agree with it.
>
>
> On Tue, Sep 20, 2022 at 3:20 PM Ming Wen <we...@apache.org> wrote:
>
> > I didn't understand your email. What are you announcing?
> > Do you want to donate `lua-resty-nonblocking-ffi` to the Apache APISIX
> > project?
> >
> > Thanks,
> > Ming Wen, Apache APISIX PMC Chair
> > Twitter: _WenMing
> >
> >
> > Jinhua Luo <lu...@gmail.com> 于2022年9月20日周二 15:00写道：
> >
> > > lua-resty-nonblocking-ffi provides an efficient and generic API to do
> > > hybrid programming in openresty with mainstream languages (Go, Python,
> > > Java, etc.).
> > >
> > > In fact, this library has been used successfully by several commercial
> > > companies for over two years. My customers come from different
> > > domains, including AI, IOT, Edge computing. Now I plan to open source
> > > it for use by the community.
> > >
> > > Features:
> > >
> > > * simple but extensible interface, supports any C ABI compliant language
> > > * once and for all, no need to write C/Lua codes to do coupling anymore
> > > * high performance, 3~4 times faster than unix domain socket way
> > > * bridges for python/java, once and for all
> > > * any serialization message format you like
> > >
> > > Background:
> > >
> > > In openresty land, when you turn to implement some logic, especially
> > > to couple with third-party popular frameworks, it's likely to suck in
> > > awkward: make bricks without straw.
> > >
> > > 1. C is a low-level language, very little ecosystem, no unified and
> > > rich libraries, and almost all modern frameworks do not support C,
> > > instead, they like Java, Python, Go. For example, when you need to do
> > > grpc to access external services, you must depend on C++ lib, which is
> > > huge and cumbersome.
> > >
> > > 2. Lua is an embedded and minority programming language, which means
> > > all the powers comes from the host. In openresty, it means all
> > > functionalities come from lua-nginx-modules. Like C, or even worse,
> > > you have to reinvent the wheels via cosocket to do modern networking
> > > stuff. A lot of lua-resty-* were born, but they are almost
> > > semi-finished compared to native lib in other languages. For example,
> > > lua-resty-kafka doesn't support consumer groups, lua-resty-postgres
> > > doesn't support notify and prepared statements, etc. Moreover, most of
> > > those authors of lua-resty-* stop development at some stage because
> > > the lua community is so small and less attractive.
> > >
> > > To implement a common function in other mainstream languages, you have
> > > to do a lot of adaptive coding back and forward between nginx and
> > > openresty. For example, http2 requires SNI and viable session reuse,
> > > then you have to patch C codes and openresty and/or nginx. Moreover,
> > > when you turn to accomplish a new job, you need to redo all such
> > > things again! Then, a lot of *-nginx-module born and recompile your
> > > nginx! Painful, right?
> > >
> > > May I extend the openresty with modern programming languages (Go,
> > > Python, Java, etc.) and reuse their rich ecosystems directly? That
> > > means, we reserved everything from those mainstream languages and let
> > > them work together with Nginx at ease.
> > >
> >
>
>
> --
>
> *MembPhis*
> My GitHub: https://github.com/membphis
> Apache APISIX: https://github.com/apache/apisix

Re: [ANNOUNCE] Open source lua-resty-nonblocking-ffi

[YuanSheng Wang <me...@apache.org>]

Jinhua:

Why are you using ANNOUNCE directly? It is not the thing that is discussed
and agreed upon in the community.

This will confuse many folks in the community.

You cannot announce results on behalf of the Apache APISIX community.

If you want to do something, we should discuss it first. Then we can do it
if most of the folks agree with it.


On Tue, Sep 20, 2022 at 3:20 PM Ming Wen <we...@apache.org> wrote:

> I didn't understand your email. What are you announcing?
> Do you want to donate `lua-resty-nonblocking-ffi` to the Apache APISIX
> project?
>
> Thanks,
> Ming Wen, Apache APISIX PMC Chair
> Twitter: _WenMing
>
>
> Jinhua Luo <lu...@gmail.com> 于2022年9月20日周二 15:00写道：
>
> > lua-resty-nonblocking-ffi provides an efficient and generic API to do
> > hybrid programming in openresty with mainstream languages (Go, Python,
> > Java, etc.).
> >
> > In fact, this library has been used successfully by several commercial
> > companies for over two years. My customers come from different
> > domains, including AI, IOT, Edge computing. Now I plan to open source
> > it for use by the community.
> >
> > Features:
> >
> > * simple but extensible interface, supports any C ABI compliant language
> > * once and for all, no need to write C/Lua codes to do coupling anymore
> > * high performance, 3~4 times faster than unix domain socket way
> > * bridges for python/java, once and for all
> > * any serialization message format you like
> >
> > Background:
> >
> > In openresty land, when you turn to implement some logic, especially
> > to couple with third-party popular frameworks, it's likely to suck in
> > awkward: make bricks without straw.
> >
> > 1. C is a low-level language, very little ecosystem, no unified and
> > rich libraries, and almost all modern frameworks do not support C,
> > instead, they like Java, Python, Go. For example, when you need to do
> > grpc to access external services, you must depend on C++ lib, which is
> > huge and cumbersome.
> >
> > 2. Lua is an embedded and minority programming language, which means
> > all the powers comes from the host. In openresty, it means all
> > functionalities come from lua-nginx-modules. Like C, or even worse,
> > you have to reinvent the wheels via cosocket to do modern networking
> > stuff. A lot of lua-resty-* were born, but they are almost
> > semi-finished compared to native lib in other languages. For example,
> > lua-resty-kafka doesn't support consumer groups, lua-resty-postgres
> > doesn't support notify and prepared statements, etc. Moreover, most of
> > those authors of lua-resty-* stop development at some stage because
> > the lua community is so small and less attractive.
> >
> > To implement a common function in other mainstream languages, you have
> > to do a lot of adaptive coding back and forward between nginx and
> > openresty. For example, http2 requires SNI and viable session reuse,
> > then you have to patch C codes and openresty and/or nginx. Moreover,
> > when you turn to accomplish a new job, you need to redo all such
> > things again! Then, a lot of *-nginx-module born and recompile your
> > nginx! Painful, right?
> >
> > May I extend the openresty with modern programming languages (Go,
> > Python, Java, etc.) and reuse their rich ecosystems directly? That
> > means, we reserved everything from those mainstream languages and let
> > them work together with Nginx at ease.
> >
>


-- 

*MembPhis*
My GitHub: https://github.com/membphis
Apache APISIX: https://github.com/apache/apisix

Re: [ANNOUNCE] Open source lua-resty-nonblocking-ffi

[Ming Wen <we...@apache.org>]

I didn't understand your email. What are you announcing?
Do you want to donate `lua-resty-nonblocking-ffi` to the Apache APISIX
project?

Thanks,
Ming Wen, Apache APISIX PMC Chair
Twitter: _WenMing


Jinhua Luo <lu...@gmail.com> 于2022年9月20日周二 15:00写道：

> lua-resty-nonblocking-ffi provides an efficient and generic API to do
> hybrid programming in openresty with mainstream languages (Go, Python,
> Java, etc.).
>
> In fact, this library has been used successfully by several commercial
> companies for over two years. My customers come from different
> domains, including AI, IOT, Edge computing. Now I plan to open source
> it for use by the community.
>
> Features:
>
> * simple but extensible interface, supports any C ABI compliant language
> * once and for all, no need to write C/Lua codes to do coupling anymore
> * high performance, 3~4 times faster than unix domain socket way
> * bridges for python/java, once and for all
> * any serialization message format you like
>
> Background:
>
> In openresty land, when you turn to implement some logic, especially
> to couple with third-party popular frameworks, it's likely to suck in
> awkward: make bricks without straw.
>
> 1. C is a low-level language, very little ecosystem, no unified and
> rich libraries, and almost all modern frameworks do not support C,
> instead, they like Java, Python, Go. For example, when you need to do
> grpc to access external services, you must depend on C++ lib, which is
> huge and cumbersome.
>
> 2. Lua is an embedded and minority programming language, which means
> all the powers comes from the host. In openresty, it means all
> functionalities come from lua-nginx-modules. Like C, or even worse,
> you have to reinvent the wheels via cosocket to do modern networking
> stuff. A lot of lua-resty-* were born, but they are almost
> semi-finished compared to native lib in other languages. For example,
> lua-resty-kafka doesn't support consumer groups, lua-resty-postgres
> doesn't support notify and prepared statements, etc. Moreover, most of
> those authors of lua-resty-* stop development at some stage because
> the lua community is so small and less attractive.
>
> To implement a common function in other mainstream languages, you have
> to do a lot of adaptive coding back and forward between nginx and
> openresty. For example, http2 requires SNI and viable session reuse,
> then you have to patch C codes and openresty and/or nginx. Moreover,
> when you turn to accomplish a new job, you need to redo all such
> things again! Then, a lot of *-nginx-module born and recompile your
> nginx! Painful, right?
>
> May I extend the openresty with modern programming languages (Go,
> Python, Java, etc.) and reuse their rich ecosystems directly? That
> means, we reserved everything from those mainstream languages and let
> them work together with Nginx at ease.
>