You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@servicecomb.apache.org by wjm wjm <zz...@gmail.com> on 2018/06/28 08:46:24 UTC
about protobuf codec switch from protoStuff to jackson
protoStuff support all protobuf v3 features and very fast
but protoStuff:
1.must based on java class, can not construct from idl, this is not good in
edge.
2.can not support customize generic class
jackson protobuf plugin is good in other feature, but :
only support protobuf v2 features
some scenes slower than protoStuff, and others faster, it's not a big
problem, we can optimize it in the future
if we extend jackson protobuf plugin to support protobuf v3 features
we must copy and modify source code, because there is no any plugable
design for v3 feature
it's "copy and modify", we will have too many problems
what your suggestion to resolve the problem？
Re: about protobuf codec switch from protoStuff to jackson
Posted by wjm wjm <zz...@gmail.com>.
i don't want to dynamic create class anymore, that cause code too complex
2018-06-29 11:02 GMT+08:00 bismy <bi...@qq.com>:
> More comments about "I think we can still use protoStuff to solve the
> problem"
> 1. We develop REST for most priority
> 2. We make highway compatible to older version, but not solve the problem
> of loading many classes
> 3. In future we can develope a new "duplexing transport" not using
> protobuffer, maybe json, for example. Encoding/Coding is not very important
> for performance. Or when jackson can implement v3 or protostuff can support
> idl, we can change it.
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "bismy"<bi...@qq.com>;
> 发送时间: 2018年6月29日(星期五) 上午10:53
> 收件人: "dev"<de...@servicecomb.apache.org>;
>
> 主题: 回复：about protobuf codec switch from protoStuff to jackson
>
>
>
> There may be some background information not given very clearly.
>
>
> If we are addressing the problem that:
>
>
> 1. Edge service dispatch requests will decode requests and encode to
> Invocation, this is not flexible and need compile different versions of
> schema to class.
> And 2. support more flexible user interfaces like custom headers not
> included in schema
>
>
> This problems come from support of Highway (which using protobuffer), but
> using REST(json based), we will not have this problem.
>
>
> Now:
>
>
> consumer side: (pojo/springmvc client) -> Invocation(args) ->
> Transport(args -> json, rest; args -> protobufer, highway)
> provideer side: Transport(json -> args, rest; protobuffer -> args,
> highway) -> Invocation(args) -> (pojo/springmvc/jaxrs)
>
>
> providers, handlers and transports can see schema info too.
>
>
> We want to change to:
> consumer side: (pojo/springmvc client) -> Invocation(HttpServletRequestExt,
> headers, body, queries. args for compatible and translate on demand) ->
> Transport(HttpServletRequestExt, rest; HttpServletRequestExt -> args ->
> protobufer, highway)
> provideer side: Transport(HttpServletRequestExt ->
> HttpServletRequestExt,, rest; protobuffer -> args -> HttpServletRequestExt,
> highway) -> Invocation(HttpServletRequestExt) -> (pojo/springmvc/jaxrs)
>
>
>
> if we use protoStuff, for highway should be:
> protobuffer -> args -> HttpServletRequestExt
>
>
> and jackson:
> protobuffer -> idl -> HttpServletRequestExt
>
>
> I think we can still use protoStuff to solve the problem. The above
> understanding may be not quite correct. If I missed something?
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "wjm wjm"<zz...@gmail.com>;
> 发送时间: 2018年6月28日(星期四) 下午4:46
> 收件人: "dev"<de...@servicecomb.apache.org>;
>
> 主题: about protobuf codec switch from protoStuff to jackson
>
>
>
> protoStuff support all protobuf v3 features and very fast
> but protoStuff:
> 1.must based on java class, can not construct from idl, this is not good in
> edge.
> 2.can not support customize generic class
>
> jackson protobuf plugin is good in other feature, but :
> only support protobuf v2 features
> some scenes slower than protoStuff, and others faster, it's not a big
> problem, we can optimize it in the future
>
> if we extend jackson protobuf plugin to support protobuf v3 features
> we must copy and modify source code, because there is no any plugable
> design for v3 feature
> it's "copy and modify", we will have too many problems
>
> what your suggestion to resolve the problem？
>
回复：about protobuf codec switch from protoStuff to jackson
Posted by bismy <bi...@qq.com>.
More comments about "I think we can still use protoStuff to solve the problem"
1. We develop REST for most priority
2. We make highway compatible to older version, but not solve the problem of loading many classes
3. In future we can develope a new "duplexing transport" not using protobuffer, maybe json, for example. Encoding/Coding is not very important for performance. Or when jackson can implement v3 or protostuff can support idl, we can change it.
------------------ 原始邮件 ------------------
发件人: "bismy"<bi...@qq.com>;
发送时间: 2018年6月29日(星期五) 上午10:53
收件人: "dev"<de...@servicecomb.apache.org>;
主题: 回复：about protobuf codec switch from protoStuff to jackson
There may be some background information not given very clearly.
If we are addressing the problem that:
1. Edge service dispatch requests will decode requests and encode to Invocation, this is not flexible and need compile different versions of schema to class.
And 2. support more flexible user interfaces like custom headers not included in schema
This problems come from support of Highway (which using protobuffer), but using REST(json based), we will not have this problem.
Now:
consumer side: (pojo/springmvc client) -> Invocation(args) -> Transport(args -> json, rest; args -> protobufer, highway)
provideer side: Transport(json -> args, rest; protobuffer -> args, highway) -> Invocation(args) -> (pojo/springmvc/jaxrs)
providers, handlers and transports can see schema info too.
We want to change to:
consumer side: (pojo/springmvc client) -> Invocation(HttpServletRequestExt, headers, body, queries. args for compatible and translate on demand) -> Transport(HttpServletRequestExt, rest; HttpServletRequestExt -> args -> protobufer, highway)
provideer side: Transport(HttpServletRequestExt -> HttpServletRequestExt,, rest; protobuffer -> args -> HttpServletRequestExt, highway) -> Invocation(HttpServletRequestExt) -> (pojo/springmvc/jaxrs)
if we use protoStuff, for highway should be:
protobuffer -> args -> HttpServletRequestExt
and jackson:
protobuffer -> idl -> HttpServletRequestExt
I think we can still use protoStuff to solve the problem. The above understanding may be not quite correct. If I missed something?
------------------ 原始邮件 ------------------
发件人: "wjm wjm"<zz...@gmail.com>;
发送时间: 2018年6月28日(星期四) 下午4:46
收件人: "dev"<de...@servicecomb.apache.org>;
主题: about protobuf codec switch from protoStuff to jackson
protoStuff support all protobuf v3 features and very fast
but protoStuff:
1.must based on java class, can not construct from idl, this is not good in
edge.
2.can not support customize generic class
jackson protobuf plugin is good in other feature, but :
only support protobuf v2 features
some scenes slower than protoStuff, and others faster, it's not a big
problem, we can optimize it in the future
if we extend jackson protobuf plugin to support protobuf v3 features
we must copy and modify source code, because there is no any plugable
design for v3 feature
it's "copy and modify", we will have too many problems
what your suggestion to resolve the problem？
Re: about protobuf codec switch from protoStuff to jackson
Posted by wjm wjm <zz...@gmail.com>.
1.protoStuff must work with class
edge not meet the condition
2.protobuf message must be a struct, that means
primitive/String/enum/map/list/array/args can not used to encode/decode
with protobuf, must wrap them to a class
3.invocation from springmvc is weak type, can not use protoStuff
2018-06-29 10:53 GMT+08:00 bismy <bi...@qq.com>:
> There may be some background information not given very clearly.
>
>
> If we are addressing the problem that:
>
>
> 1. Edge service dispatch requests will decode requests and encode to
> Invocation, this is not flexible and need compile different versions of
> schema to class.
> And 2. support more flexible user interfaces like custom headers not
> included in schema
>
>
> This problems come from support of Highway (which using protobuffer), but
> using REST(json based), we will not have this problem.
>
>
> Now:
>
>
> consumer side: (pojo/springmvc client) -> Invocation(args) ->
> Transport(args -> json, rest; args -> protobufer, highway)
> provideer side: Transport(json -> args, rest; protobuffer -> args,
> highway) -> Invocation(args) -> (pojo/springmvc/jaxrs)
>
>
> providers, handlers and transports can see schema info too.
>
>
> We want to change to:
> consumer side: (pojo/springmvc client) -> Invocation(HttpServletRequestExt,
> headers, body, queries. args for compatible and translate on demand) ->
> Transport(HttpServletRequestExt, rest; HttpServletRequestExt -> args ->
> protobufer, highway)
> provideer side: Transport(HttpServletRequestExt ->
> HttpServletRequestExt,, rest; protobuffer -> args -> HttpServletRequestExt,
> highway) -> Invocation(HttpServletRequestExt) -> (pojo/springmvc/jaxrs)
>
>
>
> if we use protoStuff, for highway should be:
> protobuffer -> args -> HttpServletRequestExt
>
>
> and jackson:
> protobuffer -> idl -> HttpServletRequestExt
>
>
> I think we can still use protoStuff to solve the problem. The above
> understanding may be not quite correct. If I missed something?
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "wjm wjm"<zz...@gmail.com>;
> 发送时间: 2018年6月28日(星期四) 下午4:46
> 收件人: "dev"<de...@servicecomb.apache.org>;
>
> 主题: about protobuf codec switch from protoStuff to jackson
>
>
>
> protoStuff support all protobuf v3 features and very fast
> but protoStuff:
> 1.must based on java class, can not construct from idl, this is not good in
> edge.
> 2.can not support customize generic class
>
> jackson protobuf plugin is good in other feature, but :
> only support protobuf v2 features
> some scenes slower than protoStuff, and others faster, it's not a big
> problem, we can optimize it in the future
>
> if we extend jackson protobuf plugin to support protobuf v3 features
> we must copy and modify source code, because there is no any plugable
> design for v3 feature
> it's "copy and modify", we will have too many problems
>
> what your suggestion to resolve the problem？
>
回复：about protobuf codec switch from protoStuff to jackson
Posted by bismy <bi...@qq.com>.
There may be some background information not given very clearly.
If we are addressing the problem that:
1. Edge service dispatch requests will decode requests and encode to Invocation, this is not flexible and need compile different versions of schema to class.
And 2. support more flexible user interfaces like custom headers not included in schema
This problems come from support of Highway (which using protobuffer), but using REST(json based), we will not have this problem.
Now:
consumer side: (pojo/springmvc client) -> Invocation(args) -> Transport(args -> json, rest; args -> protobufer, highway)
provideer side: Transport(json -> args, rest; protobuffer -> args, highway) -> Invocation(args) -> (pojo/springmvc/jaxrs)
providers, handlers and transports can see schema info too.
We want to change to:
consumer side: (pojo/springmvc client) -> Invocation(HttpServletRequestExt, headers, body, queries. args for compatible and translate on demand) -> Transport(HttpServletRequestExt, rest; HttpServletRequestExt -> args -> protobufer, highway)
provideer side: Transport(HttpServletRequestExt -> HttpServletRequestExt,, rest; protobuffer -> args -> HttpServletRequestExt, highway) -> Invocation(HttpServletRequestExt) -> (pojo/springmvc/jaxrs)
if we use protoStuff, for highway should be:
protobuffer -> args -> HttpServletRequestExt
and jackson:
protobuffer -> idl -> HttpServletRequestExt
I think we can still use protoStuff to solve the problem. The above understanding may be not quite correct. If I missed something?
------------------ 原始邮件 ------------------
发件人: "wjm wjm"<zz...@gmail.com>;
发送时间: 2018年6月28日(星期四) 下午4:46
收件人: "dev"<de...@servicecomb.apache.org>;
主题: about protobuf codec switch from protoStuff to jackson
protoStuff support all protobuf v3 features and very fast
but protoStuff:
1.must based on java class, can not construct from idl, this is not good in
edge.
2.can not support customize generic class
jackson protobuf plugin is good in other feature, but :
only support protobuf v2 features
some scenes slower than protoStuff, and others faster, it's not a big
problem, we can optimize it in the future
if we extend jackson protobuf plugin to support protobuf v3 features
we must copy and modify source code, because there is no any plugable
design for v3 feature
it's "copy and modify", we will have too many problems
what your suggestion to resolve the problem？
答复: about protobuf codec switch from protoStuff to jackson
Posted by YangYongZheng <34...@qq.com>.
I think we can communicate with protostuff let them know what we need;
If they feel unnecessary, we may extend jackson protobuf plugin in order to support this.
Best Regards!
YangYongZheng
-----邮件原件-----
发件人: wjm wjm [mailto:zzzwjm@gmail.com]
发送时间: 2018年6月28日 16:46
收件人: dev@servicecomb.apache.org
主题: about protobuf codec switch from protoStuff to jackson
protoStuff support all protobuf v3 features and very fast but protoStuff:
1.must based on java class, can not construct from idl, this is not good in edge.
2.can not support customize generic class
jackson protobuf plugin is good in other feature, but :
only support protobuf v2 features
some scenes slower than protoStuff, and others faster, it's not a big problem, we can optimize it in the future
if we extend jackson protobuf plugin to support protobuf v3 features we must copy and modify source code, because there is no any plugable design for v3 feature it's "copy and modify", we will have too many problems
what your suggestion to resolve the problem？
Re: 答复: about protobuf codec switch from protoStuff to jackson
Posted by Willem Jiang <wi...@gmail.com>.
Did you try to proposal the feature request to the protoStuff before?
The suggestion that Zhengyangyong provided could save us lots of
maintenance effort if we can work with the upstream project.
Willem Jiang
Twitter: willemjiang
Weibo: 姜宁willem
On Fri, Jun 29, 2018 at 9:29 AM, wjm wjm <zz...@gmail.com> wrote:
> i think it's impossible that we wait their decision and develop plan......
>
> 2018-06-29 9:00 GMT+08:00 YangYongZheng <34...@qq.com>:
>
> > I think we can communicate with protostuff let them know what we need;
> > If they feel unnecessary, we may extend jackson protobuf plugin in order
> > to support this.
> >
> >
> > Best Regards!
> > YangYongZheng
> >
> > -----邮件原件-----
> > 发件人: wjm wjm [mailto:zzzwjm@gmail.com]
> > 发送时间: 2018年6月28日 16:46
> > 收件人: dev@servicecomb.apache.org
> > 主题: about protobuf codec switch from protoStuff to jackson
> >
> > protoStuff support all protobuf v3 features and very fast but protoStuff:
> > 1.must based on java class, can not construct from idl, this is not good
> > in edge.
> > 2.can not support customize generic class
> >
> > jackson protobuf plugin is good in other feature, but :
> > only support protobuf v2 features
> > some scenes slower than protoStuff, and others faster, it's not a big
> > problem, we can optimize it in the future
> >
> > if we extend jackson protobuf plugin to support protobuf v3 features we
> > must copy and modify source code, because there is no any plugable design
> > for v3 feature it's "copy and modify", we will have too many problems
> >
> > what your suggestion to resolve the problem？
> >
> >
> >
> >
>
Re: 答复: about protobuf codec switch from protoStuff to jackson
Posted by wjm wjm <zz...@gmail.com>.
i think it's impossible that we wait their decision and develop plan......
2018-06-29 9:00 GMT+08:00 YangYongZheng <34...@qq.com>:
> I think we can communicate with protostuff let them know what we need;
> If they feel unnecessary, we may extend jackson protobuf plugin in order
> to support this.
>
>
> Best Regards!
> YangYongZheng
>
> -----邮件原件-----
> 发件人: wjm wjm [mailto:zzzwjm@gmail.com]
> 发送时间: 2018年6月28日 16:46
> 收件人: dev@servicecomb.apache.org
> 主题: about protobuf codec switch from protoStuff to jackson
>
> protoStuff support all protobuf v3 features and very fast but protoStuff:
> 1.must based on java class, can not construct from idl, this is not good
> in edge.
> 2.can not support customize generic class
>
> jackson protobuf plugin is good in other feature, but :
> only support protobuf v2 features
> some scenes slower than protoStuff, and others faster, it's not a big
> problem, we can optimize it in the future
>
> if we extend jackson protobuf plugin to support protobuf v3 features we
> must copy and modify source code, because there is no any plugable design
> for v3 feature it's "copy and modify", we will have too many problems
>
> what your suggestion to resolve the problem？
>
>
>
>