Re:Re: [DISCUSS] Optimize model of ColumnMetaData

[KimmKing <ki...@apache.org>]

We have persist meta data of logic schema to center repository like zk. And add a global notification to all proxy/jdbc nodes when a ddl statement executed.


More details also see:
[1] [DISCUSS]MetadataCenter Design 5.x https://github.com/apache/incubator-shardingsphere/issues/4896

At 2020-01-20 22:11:13, "guangyuan wang" <wa...@apache.org> wrote:
>For byte[] value, I think we could find a way to resolve.
>But refresh metadata without affecting running instance, I am not sure.
>Registry Center provides a watching function, does it could be used to
>refresh metadata?
>
>zhaojun <zh...@126.com> 于2020年1月20日周一 下午5:23写道：
>
>> If we store raw column metadata into orchestration, byte[] value should be
>> supported in orchestration.
>> metaData could be serialized into byte array using kryo framework etc
>>
>> Also we should consider about refresh metadata without affecting running
>> instance.
>>
>> ------------------
>> Zhao Jun (cherrylzhao)
>> Apache ShardingSphere & ServiceComb
>>
>> > On Jan 20, 2020, at 5:02 PM, guangyuan wang <wa...@apache.org>
>> wrote:
>> >
>> > What kind of help do you need in the orchestration module?  The
>> > orchestration module provides functions: writing and reading key-value
>> pair.
>> >
>> > Nicholas <th...@gmail.com> 于2020年1月17日周五 下午4:02写道：
>> >
>> >> Hi zhaojun,
>> >>   About ColumnMeta storage, we need ShardingSphere's orchestration
>> module
>> >> to help register column metadata. And at best establish metadata module
>> to
>> >> manage metadata of table, column etc. Field property like primary key
>> etc
>> >> should be put into ColumnMeta to maintain. At last, could you please
>> >> provide the design document of MetaData Model & MetaData service, and
>> take
>> >> distributed scenario into consideration.
>> >>
>> >> Thanks,
>> >> Nicholas
>> >>
>> >> On 2020/01/15 04:26:48, zhaojun <zh...@126.com> wrote:
>> >>> Hi, all
>> >>>
>> >>> I have created a issue[1] for discussing about optimizing metadata
>> model.
>> >>>
>> >>> Currently ShardingSphere metadata was only used by sharding & encrypt
>> >> scenario,
>> >>> master-slave case have to get them from underlying ResultSet meta in
>> >> sharding-proxy.
>> >>> It is better to establish an global MetaData mechanism which provide
>> >>> uniform interface for initialization, read, refreshing instead of
>> >> distinguish with usage scenario
>> >>>
>> >>> I think it would be a big refactor for sharding-jdbc & sharding-proxy.
>> >>> assume that task list is as below
>> >>>
>> >>> 1. investigate the usage of current metadata, clarify the function list
>> >> of metadata (SQLParse, Route,Rewrite etc)
>> >>> 2. design a MetaData Model & MetaData service, at first only consider
>> >> about storing in memory
>> >>> 3. then consider about migrating to zookeeper or other registration
>> >>>
>> >>> [1]: https://github.com/apache/incubator-shardingsphere/issues/3922 <
>> >> https://github.com/apache/incubator-shardingsphere/issues/3922>
>> >>>
>> >>> Any thought?
>> >>>
>> >>> ------------------
>> >>> Zhao Jun (cherrylzhao)
>> >>> Apache ShardingSphere & ServiceComb
>> >>>
>> >>>
>> >>
>>
>>