You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@eventmesh.apache.org by GitBox <gi...@apache.org> on 2022/04/03 09:41:04 UTC
[GitHub] [incubator-eventmesh] jzhou59 opened a new pull request #821: [ISSUE #614] Add schema plugin and its api
jzhou59 opened a new pull request #821:
URL: https://github.com/apache/incubator-eventmesh/pull/821
<!--
### Contribution Checklist
- Name the pull request in the form "[ISSUE #XXXX] Title of the pull request",
where *XXXX* should be replaced by the actual issue number.
Skip *[ISSUE #XXXX]* if there is no associated github issue for this pull request.
- Fill out the template below to describe the changes contributed by the pull request.
That will give reviewers the context they need to do the review.
- Each pull request should address only one issue.
Please do not mix up code from multiple issues.
- Each commit in the pull request should have a meaningful commit message.
- Once all items of the checklist are addressed, remove the above text and this checklist,
leaving only the filled out template below.
(The sections below can be removed for hotfixes of typos)
-->
<!--
(If this PR fixes a GitHub issue, please add `Fixes ISSUE #<XXX>`.)
-->
Fixes ISSUE #614 .
### Motivation
This pr adds a plugin for schema and adds the spi for it.
### Documentation
This pr introduces a new feature schema client.
The general schema design is presented in the repo, but the client is not documented yet.
I will update the schema documentation later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org
[GitHub] [incubator-eventmesh] jzhou59 commented on pull request #821: [ISSUE #614] Add schema plugin and its api
Posted by GitBox <gi...@apache.org>.
jzhou59 commented on pull request #821:
URL: https://github.com/apache/incubator-eventmesh/pull/821#issuecomment-1086895286
> You may need to add `gradle.properties`, this will be used when packaging.
>
> BTY, I think your `SchemaRegistry` design is not great. I suggest the schema just need to considerate how to serialize and deserialize. It doesn't need to take care about how to register schema, this should be implemen at `RegisterCenter` plugin.
Sure, currently it is only ```api``` package and I will add ```gradle.properties``` in implementations.
The registering-related and retrieving-related procedures are indeed better in ```registry```-related plugin and I will modify it later.
However, I am not very clear about what should Eventmesh do with schema.
As far as I am concerned, there are four main concepts of schema: contract, serialization, deserialization, and validity.
- For contract, a schema is a contracted format of message between producer and consumer.
- For serialization, a producer serializes a message according to the contracted schema.
- For deserialization, a consumer deserializes a message according to the contracted schema.
- For validity, usually, a consumer checks the validity of the inbound serialized message before deserializing it.
So I was thinking that Eventmesh could take care of validity. If the inbound message from the producer is not obeying the specified schema, Eventmesh will discard this message and send an error status to the producer. The corresponding modules could be ```eventmesh-schema-json ```, ```eventmesh-schema-avro ```, and ```eventmesh-schema-protobuf ```.
Also, I get your point. Thinking from object-oriented programming, schema plugins could also provide the functions of ```serialize``` and ```deserialize```. Correct me if I had a misunderstanding.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org
[GitHub] [incubator-eventmesh] ruanwenjun commented on a change in pull request #821: [ISSUE #614] Add schema plugin and its api
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on a change in pull request #821:
URL: https://github.com/apache/incubator-eventmesh/pull/821#discussion_r841227912
##########
File path: eventmesh-schema-plugin/eventmesh-schema-api/src/main/java/org/apache/eventmesh/schema/api/SchemaPluginFactory.java
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.eventmesh.schema.api;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import org.apache.eventmesh.spi.EventMeshExtensionFactory;
+
+import lombok.experimental.UtilityClass;
+
+@UtilityClass
+public class SchemaPluginFactory {
+
+ /**
+ * Get {@code SchemaRegistry}
+ *
+ * @param schemaRegistryType the name of schema registry that configured in properties file
+ * @return schema registry plugin or null
+ */
+ public static SchemaRegistry getSchemaRegistry(String schemaRegistryType) {
+ checkNotNull(schemaRegistryType, "SchemaRegistryType cannot be null");
+ SchemaRegistry schemaRegistry = EventMeshExtensionFactory.getExtension(SchemaRegistry.class, schemaRegistryType);
+ return checkNotNull(schemaRegistry, "MetricsRegistryType: " + schemaRegistryType + " is not supported");
+ }
Review comment:
```suggestion
return checkNotNull(schemaRegistry, "SchemaRegistryType: " + schemaRegistryType + " is not supported");
```
##########
File path: eventmesh-schema-plugin/eventmesh-schema-api/src/main/java/org/apache/eventmesh/schema/api/SchemaType.java
##########
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.eventmesh.schema.api;
+
+import org.apache.commons.lang3.StringUtils;
+
+public enum SchemaType {
+ JSON("JSON"),
+ AVRO("AVRO"),
+ PROTOBUF("PROTOBUF");
+
+ private final String schemaTypeName;
+
+ SchemaType(String schemaTypeName) {
+ this.schemaTypeName = StringUtils.upperCase(StringUtils.deleteWhitespace(schemaTypeName));
+ }
Review comment:
Use `this.schemaTypeName = schemaTypeName` is better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org
[GitHub] [incubator-eventmesh] jzhou59 commented on a change in pull request #821: [ISSUE #614] Add schema plugin and its api
Posted by GitBox <gi...@apache.org>.
jzhou59 commented on a change in pull request #821:
URL: https://github.com/apache/incubator-eventmesh/pull/821#discussion_r841237864
##########
File path: eventmesh-schema-plugin/eventmesh-schema-api/src/main/java/org/apache/eventmesh/schema/api/SchemaType.java
##########
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.eventmesh.schema.api;
+
+import org.apache.commons.lang3.StringUtils;
+
+public enum SchemaType {
+ JSON("JSON"),
+ AVRO("AVRO"),
+ PROTOBUF("PROTOBUF");
+
+ private final String schemaTypeName;
+
+ SchemaType(String schemaTypeName) {
+ this.schemaTypeName = StringUtils.upperCase(StringUtils.deleteWhitespace(schemaTypeName));
+ }
Review comment:
Ok, I was trying to specify all schema types to uppercase.
But you are right, I have altered the schema type according to IETF(or official) abbreviation.
References:
- [JSON](https://datatracker.ietf.org/doc/html/rfc8259)
- [Avro](https://avro.apache.org/docs/current/spec.html)
- [Protobuf](https://datatracker.ietf.org/doc/html/draft-rfernando-protocol-buffers-00#section-1.1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org
[GitHub] [incubator-eventmesh] jzhou59 commented on a change in pull request #821: [ISSUE #614] Add schema plugin and its api
Posted by GitBox <gi...@apache.org>.
jzhou59 commented on a change in pull request #821:
URL: https://github.com/apache/incubator-eventmesh/pull/821#discussion_r841237480
##########
File path: eventmesh-schema-plugin/eventmesh-schema-api/src/main/java/org/apache/eventmesh/schema/api/SchemaPluginFactory.java
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.eventmesh.schema.api;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import org.apache.eventmesh.spi.EventMeshExtensionFactory;
+
+import lombok.experimental.UtilityClass;
+
+@UtilityClass
+public class SchemaPluginFactory {
+
+ /**
+ * Get {@code SchemaRegistry}
+ *
+ * @param schemaRegistryType the name of schema registry that configured in properties file
+ * @return schema registry plugin or null
+ */
+ public static SchemaRegistry getSchemaRegistry(String schemaRegistryType) {
+ checkNotNull(schemaRegistryType, "SchemaRegistryType cannot be null");
+ SchemaRegistry schemaRegistry = EventMeshExtensionFactory.getExtension(SchemaRegistry.class, schemaRegistryType);
+ return checkNotNull(schemaRegistry, "MetricsRegistryType: " + schemaRegistryType + " is not supported");
+ }
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org