You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by "featzhang (via GitHub)" <gi...@apache.org> on 2023/04/22 04:56:46 UTC
[GitHub] [inlong] featzhang opened a new pull request, #7894: [INLONG-7893][Manager] Support field description when parsing field by json
featzhang opened a new pull request, #7894:
URL: https://github.com/apache/inlong/pull/7894
[INLONG-7893][Manager] Support field description when parsing field by json
Fixes #7893
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] fuweng11 commented on a diff in pull request #7894: [INLONG-7893][Manager] Support field description when parsing field by JSON
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #7894:
URL: https://github.com/apache/inlong/pull/7894#discussion_r1174540237
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/InlongConstants.java:
##########
@@ -167,5 +167,8 @@ public class InlongConstants {
public static final Set<String> STREAM_FIELD_TYPES =
Sets.newHashSet("string", "int", "long", "float", "double", "date", "timestamp");
+ public static final String STREAM_FILED_JSON_NAME_PROP = "name";
Review Comment:
This is also used by `sinkFIeld`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] fuweng11 commented on a diff in pull request #7894: [INLONG-7893][Manager] Support field description when parsing field by JSON
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #7894:
URL: https://github.com/apache/inlong/pull/7894#discussion_r1174537279
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java:
##########
@@ -791,11 +794,19 @@ private Map<String, String> parseFieldsBySql(String sql) throws JSQLParserExcept
return fields;
}
- private Map<String, String> parseFieldsByJson(String statement) throws JsonProcessingException {
- // Use LinkedHashMap deserialization to keep the order of the fields
- return objectMapper.readValue(statement,
- new TypeReference<LinkedHashMap<String, String>>() {
- });
+ private List<SinkField> parseFieldsByJson(String statement) throws JsonProcessingException {
+ return objectMapper.readValue(statement, new TypeReference<List<Map<String, String>>>() {
+ }).stream().map(line -> {
+ String name = line.get(STREAM_FILED_JSON_NAME_PROP);
+ String type = line.get(STREAM_FILED_JSON_TYPE_PROP);
+ String desc = line.get(STREAM_FILED_JSON_COMMENT_PROP);
+ Map.Entry<String, String> next = line.entrySet().iterator().next();
+ SinkField streamField = new SinkField();
Review Comment:
`streamField` -> `sinkField`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] featzhang commented on a diff in pull request #7894: [INLONG-7893][Manager] Support field description when parsing field by JSON
Posted by "featzhang (via GitHub)" <gi...@apache.org>.
featzhang commented on code in PR #7894:
URL: https://github.com/apache/inlong/pull/7894#discussion_r1174726860
##########
inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/consts/InlongConstants.java:
##########
@@ -167,5 +167,8 @@ public class InlongConstants {
public static final Set<String> STREAM_FIELD_TYPES =
Sets.newHashSet("string", "int", "long", "float", "double", "date", "timestamp");
+ public static final String STREAM_FILED_JSON_NAME_PROP = "name";
Review Comment:
Fixed, change to `BATCH_PARSING_FIELD_JSON_NAME_PROP`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] dockerzhang merged pull request #7894: [INLONG-7893][Manager] Support field description when parsing field by JSON
Posted by "dockerzhang (via GitHub)" <gi...@apache.org>.
dockerzhang merged PR #7894:
URL: https://github.com/apache/inlong/pull/7894
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] featzhang commented on a diff in pull request #7894: [INLONG-7893][Manager] Support field description when parsing field by JSON
Posted by "featzhang (via GitHub)" <gi...@apache.org>.
featzhang commented on code in PR #7894:
URL: https://github.com/apache/inlong/pull/7894#discussion_r1174726532
##########
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java:
##########
@@ -791,11 +794,19 @@ private Map<String, String> parseFieldsBySql(String sql) throws JSQLParserExcept
return fields;
}
- private Map<String, String> parseFieldsByJson(String statement) throws JsonProcessingException {
- // Use LinkedHashMap deserialization to keep the order of the fields
- return objectMapper.readValue(statement,
- new TypeReference<LinkedHashMap<String, String>>() {
- });
+ private List<SinkField> parseFieldsByJson(String statement) throws JsonProcessingException {
+ return objectMapper.readValue(statement, new TypeReference<List<Map<String, String>>>() {
+ }).stream().map(line -> {
+ String name = line.get(STREAM_FILED_JSON_NAME_PROP);
+ String type = line.get(STREAM_FILED_JSON_TYPE_PROP);
+ String desc = line.get(STREAM_FILED_JSON_COMMENT_PROP);
+ Map.Entry<String, String> next = line.entrySet().iterator().next();
+ SinkField streamField = new SinkField();
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org