You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/01/10 18:00:46 UTC
[GitHub] [flink] lmagic233 opened a new pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
lmagic233 opened a new pull request #14604:
URL: https://github.com/apache/flink/pull/14604
## What is the purpose of the change
This PR introduces option `json.encode.decimal-as-plain-number` for JSON formats.
## Brief change log
- Add this option in `JsonOptions`.
- Let `JsonFormatFactory`s and `JsonRowDataSerializationSchema`s be aware of this option.
## Verifying this change
This change added tests and can be verified as follows:
- Added test in `JsonRowDataSerDeSchemaTest` to cover different decimal cases.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (**yes** / no)
- If yes, how is the feature documented? (not applicable / **docs** / JavaDocs / not documented)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
}, {
"hash" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959",
"triggerID" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"triggerType" : "PUSH"
}, {
"hash" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962",
"triggerID" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"triggerType" : "PUSH"
}, {
"hash" : "14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11968",
"triggerID" : "14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11968)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wuchong merged pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
wuchong merged pull request #14604:
URL: https://github.com/apache/flink/pull/14604
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lmagic233 commented on a change in pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
lmagic233 commented on a change in pull request #14604:
URL: https://github.com/apache/flink/pull/14604#discussion_r556286068
##########
File path: flink-formats/flink-json/src/test/java/org/apache/flink/formats/json/JsonRowDataSerDeSchemaTest.java
##########
@@ -562,6 +579,56 @@ public void testSerializationMapNullKey() throws Exception {
assertEquals(expectResult3, new String(actual3));
}
+ @Test
+ public void testSerializationDecimalEncode() throws Exception {
+ BigDecimal decimal1 = new BigDecimal("123.456789");
+ BigDecimal decimal2 = new BigDecimal("454621864049246170");
+ BigDecimal decimal3 = new BigDecimal("0.000000027");
+
+ ObjectMapper objectMapper = new ObjectMapper();
+ objectMapper.configure(JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN, true);
+
+ ObjectNode root = objectMapper.createObjectNode();
+ root.put("decimal1", decimal1);
+ root.put("decimal2", decimal2);
+ root.put("decimal3", decimal3);
+
+ byte[] serializedJson = objectMapper.writeValueAsBytes(root);
Review comment:
Modified test case to use explicit JSON strings.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
}, {
"hash" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959",
"triggerID" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"triggerType" : "PUSH"
}, {
"hash" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962",
"triggerID" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6f16ecba1741139347662f1a892cca2885b603c9 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862)
* 34a633e801bcb8100e0962b358b4e400a47c7047 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959)
* 095aa26dba7a04800cb203e252774d0d7655cc25 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
}, {
"hash" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959",
"triggerID" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"triggerType" : "PUSH"
}, {
"hash" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6f16ecba1741139347662f1a892cca2885b603c9 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862)
* 34a633e801bcb8100e0962b358b4e400a47c7047 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959)
* 095aa26dba7a04800cb203e252774d0d7655cc25 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lmagic233 commented on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
lmagic233 commented on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-759194539
Copy that @wuchong , I have rebased the branch. Also I have rewrote the test case, plase verify.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1cd9b1e046103d9209dfaaa3054dfae600c14de5 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857)
* 6f16ecba1741139347662f1a892cca2885b603c9 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1cd9b1e046103d9209dfaaa3054dfae600c14de5 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wuchong commented on a change in pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
wuchong commented on a change in pull request #14604:
URL: https://github.com/apache/flink/pull/14604#discussion_r556296636
##########
File path: flink-formats/flink-json/src/test/java/org/apache/flink/formats/json/JsonRowDataSerDeSchemaTest.java
##########
@@ -562,6 +578,55 @@ public void testSerializationMapNullKey() throws Exception {
assertEquals(expectResult3, new String(actual3));
}
+ @Test
+ public void testSerializationDecimalEncode() throws Exception {
+ String original =
+ "{\"decimal1\":123.456789,\"decimal2\":454621864049246170,\"decimal3\":0.000000027}";
+ String expectedWithPlainDecimal =
+ "{\"decimal1\":123.456789,\"decimal2\":454621864049246170,\"decimal3\":0.000000027}";
Review comment:
1. They are equal? We may only need to declare just one `plainDecimalJson`?
2. Please move the variable near where it is used.
##########
File path: flink-formats/flink-json/src/test/java/org/apache/flink/formats/json/JsonRowDataSerDeSchemaTest.java
##########
@@ -562,6 +578,55 @@ public void testSerializationMapNullKey() throws Exception {
assertEquals(expectResult3, new String(actual3));
}
+ @Test
+ public void testSerializationDecimalEncode() throws Exception {
+ String original =
+ "{\"decimal1\":123.456789,\"decimal2\":454621864049246170,\"decimal3\":0.000000027}";
+ String expectedWithPlainDecimal =
+ "{\"decimal1\":123.456789,\"decimal2\":454621864049246170,\"decimal3\":0.000000027}";
+ String expectedWithoutPlainDecimal =
+ "{\"decimal1\":123.456789,\"decimal2\":4.5462186404924617E+17,\"decimal3\":2.7E-8}";
Review comment:
1. We can call this `scientificDecimalJson`.
2. Please move the variable near where it is used.
##########
File path: flink-formats/flink-json/src/test/java/org/apache/flink/formats/json/JsonRowDataSerDeSchemaTest.java
##########
@@ -562,6 +578,55 @@ public void testSerializationMapNullKey() throws Exception {
assertEquals(expectResult3, new String(actual3));
}
+ @Test
+ public void testSerializationDecimalEncode() throws Exception {
+ String original =
+ "{\"decimal1\":123.456789,\"decimal2\":454621864049246170,\"decimal3\":0.000000027}";
+ String expectedWithPlainDecimal =
+ "{\"decimal1\":123.456789,\"decimal2\":454621864049246170,\"decimal3\":0.000000027}";
+ String expectedWithoutPlainDecimal =
+ "{\"decimal1\":123.456789,\"decimal2\":4.5462186404924617E+17,\"decimal3\":2.7E-8}";
+
+ RowType schema =
+ (RowType)
+ ROW(
+ FIELD("decimal1", DECIMAL(9, 6)),
+ FIELD("decimal2", DECIMAL(20, 0)),
+ FIELD("decimal3", DECIMAL(11, 9)))
+ .getLogicalType();
+
+ TypeInformation<RowData> resultTypeInfo = InternalTypeInfo.of(schema);
+
+ JsonRowDataDeserializationSchema deserializationSchema =
+ new JsonRowDataDeserializationSchema(
+ schema, resultTypeInfo, false, false, TimestampFormat.ISO_8601);
+
+ JsonRowDataSerializationSchema serializationSchemaWithPlainDecimal =
+ new JsonRowDataSerializationSchema(
+ schema,
+ TimestampFormat.ISO_8601,
+ JsonOptions.MapNullKeyMode.LITERAL,
+ "null",
+ true);
+ JsonRowDataSerializationSchema serializationSchemaWithoutPlainDecimal =
Review comment:
`scientificDecimalSerializer`
##########
File path: flink-formats/flink-json/src/test/java/org/apache/flink/formats/json/JsonRowDataSerDeSchemaTest.java
##########
@@ -562,6 +578,55 @@ public void testSerializationMapNullKey() throws Exception {
assertEquals(expectResult3, new String(actual3));
}
+ @Test
+ public void testSerializationDecimalEncode() throws Exception {
+ String original =
+ "{\"decimal1\":123.456789,\"decimal2\":454621864049246170,\"decimal3\":0.000000027}";
+ String expectedWithPlainDecimal =
+ "{\"decimal1\":123.456789,\"decimal2\":454621864049246170,\"decimal3\":0.000000027}";
+ String expectedWithoutPlainDecimal =
+ "{\"decimal1\":123.456789,\"decimal2\":4.5462186404924617E+17,\"decimal3\":2.7E-8}";
+
+ RowType schema =
+ (RowType)
+ ROW(
+ FIELD("decimal1", DECIMAL(9, 6)),
+ FIELD("decimal2", DECIMAL(20, 0)),
+ FIELD("decimal3", DECIMAL(11, 9)))
+ .getLogicalType();
+
+ TypeInformation<RowData> resultTypeInfo = InternalTypeInfo.of(schema);
+
+ JsonRowDataDeserializationSchema deserializationSchema =
+ new JsonRowDataDeserializationSchema(
+ schema, resultTypeInfo, false, false, TimestampFormat.ISO_8601);
+
+ JsonRowDataSerializationSchema serializationSchemaWithPlainDecimal =
Review comment:
`plainDecimalSerializer`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1cd9b1e046103d9209dfaaa3054dfae600c14de5 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857)
* 6f16ecba1741139347662f1a892cca2885b603c9 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
}, {
"hash" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959",
"triggerID" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"triggerType" : "PUSH"
}, {
"hash" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962",
"triggerID" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"triggerType" : "PUSH"
}, {
"hash" : "14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11968",
"triggerID" : "14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 095aa26dba7a04800cb203e252774d0d7655cc25 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962)
* 14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11968)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
}, {
"hash" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6f16ecba1741139347662f1a892cca2885b603c9 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862)
* 34a633e801bcb8100e0962b358b4e400a47c7047 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757517636
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a (Fri May 28 07:06:46 UTC 2021)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wuchong commented on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
wuchong commented on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757593071
cc @fsk119
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lmagic233 commented on a change in pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
lmagic233 commented on a change in pull request #14604:
URL: https://github.com/apache/flink/pull/14604#discussion_r556286139
##########
File path: flink-formats/flink-json/src/test/java/org/apache/flink/formats/json/JsonRowDataSerDeSchemaTest.java
##########
@@ -562,6 +579,56 @@ public void testSerializationMapNullKey() throws Exception {
assertEquals(expectResult3, new String(actual3));
}
+ @Test
+ public void testSerializationDecimalEncode() throws Exception {
+ BigDecimal decimal1 = new BigDecimal("123.456789");
+ BigDecimal decimal2 = new BigDecimal("454621864049246170");
+ BigDecimal decimal3 = new BigDecimal("0.000000027");
+
+ ObjectMapper objectMapper = new ObjectMapper();
+ objectMapper.configure(JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN, true);
+
+ ObjectNode root = objectMapper.createObjectNode();
+ root.put("decimal1", decimal1);
+ root.put("decimal2", decimal2);
+ root.put("decimal3", decimal3);
+
+ byte[] serializedJson = objectMapper.writeValueAsBytes(root);
+
+ DataType dataType =
+ ROW(
+ FIELD("decimal1", DECIMAL(9, 6)),
+ FIELD("decimal2", DECIMAL(20, 0)),
+ FIELD("decimal3", DECIMAL(11, 9)));
+
+ RowType schema = (RowType) dataType.getLogicalType();
+ TypeInformation<RowData> resultTypeInfo = InternalTypeInfo.of(schema);
+
+ JsonRowDataDeserializationSchema deserializationSchema =
+ new JsonRowDataDeserializationSchema(
+ schema, resultTypeInfo, false, false, TimestampFormat.ISO_8601);
+
+ Row expected = new Row(3);
+ expected.setField(0, decimal1);
+ expected.setField(1, decimal2);
+ expected.setField(2, decimal3);
+
+ RowData rowData = deserializationSchema.deserialize(serializedJson);
Review comment:
Ditto.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757517636
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 1cd9b1e046103d9209dfaaa3054dfae600c14de5 (Sun Jan 10 18:03:54 UTC 2021)
**Warnings:**
* Documentation files were touched, but no `.zh.md` files: Update Chinese documentation or file Jira ticket.
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1cd9b1e046103d9209dfaaa3054dfae600c14de5 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6f16ecba1741139347662f1a892cca2885b603c9 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
}, {
"hash" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959",
"triggerID" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"triggerType" : "PUSH"
}, {
"hash" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962",
"triggerID" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"triggerType" : "PUSH"
}, {
"hash" : "14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11968",
"triggerID" : "14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6f16ecba1741139347662f1a892cca2885b603c9 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862)
* 34a633e801bcb8100e0962b358b4e400a47c7047 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959)
* 095aa26dba7a04800cb203e252774d0d7655cc25 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962)
* 14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11968)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wuchong commented on a change in pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
wuchong commented on a change in pull request #14604:
URL: https://github.com/apache/flink/pull/14604#discussion_r555766824
##########
File path: docs/dev/table/connectors/formats/json.md
##########
@@ -136,6 +136,13 @@ Format Options
<td>String</td>
<td>Specify string literal to replace null key when <code>'json.map-null-key.mode'</code> is LITERAL.</td>
</tr>
+ <tr>
+ <td><h5>json.encode.decimal-as-plain-number</h5></td>
+ <td>optional</td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>Encode all decimals as plain numbers instead of possible scientific notations. e.g. <code>0.000000027</code> is encoded as <code>2.7E-8</code> when this option is set to false.</td>
Review comment:
```suggestion
<td>Encode all decimals as plain numbers instead of possible scientific notations. By default decimals are written using scientific notation. For example, <code>0.000000027</code> is encoded as <code>2.7E-8</code> by default, and will be written as <code>0.000000027</code> if set this option to true.</td>
```
##########
File path: flink-formats/flink-json/src/test/java/org/apache/flink/formats/json/JsonRowDataSerDeSchemaTest.java
##########
@@ -562,6 +579,56 @@ public void testSerializationMapNullKey() throws Exception {
assertEquals(expectResult3, new String(actual3));
}
+ @Test
+ public void testSerializationDecimalEncode() throws Exception {
+ BigDecimal decimal1 = new BigDecimal("123.456789");
+ BigDecimal decimal2 = new BigDecimal("454621864049246170");
+ BigDecimal decimal3 = new BigDecimal("0.000000027");
+
+ ObjectMapper objectMapper = new ObjectMapper();
+ objectMapper.configure(JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN, true);
+
+ ObjectNode root = objectMapper.createObjectNode();
+ root.put("decimal1", decimal1);
+ root.put("decimal2", decimal2);
+ root.put("decimal3", decimal3);
+
+ byte[] serializedJson = objectMapper.writeValueAsBytes(root);
+
+ DataType dataType =
+ ROW(
+ FIELD("decimal1", DECIMAL(9, 6)),
+ FIELD("decimal2", DECIMAL(20, 0)),
+ FIELD("decimal3", DECIMAL(11, 9)));
+
+ RowType schema = (RowType) dataType.getLogicalType();
+ TypeInformation<RowData> resultTypeInfo = InternalTypeInfo.of(schema);
+
+ JsonRowDataDeserializationSchema deserializationSchema =
+ new JsonRowDataDeserializationSchema(
+ schema, resultTypeInfo, false, false, TimestampFormat.ISO_8601);
+
+ Row expected = new Row(3);
+ expected.setField(0, decimal1);
+ expected.setField(1, decimal2);
+ expected.setField(2, decimal3);
+
+ RowData rowData = deserializationSchema.deserialize(serializedJson);
Review comment:
We can manually construct the input RowData by using `GenericRowData` and `DecimalData`.
##########
File path: flink-formats/flink-json/src/test/java/org/apache/flink/formats/json/JsonRowDataSerDeSchemaTest.java
##########
@@ -562,6 +579,56 @@ public void testSerializationMapNullKey() throws Exception {
assertEquals(expectResult3, new String(actual3));
}
+ @Test
+ public void testSerializationDecimalEncode() throws Exception {
+ BigDecimal decimal1 = new BigDecimal("123.456789");
+ BigDecimal decimal2 = new BigDecimal("454621864049246170");
+ BigDecimal decimal3 = new BigDecimal("0.000000027");
+
+ ObjectMapper objectMapper = new ObjectMapper();
+ objectMapper.configure(JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN, true);
+
+ ObjectNode root = objectMapper.createObjectNode();
+ root.put("decimal1", decimal1);
+ root.put("decimal2", decimal2);
+ root.put("decimal3", decimal3);
+
+ byte[] serializedJson = objectMapper.writeValueAsBytes(root);
Review comment:
I think we can manually construct the serialized JSON string.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
}, {
"hash" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959",
"triggerID" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"triggerType" : "PUSH"
}, {
"hash" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962",
"triggerID" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"triggerType" : "PUSH"
}, {
"hash" : "14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6f16ecba1741139347662f1a892cca2885b603c9 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862)
* 34a633e801bcb8100e0962b358b4e400a47c7047 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959)
* 095aa26dba7a04800cb203e252774d0d7655cc25 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962)
* 14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wuchong commented on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
wuchong commented on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-759192699
Hi @lmagic233 , please use `git rebase` instead of `git merge` or `git pull`. You are pulling other commits into this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1cd9b1e046103d9209dfaaa3054dfae600c14de5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-757521857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11857",
"triggerID" : "1cd9b1e046103d9209dfaaa3054dfae600c14de5",
"triggerType" : "PUSH"
}, {
"hash" : "6f16ecba1741139347662f1a892cca2885b603c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11862",
"triggerID" : "6f16ecba1741139347662f1a892cca2885b603c9",
"triggerType" : "PUSH"
}, {
"hash" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959",
"triggerID" : "34a633e801bcb8100e0962b358b4e400a47c7047",
"triggerType" : "PUSH"
}, {
"hash" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962",
"triggerID" : "095aa26dba7a04800cb203e252774d0d7655cc25",
"triggerType" : "PUSH"
}, {
"hash" : "14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11968",
"triggerID" : "14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 34a633e801bcb8100e0962b358b4e400a47c7047 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11959)
* 095aa26dba7a04800cb203e252774d0d7655cc25 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11962)
* 14ecbcb1bc4fc86bcdda844a48b8c187f6c9a96a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11968)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wuchong commented on a change in pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
wuchong commented on a change in pull request #14604:
URL: https://github.com/apache/flink/pull/14604#discussion_r556294884
##########
File path: docs/dev/table/connectors/formats/json.md
##########
@@ -136,6 +136,13 @@ Format Options
<td>String</td>
<td>Specify string literal to replace null key when <code>'json.map-null-key.mode'</code> is LITERAL.</td>
</tr>
+ <tr>
+ <td><h5>json.encode.decimal-as-plain-number</h5></td>
+ <td>optional</td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>Encode all decimals as plain numbers instead of possible scientific notations. e.g. <code>0.000000027</code> is encoded as <code>2.7E-8</code> when this option is set to false.</td>
Review comment:
Please also update this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lmagic233 commented on pull request #14604: [FLINK-20861][format/json] Provide an option for serializing DECIMALs in JSON as plain number instead of scientific notation
Posted by GitBox <gi...@apache.org>.
lmagic233 commented on pull request #14604:
URL: https://github.com/apache/flink/pull/14604#issuecomment-759259586
@wuchong Squashed commits and optimized test code naming & documentations according to your revision, please verify.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org