You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/03/10 08:55:25 UTC
[GitHub] [flink] alpreu opened a new pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
alpreu opened a new pull request #19035:
URL: https://github.com/apache/flink/pull/19035
## What is the purpose of the change
The default delivery guarantee of the Elasticsearch sink should be AT_LEAST_ONCE as it was the case in 1.14 for upgradability/consistency reasons.
Chinese docs have to be updated still.
## Brief change log
- made AT_LEAST_ONCE the default delivery guarantee
- updated english docs
## Verifying this change
This change is already covered by existing tests, such as ElasticsearchXSinkITCase, E2E Tests
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (**yes** / no)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no**)
- If yes, how is the feature documented? (not applicable / **docs** / JavaDocs / not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19035:
URL: https://github.com/apache/flink/pull/19035#issuecomment-1063817920
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10cb63b99fd9289856226edc362077c2b97a2438",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818",
"triggerID" : "10cb63b99fd9289856226edc362077c2b97a2438",
"triggerType" : "PUSH"
}, {
"hash" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10cb63b99fd9289856226edc362077c2b97a2438 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818)
* 1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #19035:
URL: https://github.com/apache/flink/pull/19035#issuecomment-1063817920
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10cb63b99fd9289856226edc362077c2b97a2438",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "10cb63b99fd9289856226edc362077c2b97a2438",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10cb63b99fd9289856226edc362077c2b97a2438 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19035:
URL: https://github.com/apache/flink/pull/19035#issuecomment-1063817920
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10cb63b99fd9289856226edc362077c2b97a2438",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818",
"triggerID" : "10cb63b99fd9289856226edc362077c2b97a2438",
"triggerType" : "PUSH"
}, {
"hash" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32918",
"triggerID" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"triggerType" : "PUSH"
}, {
"hash" : "204a2693a3b7cd1a07f7b33d8dee1310601bd273",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33112",
"triggerID" : "204a2693a3b7cd1a07f7b33d8dee1310601bd273",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32918)
* 204a2693a3b7cd1a07f7b33d8dee1310601bd273 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33112)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19035:
URL: https://github.com/apache/flink/pull/19035#issuecomment-1063817920
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10cb63b99fd9289856226edc362077c2b97a2438",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818",
"triggerID" : "10cb63b99fd9289856226edc362077c2b97a2438",
"triggerType" : "PUSH"
}, {
"hash" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32918",
"triggerID" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"triggerType" : "PUSH"
}, {
"hash" : "204a2693a3b7cd1a07f7b33d8dee1310601bd273",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "204a2693a3b7cd1a07f7b33d8dee1310601bd273",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32918)
* 204a2693a3b7cd1a07f7b33d8dee1310601bd273 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19035:
URL: https://github.com/apache/flink/pull/19035#issuecomment-1063817920
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10cb63b99fd9289856226edc362077c2b97a2438",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818",
"triggerID" : "10cb63b99fd9289856226edc362077c2b97a2438",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10cb63b99fd9289856226edc362077c2b97a2438 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19035:
URL: https://github.com/apache/flink/pull/19035#issuecomment-1063817920
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10cb63b99fd9289856226edc362077c2b97a2438",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818",
"triggerID" : "10cb63b99fd9289856226edc362077c2b97a2438",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10cb63b99fd9289856226edc362077c2b97a2438 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19035:
URL: https://github.com/apache/flink/pull/19035#issuecomment-1063817920
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10cb63b99fd9289856226edc362077c2b97a2438",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818",
"triggerID" : "10cb63b99fd9289856226edc362077c2b97a2438",
"triggerType" : "PUSH"
}, {
"hash" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32918",
"triggerID" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"triggerType" : "PUSH"
}, {
"hash" : "204a2693a3b7cd1a07f7b33d8dee1310601bd273",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33112",
"triggerID" : "204a2693a3b7cd1a07f7b33d8dee1310601bd273",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 204a2693a3b7cd1a07f7b33d8dee1310601bd273 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33112)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] fapaul merged pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
fapaul merged pull request #19035:
URL: https://github.com/apache/flink/pull/19035
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19035:
URL: https://github.com/apache/flink/pull/19035#issuecomment-1063817920
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10cb63b99fd9289856226edc362077c2b97a2438",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818",
"triggerID" : "10cb63b99fd9289856226edc362077c2b97a2438",
"triggerType" : "PUSH"
}, {
"hash" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32918",
"triggerID" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 10cb63b99fd9289856226edc362077c2b97a2438 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818)
* 1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32918)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] alpreu commented on pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
alpreu commented on pull request #19035:
URL: https://github.com/apache/flink/pull/19035#issuecomment-1065090962
> Also I would like to clarify one thing now that at-least-once delivery is default it might happen that users run the pipeline without checkpointing and without buffer flushing configured. I am afraid this leads to that the sink never flushes until it finishes. How did the old sink deal with that?
It looks like the old sink always had `flushOnCheckpoint=true`, it seems that it was only possible to disable it when using the Table API and changing it's connector option
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #19035: [FLINK-26281][connectors/elasticsearch] setting default delivery guarantee to AT_LEAST_ONCE
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19035:
URL: https://github.com/apache/flink/pull/19035#issuecomment-1063817920
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "10cb63b99fd9289856226edc362077c2b97a2438",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32818",
"triggerID" : "10cb63b99fd9289856226edc362077c2b97a2438",
"triggerType" : "PUSH"
}, {
"hash" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32918",
"triggerID" : "1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1e3fc93bddb8f158f623b1b3f6cd76f40b474d5d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32918)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org