You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/10/20 19:54:28 UTC
[GitHub] [flink] kottmann opened a new pull request #13457: [FLINK-8357] Use rolling logs as default
kottmann opened a new pull request #13457:
URL: https://github.com/apache/flink/pull/13457
<!--
*Thank you very much for contributing to Apache Flink - we are happy that you want to help us improve Flink. To help the community review your contribution in the best possible way, please go through the checklist below, which will get the contribution into a shape in which it can be best reviewed.*
*Please understand that we do not do this to make contributions to Flink a hassle. In order to uphold a high standard of quality for code contributions, while at the same time managing a large number of contributions, we need contributors to prepare the contributions well, and give reviewers enough contextual information for the review. Please also understand that contributions that do not follow this guide will take longer to review and thus typically be picked up with lower priority by the community.*
## Contribution Checklist
- Make sure that the pull request corresponds to a [JIRA issue](https://issues.apache.org/jira/projects/FLINK/issues). Exceptions are made for typos in JavaDoc or documentation files, which need no JIRA issue.
- Name the pull request in the form "[FLINK-XXXX] [component] Title of the pull request", where *FLINK-XXXX* should be replaced by the actual issue number. Skip *component* if you are unsure about which is the best component.
Typo fixes that have no associated JIRA issue should be named following this pattern: `[hotfix] [docs] Fix typo in event time introduction` or `[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Make sure that the change passes the automated tests, i.e., `mvn clean verify` passes. You can set up Azure Pipelines CI to do that following [this guide](https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-Tutorial:SettingupAzurePipelinesforaforkoftheFlinkrepository).
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message (including the JIRA id)
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
## What is the purpose of the change
This PR adds rolling logs to the default configuration.
## Brief change log
- Rolling logs in log4j.properties
- Roll after 100MB
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage. I tested the change by running Flink on EMR (yarn) and local.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger merged pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger merged pull request #13457:
URL: https://github.com/apache/flink/pull/13457
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
}, {
"hash" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970",
"triggerID" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"triggerType" : "PUSH"
}, {
"hash" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8042",
"triggerID" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"triggerType" : "PUSH"
}, {
"hash" : "1d1ced78d39676c69138e91b28bc138cb5fa71f1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8382",
"triggerID" : "1d1ced78d39676c69138e91b28bc138cb5fa71f1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1d1ced78d39676c69138e91b28bc138cb5fa71f1 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8382)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-717725432
Merging ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
}, {
"hash" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970",
"triggerID" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"triggerType" : "PUSH"
}, {
"hash" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8042",
"triggerID" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bf35025687d8e1bd4e14e40444536ba0535e0d5e Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8042)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] kottmann commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
kottmann commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697013989
Things to be considered
- Any opinions about the file size? What is a good size?
- The max number of files could be set
- OnStartupTriggeringPolicy can be used to replace the rolling on startup code in the shell scripts
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on a change in pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on a change in pull request #13457:
URL: https://github.com/apache/flink/pull/13457#discussion_r512183100
##########
File path: flink-dist/src/main/flink-bin/bin/config.sh
##########
@@ -241,6 +241,7 @@ fi
if [ -z "${MAX_LOG_FILE_NUMBER}" ]; then
MAX_LOG_FILE_NUMBER=$(readFromConfig ${KEY_ENV_LOG_MAX} ${DEFAULT_ENV_LOG_MAX} "${YAML_CONF}")
+ export MAX_LOG_FILE_NUMBER
Review comment:
at least on GitHub, the indentation seems to be off
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 917848f48bff5032c57bedab45d30dbbb66f664a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807)
* 6200c8fb624b8a6415f983c705336524a9a8a342 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] kottmann commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
kottmann commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-717195148
Is it possible for contributors to see the CI output?
I actually didn't manage to get every tests to run on my setup (even with
latest master and without my changes).
On Tue, Oct 27, 2020, 11:35 Robert Metzger <no...@github.com> wrote:
> Thanks. The CI system should pick up your changes and verify them. Let's
> wait and see.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/flink/pull/13457#issuecomment-717148013>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAJJJ6Q7JNWX2NP6NI7TRX3SM2PAXANCNFSM4RWJQCEA>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 917848f48bff5032c57bedab45d30dbbb66f664a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807)
* 6200c8fb624b8a6415f983c705336524a9a8a342 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] kottmann commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
kottmann commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-706996982
@rmetzger the last change adds the OnStartupTriggeringPolicy and the files roll now correctly.
I looked at the log4j-session.properties and they currently log only to the console and not to file. Therefore I didn't change it.
The log4j-cli.properties logs to a file and doesn't use a rolling policy, I can change that to rolling.
The log4j-console.properties uses the rolling policy and max files to keep is set to 10. Maybe it makes sense to do the same for log4j.properties.
Then I noticed that there is a CoreOptions.FLINK_LOG_MAX config parameter, is it save to remove it? This will not be used anymore.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-708604644
If we are going to remove the config param: There are a few more usages of it in `config.sh`. But let's wait for Zentol first.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-713101853
(Sry, hit the wrong button -- it's too late 🤕 )
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-713101595
I'm really sorry that I'm only responding to you every 5 days. I'd actually like to keep a certain momentum going with this change. I'll try to get back faster!
I had an offline discussion with @zentol about the configuration parameter. Since it is documented, we treat it as public API, which we cannot break between minor releases.
However, he also wondered why we can't forward the configured FLINK_LOG_MAX value to the log4j settings?
```
appender.main.policies.type = Policies
appender.main.policies.size.type = SizeBasedTriggeringPolicy
appender.main.policies.size.size = 100MB
appender.main.policies.startup.type = OnStartupTriggeringPolicy
appender.main.strategy.type = DefaultRolloverStrategy
appender.main.strategy.max = ${env:MAX_LOG_FILE_NUMBER:-10}
```
I have tried this locally and it seems to work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6200c8fb624b8a6415f983c705336524a9a8a342 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423)
* 72242a97ba40dedecb60632bcd8bd7ac3d1459ff UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6200c8fb624b8a6415f983c705336524a9a8a342 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423)
* 72242a97ba40dedecb60632bcd8bd7ac3d1459ff Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-717148013
Thanks. The CI system should pick up your changes and verify them. Let's wait and see.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
}, {
"hash" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970",
"triggerID" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 72242a97ba40dedecb60632bcd8bd7ac3d1459ff Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710)
* 14db2d58abff9a9b1ab6c997af66180c6301373c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] kottmann commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
kottmann commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-700636398
Let me add this to log4j-session.properties too. What do you think about the On Startup Triggering Policy? This would make log4j to roll the file on startup and the logic form the shell scripts could be removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 917848f48bff5032c57bedab45d30dbbb66f664a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
}, {
"hash" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970",
"triggerID" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 14db2d58abff9a9b1ab6c997af66180c6301373c Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-700006536
The change works with the web UI ✅
The rolling works (tested it with 1MB instead of 100MB 😄 ) ✅
In my opinion, we should add the rolling also to `log4j-session.properties`, so that YARN and K8s sessions use it as well?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] kottmann commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
kottmann commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-709642130
> If we are going to remove the config param: There are a few more usages of it in `config.sh`. But let's wait for Zentol first.
I added a commit to remove it, in case we want to keep that change I will squash it into the first commit before merge.
log4j.properties now also contains the max limit of 10 files
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-699858888
Thanks a lot for your pull request. I'll soon take a look!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
}, {
"hash" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970",
"triggerID" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"triggerType" : "PUSH"
}, {
"hash" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8042",
"triggerID" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"triggerType" : "PUSH"
}, {
"hash" : "1d1ced78d39676c69138e91b28bc138cb5fa71f1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8382",
"triggerID" : "1d1ced78d39676c69138e91b28bc138cb5fa71f1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bf35025687d8e1bd4e14e40444536ba0535e0d5e Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8042)
* 1d1ced78d39676c69138e91b28bc138cb5fa71f1 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8382)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-713102468
So I'd say, let's re-introduce the config param, add it to the log4j.properties and then we'll finally merge this change.
I quite like this now, as it does the rotation stuff in log4j, not using some home-cooked bash stuff.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
}, {
"hash" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970",
"triggerID" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"triggerType" : "PUSH"
}, {
"hash" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8042",
"triggerID" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 14db2d58abff9a9b1ab6c997af66180c6301373c Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970)
* bf35025687d8e1bd4e14e40444536ba0535e0d5e Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8042)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] kottmann commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
kottmann commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697013989
Things to be considered
- Any opinions about the file size? What is a good size?
- The max number of files could be set
- OnStartupTriggeringPolicy can be used to replace the rolling on startup code in the shell scripts
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-717332311
The CI for your latest push has failed because of some transient network error in our build environment. This stuff sometimes happens. I manually triggered a re-start of the failed builds.
I've pushed your latest commit to my personal azure account, and it has passed: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8539&view=results
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-717725432
Merging ... Thanks a lot for your contribution!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger closed pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger closed pull request #13457:
URL: https://github.com/apache/flink/pull/13457
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697009388
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 917848f48bff5032c57bedab45d30dbbb66f664a (Tue Sep 22 22:10:59 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-700010022
I believe you also need to adopt the `config.sh#rotateLogFile` function, as it rotates existing logs. (basically when you run start-cluster, stop-cluster, start-cluster, the old log files will have a `.1` in the end)
With your change, now the .1 file is empty on the second start.
I don't have a good idea how to solve this right away.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-703604630
That sounds like a good plan. The more complexity we can remove from the bash scripts, the better.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
}, {
"hash" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970",
"triggerID" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"triggerType" : "PUSH"
}, {
"hash" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8042",
"triggerID" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"triggerType" : "PUSH"
}, {
"hash" : "1d1ced78d39676c69138e91b28bc138cb5fa71f1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1d1ced78d39676c69138e91b28bc138cb5fa71f1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bf35025687d8e1bd4e14e40444536ba0535e0d5e Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8042)
* 1d1ced78d39676c69138e91b28bc138cb5fa71f1 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-717209168
Yes, this comment is tracking the build results: https://github.com/apache/flink/pull/13457#issuecomment-697012704
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 917848f48bff5032c57bedab45d30dbbb66f664a UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
}, {
"hash" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 72242a97ba40dedecb60632bcd8bd7ac3d1459ff Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710)
* 14db2d58abff9a9b1ab6c997af66180c6301373c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-714562274
Thanks for the push. It seems that the YARN test is failing because of environment variable is not available in the tests.
It's probably easy to add to the YarnTestBase.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-713381865
For some reason, the environment variable was set when I tested it, not sure what was going on (I'm on macos with bash 5.0.17), maybe different behavior?
I agree to all your proposals!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] kottmann commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
kottmann commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-713200967
I changed it as suggested in the log4j.properties and the log4j-console.properties.
The MAX_LOG_FILE_NUMBER environment variable is not set for the started Java process, therefore I added an export to expose it, otherwise the default (10 as specified in the log4j properties) is always used.
The two defaults could be confusing, what do you think about not specifying it in the log4j properties? If there are no bugs in the shell scripts MAX_LOG_FILE_NUMBER should be used anyway (defaults to 5) or to the user configured value. If the variable fails to be set for some reason log4j defaults to 7.
I would still propose these changes before merge:
- Remove the default form the log4j properties files
- Increase the default from 5 to 10
- Add the on startup triggering policy for the log4j-console.properties file
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697009388
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-717723501
The Flink CI has passed as well: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=8382&view=results
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-708603602
> The log4j-cli.properties logs to a file and doesn't use a rolling policy, I can change that to rolling.
Let's leave that as-is. The logfile is overwritten on every cli start anyways.
> The log4j-console.properties uses the rolling policy and max files to keep is set to 10. Maybe it makes sense to do the same for log4j.properties.
Yes, I agree. That way, we bound the log files directory size to roughly 1GB, which is a good safety net.
> Then I noticed that there is a CoreOptions.FLINK_LOG_MAX config parameter, is it save to remove it? This will not be used anymore.
@zentol are we considering these config options part of the stable API? I don't think this setting is used much, if at all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 917848f48bff5032c57bedab45d30dbbb66f664a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 72242a97ba40dedecb60632bcd8bd7ac3d1459ff Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
}, {
"hash" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7710",
"triggerID" : "72242a97ba40dedecb60632bcd8bd7ac3d1459ff",
"triggerType" : "PUSH"
}, {
"hash" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970",
"triggerID" : "14db2d58abff9a9b1ab6c997af66180c6301373c",
"triggerType" : "PUSH"
}, {
"hash" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bf35025687d8e1bd4e14e40444536ba0535e0d5e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 14db2d58abff9a9b1ab6c997af66180c6301373c Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7970)
* bf35025687d8e1bd4e14e40444536ba0535e0d5e UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-697012704
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6807",
"triggerID" : "917848f48bff5032c57bedab45d30dbbb66f664a",
"triggerType" : "PUSH"
}, {
"hash" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423",
"triggerID" : "6200c8fb624b8a6415f983c705336524a9a8a342",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6200c8fb624b8a6415f983c705336524a9a8a342 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7423)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] kottmann commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
kottmann commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-717048006
I didn't manage to get the flink-yarn-tests to fail with running the tests on Linux. I tried to add the env as instructed. Can you test this again and see if it helps?
Indentation was by tab, sorry for that, changed it to space now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #13457: [FLINK-8357] Use rolling logs as default
Posted by GitBox <gi...@apache.org>.
rmetzger commented on pull request #13457:
URL: https://github.com/apache/flink/pull/13457#issuecomment-716746473
I guess you can set the env value here: https://github.com/apache/flink/blob/master/flink-yarn-tests/src/test/java/org/apache/flink/yarn/YarnTestBase.java#L689
Let me know if you don't have time to address this (I would also understand if you are a bit annoyed by this lengthy review) .. I can also do the last step if you want. I would like this change to be included in the Flink 1.12 release.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org