You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by tz...@apache.org on 2017/09/22 12:32:36 UTC
[1/2] flink git commit: [FLINK-7600] [kinesis] Shorten credential
update delay to avoid updateCredentials Exception
Repository: flink
Updated Branches:
refs/heads/release-1.3 571cda729 -> 37ded17a6
[FLINK-7600] [kinesis] Shorten credential update delay to avoid updateCredentials Exception
The updateCredentials delay is an ignorable warning that occurs due to
the fact that the default credential update delay is longer than the
await termination timeout when shutting down KPL.
See https://github.com/awslabs/amazon-kinesis-producer/issues/10 for
details.
This closes #4657.
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/cc0de848
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/cc0de848
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/cc0de848
Branch: refs/heads/release-1.3
Commit: cc0de8486b2c83bf03da0f8115df4b5ff72f6ed6
Parents: 571cda7
Author: Bowen Li <bo...@gmail.com>
Authored: Thu Sep 7 14:36:28 2017 -0700
Committer: Tzu-Li (Gordon) Tai <tz...@apache.org>
Committed: Fri Sep 22 11:50:22 2017 +0200
----------------------------------------------------------------------
.../streaming/connectors/kinesis/FlinkKinesisProducer.java | 5 +++++
1 file changed, 5 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/cc0de848/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisProducer.java
----------------------------------------------------------------------
diff --git a/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisProducer.java b/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisProducer.java
index 579bd6b..1cca822 100644
--- a/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisProducer.java
+++ b/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisProducer.java
@@ -181,6 +181,11 @@ public class FlinkKinesisProducer<OUT> extends RichSinkFunction<OUT> {
ProducerConfigConstants.AGGREGATION_MAX_COUNT, producerConfig.getAggregationMaxCount(), LOG));
}
+ // we explicitly lower the credential refresh delay (default is 5 seconds)
+ // to avoid a ignorable interruption warning that occurs when shutting down the
+ // KPL client. See https://github.com/awslabs/amazon-kinesis-producer/issues/10.
+ producerConfig.setCredentialsRefreshDelay(100);
+
producer = new KinesisProducer(producerConfig);
callback = new FutureCallback<UserRecordResult>() {
@Override
[2/2] flink git commit: [hotfix] Fix typo in Debugging Classloading
documentation
Posted by tz...@apache.org.
[hotfix] Fix typo in Debugging Classloading documentation
This closes #4685.
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/37ded17a
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/37ded17a
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/37ded17a
Branch: refs/heads/release-1.3
Commit: 37ded17a694a9550c9d3fed36d85713327d9e55f
Parents: cc0de84
Author: desktop <yu...@kddi-research.jp>
Authored: Wed Sep 20 12:38:26 2017 +0900
Committer: Tzu-Li (Gordon) Tai <tz...@apache.org>
Committed: Fri Sep 22 11:50:36 2017 +0200
----------------------------------------------------------------------
docs/monitoring/debugging_classloading.md | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/37ded17a/docs/monitoring/debugging_classloading.md
----------------------------------------------------------------------
diff --git a/docs/monitoring/debugging_classloading.md b/docs/monitoring/debugging_classloading.md
index 8c91e0f..e33cbb7 100644
--- a/docs/monitoring/debugging_classloading.md
+++ b/docs/monitoring/debugging_classloading.md
@@ -28,7 +28,7 @@ under the License.
## Overview of Classloading in Flink
When running Flink applications, the JVM will load various classes over time.
-These classes can be devided into two domains:
+These classes can be divided into two domains:
- The **Flink Framework** domain: This includes all code in the `/lib` directory in the Flink directory.
By default these are the classes of Apache Flink and its core dependencies.
@@ -41,12 +41,12 @@ The class loading behaves slightly different for various Flink setups:
**Standalone**
-When starting a the Flink cluster, the JobManagers and TaskManagers are started with the Flink framework classes in the
+When starting a Flink cluster, the JobManagers and TaskManagers are started with the Flink framework classes in the
classpath. The classes from all jobs that are submitted against the cluster are loaded *dynamically*.
**YARN**
-YARN classloading differs between single job deploymens and sessions:
+YARN classloading differs between single job deployments and sessions:
- When submitting a Flink job directly to YARN (via `bin/flink run -m yarn-cluster ...`), dedicated TaskManagers and
JobManagers are started for that job. Those JVMs have both Flink framework classes and user code classes in their classpath.
@@ -65,7 +65,7 @@ classes are loaded dynamically when the jobs are submitted.
## Avoiding Dynamic Classloading
All components (JobManger, TaskManager, Client, ApplicationMaster, ...) log their classpath setting on startup.
-They can be found as part of the environment information at the beginnign of the log.
+They can be found as part of the environment information at the beginning of the log.
When running a setup where the Flink JobManager and TaskManagers are exclusive to one particular job, one can put JAR files
directly into the `/lib` folder to make sure they are part of the classpath and not loaded dynamically.
@@ -76,7 +76,7 @@ Because the AppClassLoader is the parent of the FlinkUserCodeClassLoader (and Ja
result in classes being loaded only once.
For setups where the job's JAR file cannot be put to the `/lib` folder (for example because the setup is a session that is
-used by multiple jobs), it may still be posible to put common libraries to the `/lib` folder, and avoid dynamic class loading
+used by multiple jobs), it may still be possible to put common libraries to the `/lib` folder, and avoid dynamic class loading
for those.