You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by rm...@apache.org on 2019/08/07 12:25:57 UTC
[flink-web] branch asf-site updated (55198a2 -> 4c53ac0)
This is an automated email from the ASF dual-hosted git repository.
rmetzger pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git.
from 55198a2 Rebuild website
new dc08831 Add 'Code Style and Quality Guide'
new d6ad861 Address PR comments
new 8a173c9 Another round of addressing comments This closes #224
new 4c53ac0 rebuild site
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
README.md | 2 +-
_includes/code-style-navbar.md | 42 ++
_includes/navbar.html | 4 +-
.../code-style-and-quality-common.html | 620 +++++++++++++++++++++
.../code-style-and-quality-components.html | 415 ++++++++++++++
...html => code-style-and-quality-formatting.html} | 176 ++++--
.../contributing/code-style-and-quality-java.html | 385 +++++++++++++
...y.html => code-style-and-quality-preamble.html} | 78 ++-
.../code-style-and-quality-pull-requests.html | 352 ++++++++++++
...tion.html => code-style-and-quality-scala.html} | 174 ++++--
content/contributing/contribute-code.html | 6 +-
content/contributing/contribute-documentation.html | 2 +-
content/contributing/how-to-contribute.html | 2 +-
content/contributing/improve-website.html | 2 +-
content/contributing/reviewing-prs.html | 4 +-
.../code-style-and-quality-common.html | 618 ++++++++++++++++++++
.../code-style-and-quality-components.html | 413 ++++++++++++++
...html => code-style-and-quality-formatting.html} | 176 ++++--
.../contributing/code-style-and-quality-java.html | 383 +++++++++++++
...y.html => code-style-and-quality-preamble.html} | 78 ++-
...tion.html => code-style-and-quality-scala.html} | 174 ++++--
content/zh/contributing/contribute-code.html | 2 +-
.../zh/contributing/contribute-documentation.html | 2 +-
content/zh/contributing/how-to-contribute.html | 2 +-
content/zh/contributing/improve-website.html | 2 +-
content/zh/contributing/reviewing-prs.html | 2 +-
contributing/code-style-and-quality-common.md | 305 ++++++++++
contributing/code-style-and-quality-common.zh.md | 305 ++++++++++
contributing/code-style-and-quality-components.md | 143 +++++
.../code-style-and-quality-components.zh.md | 143 +++++
contributing/code-style-and-quality-formatting.md | 87 +++
.../code-style-and-quality-formatting.zh.md | 87 +++
contributing/code-style-and-quality-java.md | 101 ++++
contributing/code-style-and-quality-java.zh.md | 101 ++++
contributing/code-style-and-quality-preamble.md | 25 +
contributing/code-style-and-quality-preamble.zh.md | 25 +
.../code-style-and-quality-pull-requests.md | 90 +++
contributing/code-style-and-quality-scala.md | 75 +++
contributing/code-style-and-quality-scala.zh.md | 75 +++
contributing/code-style-and-quality.md | 10 -
contributing/code-style-and-quality.zh.md | 10 -
contributing/contribute-code.md | 4 +-
contributing/reviewing-prs.md | 2 +-
43 files changed, 5429 insertions(+), 275 deletions(-)
create mode 100644 _includes/code-style-navbar.md
create mode 100644 content/contributing/code-style-and-quality-common.html
create mode 100644 content/contributing/code-style-and-quality-components.html
copy content/contributing/{contribute-documentation.html => code-style-and-quality-formatting.html} (55%)
create mode 100644 content/contributing/code-style-and-quality-java.html
rename content/contributing/{code-style-and-quality.html => code-style-and-quality-preamble.html} (69%)
create mode 100644 content/contributing/code-style-and-quality-pull-requests.html
copy content/contributing/{contribute-documentation.html => code-style-and-quality-scala.html} (56%)
create mode 100644 content/zh/contributing/code-style-and-quality-common.html
create mode 100644 content/zh/contributing/code-style-and-quality-components.html
copy content/zh/contributing/{contribute-documentation.html => code-style-and-quality-formatting.html} (55%)
create mode 100644 content/zh/contributing/code-style-and-quality-java.html
rename content/zh/contributing/{code-style-and-quality.html => code-style-and-quality-preamble.html} (69%)
copy content/zh/contributing/{contribute-documentation.html => code-style-and-quality-scala.html} (56%)
create mode 100644 contributing/code-style-and-quality-common.md
create mode 100644 contributing/code-style-and-quality-common.zh.md
create mode 100644 contributing/code-style-and-quality-components.md
create mode 100644 contributing/code-style-and-quality-components.zh.md
create mode 100644 contributing/code-style-and-quality-formatting.md
create mode 100644 contributing/code-style-and-quality-formatting.zh.md
create mode 100644 contributing/code-style-and-quality-java.md
create mode 100644 contributing/code-style-and-quality-java.zh.md
create mode 100644 contributing/code-style-and-quality-preamble.md
create mode 100644 contributing/code-style-and-quality-preamble.zh.md
create mode 100644 contributing/code-style-and-quality-pull-requests.md
create mode 100644 contributing/code-style-and-quality-scala.md
create mode 100644 contributing/code-style-and-quality-scala.zh.md
delete mode 100644 contributing/code-style-and-quality.md
delete mode 100644 contributing/code-style-and-quality.zh.md
[flink-web] 01/04: Add 'Code Style and Quality Guide'
Posted by rm...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
rmetzger pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit dc08831e5c7da4563ccd6e6e2ee76c3df1ac1dc9
Author: Robert Metzger <rm...@apache.org>
AuthorDate: Mon Jul 8 17:50:29 2019 +0200
Add 'Code Style and Quality Guide'
---
_includes/code-style-navbar.md | 22 +
_includes/navbar.html | 4 +-
contributing/code-style-and-quality-general.md | 667 +++++++++++++++++++++
contributing/code-style-and-quality-preamble.md | 27 +
.../code-style-and-quality-pull-requests.md | 93 +++
contributing/code-style-and-quality.md | 10 -
6 files changed, 811 insertions(+), 12 deletions(-)
diff --git a/_includes/code-style-navbar.md b/_includes/code-style-navbar.md
new file mode 100644
index 0000000..101925b
--- /dev/null
+++ b/_includes/code-style-navbar.md
@@ -0,0 +1,22 @@
+<hr/>
+
+<div class="row">
+ <div class="col-sm-12" style="background-color: #f8f8f8;">
+ <h2>
+ <a href="{{ site.baseurl }}/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+
+ <span class="glyphicon glyphicon-chevron-right"></span>
+ <a href="{{ site.baseurl }}/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ <span class="glyphicon glyphicon-chevron-right"></span>
+ <a href="{{ site.baseurl }}/contributing/code-style-and-quality-general.html">
+ Code Style and Quality Guidelines
+ </a>
+ </h2>
+ </div>
+</div>
+
+<hr/>
\ No newline at end of file
diff --git a/_includes/navbar.html b/_includes/navbar.html
index 3f21a24..8b95b77 100755
--- a/_includes/navbar.html
+++ b/_includes/navbar.html
@@ -103,8 +103,8 @@
<li {% if page.url contains '/contributing/reviewing-prs.html' %} class="active"{% endif %}>
<a href="{{ baseurl_i18n }}/contributing/reviewing-prs.html">{{ site.data.i18n[page.language].review_code }}</a>
</li>
- <li {% if page.url contains '/contributing/code-style-and-quality.html' %} class="active"{% endif %}>
- <a href="{{ baseurl_i18n }}/contributing/code-style-and-quality.html">{{ site.data.i18n[page.language].code_style_guide }}</a>
+ <li {% if page.url contains '/contributing/code-style-and-quality-preamble.html' %} class="active"{% endif %}>
+ <a href="{{ baseurl_i18n }}/contributing/code-style-and-quality-preamble.html">{{ site.data.i18n[page.language].code_style_guide }}</a>
</li>
<li {% if page.url contains '/contributing/contribute-documentation.html' %} class="active"{% endif %}>
<a href="{{ baseurl_i18n }}/contributing/contribute-documentation.html">{{ site.data.i18n[page.language].contribute_docs }}</a>
diff --git a/contributing/code-style-and-quality-general.md b/contributing/code-style-and-quality-general.md
new file mode 100644
index 0000000..e782723
--- /dev/null
+++ b/contributing/code-style-and-quality-general.md
@@ -0,0 +1,667 @@
+---
+title: "Apache Flink Code Style and Quality Guide"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+## 1. Tools
+
+We recommend to follow the [IDE Setup Guide](https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java) to get IDE tooling configured.
+
+
+### Use inspections in IntelliJ
+
+* Import the inspections settings into the IDE (see IDE setup guide)
+ * TODO: Need to agree on a profile and export it (like checkstyle)
+* Write the code such that inspection warnings are addressed
+ * There are few exceptions where an inspection warning is not meaningful. In that case, suppress the inspection warning.
+
+
+### Warnings
+
+* We strive for zero warnings
+* Even though there are many warnings in existing code, new changes should not add any additional compiler warnings
+* If it is not possible to address the warning in a sane way (in some cases when working with generics) add an annotation to suppress the warning
+* When deprecating methods, check that this does not introduce additional warnings
+
+
+
+## 2. Comments And Code Readability
+
+
+### Comments
+
+**Golden rule: Comment as much as necessary to support code understanding, but don’t add redundant information.**
+
+Think about
+
+* <span style="text-decoration:underline;">What</span> is the code doing?
+* <span style="text-decoration:underline;">How</span> does the code do this?
+* <span style="text-decoration:underline;">Why</span> is the code like that?
+
+The code alone should explain as much as possible the “<span style="text-decoration:underline;">what</span>” and the “<span style="text-decoration:underline;">how</span>”
+
+* Use JavaDocs to describe the roles of classes and the contracts of methods, in cases where the contract is not obvious or intuitive from the method name (the “what”).
+* The flow of the code should give a good description of the “how”.
+Think of variable and method names as part of the code documenting itself.
+* It often makes reading the code easier if larger blocks that form a unit are moved into a private method with a descriptive name of what that block is doing
+
+In-code comments help explain the <span style="text-decoration:underline;">“why”</span>
+
+* For example `// this specific code layout helps the JIT to better do this or that`
+* Or `// nulling out this field here means future write attempts are fail-fast`
+* Or `// for arguments with which this method is actually called, this seemingly naive approach works actually better than any optimized/smart version`
+
+In-code comments should not state redundant information about the “what” and “how” that is already obvious in the code itself.
+
+JavaDocs should not state meaningless information (just to satisfy the Checkstyle checker).
+
+__Don’t:__
+
+```
+/**
+ * The symbol expression.
+ */
+public class CommonSymbolExpression {}
+```
+__Do:__
+
+```
+/**
+ * An expression that wraps a single specific symbol.
+ * A symbol could be a unit, an alias, a variable, etc.
+ */
+public class CommonSymbolExpression {}
+```
+
+
+### Branches and Nesting
+
+Avoid deep nesting of scopes, by flipping the if condition and exiting early.
+
+__Don’t:__
+
+```
+if (a) {
+ if (b) {
+ if (c) {
+ the main path
+ }
+ }
+}
+```
+
+__Do__
+
+```
+if (!a) {
+ return ..
+}
+
+if (!b) {
+ return ...
+}
+
+if (!c) {
+ return ...
+}
+
+the main path
+```
+
+
+## 3. Design and Structure
+
+While it is hard to exactly specify what constitutes a good design, there are some properties that can serve as a _litmus test_ for a good design. If these properties are given, the chances are good that the design is going into a good direction. If these properties cannot be achieved, there is a high probability that the design is flawed.
+
+
+### Immutability and Eager Initialization
+
+1. Try to use immutable types where possible, especially for APIs, messages, identifiers, properties, configuration, etc.
+2. A good general approach is to try and make as many fields of a class `final` as possible.
+3. Classes that are used as keys in maps should be strictly immutable and only have `final` fields (except maybe auxiliary fields, like lazy cached hash codes).
+4. Eagerly initialize classes. There should be no `init()` or `setup()` methods. Once the constructor completes, the object should be usable.
+
+
+### Nullability of the Mutable Parts
+
+For nullability, the Flink codebase aims to follow these conventions:
+
+* Fields, parameters, and return types are always non-null, unless indicated otherwise
+* All fields, parameters and method types that can be null should be annotated with `@javax.annotation.Nullable`.
+That way, you get warnings from IntelliJ about all sections where you have to reason about potential null values.
+* For all mutable (non-final) fields that are not annotated, the assumption is that while the field value changes, there always is a value.
+ * This should be double check whether these can in fact not be null throughout the lifetime of the object.
+
+_Note: This means that `@Nonnull` annotations are usually not necessary, but can be used in certain cases to override a previous annotation, or to point non-nullability out in a context where one would expect a nullable value._
+
+`Optional` is a good solution as a return type for method that may or may not have a result, so nullable return types are good candidates to be replaced with `Optional`.
+For fields and parameters, `Optional` is disputed in Java and most parts of the Flink code case don’t use optional for fields.
+
+
+### Avoid Code Duplication
+
+1. Whenever you are about to copy/paste some code, or reproduce a similar type of functionality in a different place, think about the ways how to refactor/reuse/abstract the changes to avoid the duplication.
+2. Common behavior between different specializations should be shared in a common component (or a shared superclass).
+3. Always use “private static final” constants instead of duplicating strings or other special values at different locations. Constants should be declared in the top member area of a class.
+
+
+### Design for Testability
+
+Code that is easily testable typically has good separation of concerns and is structured to be reusable outside the original context (by being easily reusable in tests).
+
+A good summary or problems / symptoms and recommended refactoring is in this PDF.
+Please note that while the examples in the PDF often use a dependency injection framework (Guice), it works in the same way without such a framework.[^1]
+
+[http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf](http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf)
+
+Here is a compact summary of the most important aspects.
+
+
+**Inject dependencies**
+
+Reusability becomes easier if constructors don’t create their dependencies (the objects assigned to the fields), but accept them as parameters.
+
+* Effectively, constructors should have no `new` keyword.
+* Exceptions are creating a new empty collection (`new ArrayList<>()`) or similar auxiliary fields (objects that have only primitive dependencies).
+
+To make instantiation easy / readable, add factory methods or additional convenience constructors to construct whole object with dependencies.
+
+In no case should it ever be required to use a reflection or a “Whitebox” util to change the fields of an object in a test, or to use PowerMock to intercept a “new” call and supply a mock.
+
+
+**Avoid “too many collaborators”**
+
+If you have to take a big set of other components into account during testing (“too many collaborators”), consider refactoring.
+
+The component/class you want to test probably depends on another broad component (and its implementation), rather than on the minimal interface (abstraction) required for its work.
+
+In that case, segregate the interfaces (factor out the minimal required interface) and supply a test stub in that case.
+
+
+* For example, if testing a S3RecoverableMultiPartUploader requires actual S3 access
+then the S3 access should be factored out into an interface and test should replace it by a test stub
+* This naturally requires to be able to inject dependencies (see above)
+
+⇒ Please note that these steps often require more effort in implementing tests (factoring out interfaces, creating dedicated test stubs), but make the tests more resilient to changes in other components, i.e., you do not need to touch the tests when making unrelated changes.
+
+
+**Write targeted tests**
+
+* <span style="text-decoration:underline;">Test contracts not implementations</span>: Test that after a sequence of actions, the components are in a certain state, rather than testing that the components followed a sequence of internal state modifications.
+ * For example, a typical antipattern is to check whether one specific method was called as part of the test
+* A way to enforce this is to try to follow the _Arrange_, _Act_, _Assert_ test structure when writing a unit test ([https://xp123.com/articles/3a-arrange-act-assert/](https://xp123.com/articles/3a-arrange-act-assert/))
+
+ This helps to communicate the intention of the test (what is the scenario under test) rather than the mechanics of the tests. The technical bits go to a static methods at the bottom of the test class.
+
+ Example of tests in Flink that follow this pattern are:
+
+ * [https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java](https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java)
+ * [https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java](https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java)
+
+
+**Avoid Mockito - Use reusable test implementations**
+
+* Mockito-based tests tend to be costly to maintain in the long run by encouraging duplication of functionality and testing for implementation rather than effect
+ * More details: [https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8](https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8)
+* Instead, create reusable test implementations and utilities
+ * That way, when some class changes, we only have to update a few test utils or mocks
+
+
+### Performance Awareness
+
+We can conceptually distinguish between code that “coordinates” and code that “processes data”. Code that coordinates should always favor simplicity and cleanness. Data processing code is highly performance critical and should optimize for performance.
+
+That means still applying the general idea of the sections above, but possibly forgoing some aspects in some place, in order to achieve higher performance.
+
+
+**Which code paths are Data Processing paths?**
+
+* <span style="text-decoration:underline;">Per-record code paths:</span> Methods and code paths that are called for each record. Found for example in Connectors, Serializers, State Backends, Formats, Tasks, Operators, Metrics, runtime data structures, etc.
+* <span style="text-decoration:underline;">I/O methods:</span> Transferring messages or chunks of data in buffers. Examples are in the RPC system, Network Stack, FileSystems, Encoders / Decoders, etc.
+
+
+**Things that performance critical code may do that we would otherwise avoid**
+
+* Using (and reusing) mutable objects to take pressure off the GC (and sometimes help with cache locality), thus forgoing the strive for immutability.
+* Using primitive types, arrays of primitive types, or MemorySegment/ByteBuffer and encoding meaning into the primitive types and byte sequences, rather than encapsulating the behavior in dedicated classes and using objects.
+* Structuring the code to amortize expensive work (allocations, lookups, virtual method calls, …) across multiple records, for example by doing the work once per buffer/bundle/batch.
+* Code layout optimized for the JIT rather than for readability. Examples are inlining fields from other classes (in cases where it is doubtful whether the JIT would do that optimization at runtime), or structuring code to help the JIT compiler with inlining, loop unrolling, vectorization, etc.
+
+
+
+## 4. Concurrency and Threading
+
+**Most code paths should not require any concurrency.** The right internal abstractions should obviate the need for concurrency in almost all cases.
+
+* The Flink core and runtime use concurrency to provide these building blocks.
+Examples are in the RPC system, Network Stack, in the Task’s mailbox model, or some predefined Source / Sink utilities.
+* We are not fully there, but any new addition that introduces implements its own concurrency should be under scrutiny, unless it falls into the above category of core system building blocks.
+* Contributors should reach out to committers if they feel they need to implement concurrent code to see if there is an existing abstraction/building-block, or if one should be added.
+
+
+**When developing a component think about threading model and synchronization points ahead.**
+
+* For example: single threaded, blocking, non-blocking, synchronous, asynchronous, multi threaded, thread pool, message queues, volatile, synchronized block/methods, mutexes, atomics, callbacks, …
+* Getting those things right and thinking about them ahead is even more important than designing classes interfaces/responsibilities, since it’s much harder to change later on.
+
+
+**Try to avoid using threads all together if possible in any way.**
+
+* If you feel you have a case for spawning a thread, point this out in the pull request as something to be explicitly reviewed.
+
+
+**Be aware that using threads is in fact much harder than it initially looks**
+
+* Clean shutdown of threads is very tricky.
+* Handling interruptions in a rock solid fashion (avoid both slow shutdown and live locks) requires almost a Java Wizard
+* Ensuring clean error propagation out of threads in all cases needs thorough design.
+* Complexity of multi-threaded application/component/class grows exponentially, with each additional synchronisation point/block/critical section. Your code initially might be easy enough to understand, but can quickly grow beyond that point.
+* Proper testing of multithreaded code is basically impossible, while alternative approaches (like asynchronous code, non-blocking code, actor model with message queues) are quite easy to test.
+* Usually multi-threaded code is often even less efficient compared to alternative approaches on modern hardware.
+
+
+**Be aware of the java.util.concurrent.CompletableFuture**
+
+* Like with other concurrent code, there should rarely be the need to use a CompletableFuture
+* Completing a future would also complete on the calling thread any chained futures that are waiting for the result to be completed, unless a completion executor specified explicitly.
+* This can be intentional, if the entire execution should be synchronous / single-threaded, as for example in parts of the Scheduler / ExecutionGraph.
+ * Flink even makes use of a “main-thread executor” to allow calling chained handlers in the same thread as a single-threaded RPC endpoint runs
+* This can be unexpected, if the thread that completes the future is a sensitive thread.
+ * It may be better to use `CompletableFuture.supplyAsync(value, executor)` in that case, instead of `future.complete(value)` when an executor is available
+* When blocking on a future awaiting completion, always supply a timeout for a result instead of waiting indefinitely, and handle timeouts explicitly.
+* Use `CompletableFuture.allOf()`/`anyOf()`, `ExecutorCompletionService`, or `org.apache.flink.runtime.concurrent.FutureUtils#waitForAll` if you need to wait for: all the results/any of the results/all the results but handled by (approximate) completion order.
+
+
+
+
+## 5. Dependencies and Modules
+
+* **Keep the dependency footprint small**
+ * The more dependencies the harder it gets for the community to manage them as a whole.
+ * Dependency management includes dependency conflicts, maintaining licenses and related notices, and handling security vulnerabilities.
+ * Discuss whether the dependency should be shaded/relocated to avoid future conflicts.
+* **Don’t add a dependency for just one method**
+ * Use Java built-in means if possible.
+ * If the method is Apache-licensed, you can copy the method into a Flink utility class with proper attribution.
+* **Declaration of dependencies**
+ * Declare dependencies that you explicitly rely on, whether it provides classes you directly import and use or it's something that provides a service you directly use, like Log4J.
+ * Transitive dependencies should only supply dependencies that are needed at runtime but that you don't use yourself.
+ * [[source](https://stackoverflow.com/questions/15177661/maven-transitive-dependencies)]
+* **Location of classes in the Maven modules**
+ * Whenever you create a new class, think about where to put it.
+ * A class might be used by multiple modules in the future and might belong into a `common` module in this case.
+
+
+## 6. Java Language Features and Libraries
+
+
+### Preconditions and Log Statements
+
+* Never concatenate strings in the parameters
+ * <span style="text-decoration:underline;">Don’t:</span> `Preconditions.checkState(value <= threshold, "value must be below " + threshold)`
+ * <span style="text-decoration:underline;">Don’t:</span> `LOG.debug("value is " + value)`
+ * <span style="text-decoration:underline;">Do:</span> `Preconditions.checkState(value <= threshold, "value must be below %s", threshold)`
+ * <span style="text-decoration:underline;">Do:</span> `LOG.debug("value is {}", value)`
+
+
+### Generics
+
+* **No raw types:** Do not use raw types, unless strictly necessary (sometimes necessary for signature matches, arrays).
+* **Suppress warnings for unchecked conversions:** Add annotations to suppress warnings, if they cannot be avoided (such as “unchecked”, or “serial”). Otherwise warnings about generics flood the build and drown relevant warnings.
+
+
+### equals() / hashCode()
+
+* **equals() / hashCode() should be added when they are well defined only.**
+* They should **not be added to enable a simpler assertion in tests** when they are not well defined. Use hamcrest matchers in that case: [https://github.com/junit-team/junit4/wiki/matchers-and-assertthat](https://github.com/junit-team/junit4/wiki/matchers-and-assertthat)
+* A common indicator that the methods are not well defined is when they take a subset of the fields into account (other than fields that are purely auxiliary).
+* When the methods take mutable fields into account, you often have a design issue. The `equals()`/`hashCode()` methods suggest to use the type as a key, but the signatures suggest it is safe to keep mutating the type.
+
+
+### Java Serialization
+
+* **Do not use Java Serialization for anything !!!**
+* **Do not use Java Serialization for anything !!! !!!**
+* **Do not use Java Serialization for anything !!! !!! !!!**
+* Internal to Flink, Java serialization is used to transport messages and programs through RPC. This is the only case where we use Java serialization. Because of that, some classes need to be serializable (if they are transported via RPC).
+* **Serializable classes must define a Serial Version UID:**
+
+ `private static final long serialVersionUID = 1L;`
+* **The Serial Version UID for new classes should start at 1** and should generally be bumped on every incompatible change to the class according to the Java serialization compatibility definition (i.e: changing the type of a field, or moving the position of a class in the class hierarchy).
+
+
+### Java Reflection
+
+**Avoid using Java’s Reflection API**
+
+* Java’s Reflection API can be a very useful tool in certain cases but in all cases it is a hack and one should research for alternatives. The only cases where Flink should use reflection are
+ * Dynamically loading implementations from another module (like webUI, additional serializers, pluggable query processors).
+ * Extracting types inside the TypeExtractor class. This is fragile enough any should not be done outside the TypeExtractor class.
+ * Some cases of cross-JDK version features, where we need to use reflection because we cannot assume a class/method to be present in all versions.
+* If you need reflection for accessing methods or fields in tests, it usually indicates some deeper architectural issues, like wrong scoping, bad separation of concerns, or that there is no clean way to provide components / dependencies to the class that is tested
+
+
+### Collections
+
+* **ArrayList and ArrayDeque are almost always superior to LinkedList**, except when frequently insert and deleting in the middle of the list
+* **For Maps, avoid patterns that require multiple lookups**
+ * `contains()` before `get()` → `get()` and check null
+ * `contains()` before `put()` → `putIfAbsent()` or `computeIfAbsent()`
+ * Iterating over keys, getting values → iterate over `entrySet()`
+
+
+### Lambdas
+
+* Prefer non-capturing lambdas (lambdas that do not contain references to the outer scope). Capturing lambdas need to create a new object instance for every call. Non-capturing lambdas can use the same instance for each invocation.
+
+ **don’t:**
+ ```
+ map.computeIfAbsent(key, x -> key.toLowerCase())
+ ```
+
+ **do:**
+ ```
+ map.computeIfAbsent(key, k -> k.toLowerCase());
+ ```
+
+* Consider method references instead of inline lambdas
+
+ **don’t**:
+ ```
+ map.computeIfAbsent(key, k-> Loader.load(k));
+ ```
+
+ **do:**
+ ```
+ map.computeIfAbsent(key, Loader::load);
+ ```
+
+
+### Java Streams
+
+* Avoid Java Streams in any performance critical code.
+* The main motivation to use Java Streams would be to improve code readability. As such, they can be a good match in parts of the code that are not data-intensive, but deal with coordination..
+* Even in the latter case, try to limit the scope to a method, or a few private methods within an internal class.
+
+
+## 7. Scala Language Features
+
+### Where to use (and not use) Scala
+
+**We use Scala for Scala APIs or pure Scala Libraries.**
+
+**We do not use Scala in the core APIs and runtime components. We aim to remove existing Scala use (code and dependencies) from those components.**
+
+⇒ This is not because we do not like Scala, it is a consequence of “the right tool for the right job” approach (see below).
+
+For APIs, we develop the foundation in Java, and layer Scala on top.
+
+* This has traditionally given the best interoperability for both Java and Scala
+* It does mean dedicated effort to keep the Scala API up to date
+
+Why don’t we use Scala in the core APIs and runtime?
+
+* The past has shown that Scala evolves too quickly with tricky changes in functionality. Each Scala version upgrade was a rather big effort process for the Flink community.
+* Scala does not always interact nicely with Java classes, e.g. Scala’s visibility scopes work differently and often expose more to Java consumers than desired
+* Scala adds an additional layer of complexity to artifact/dependency management.
+ * We may want to keep Scala dependent libraries like Akka in the runtime, but abstract them via an interface and load them in a separate classloader, to keep them shielded and avoid version conflicts.
+* Scala makes it very easy for knowledgeable Scala programmers to write code that is very hard to understand for programmers that are less knowledgeable in Scala. That is especially tricky for an open source project with a broad community of diverse experience levels. Working around this means restricting the Scala feature set by a lot, which defeats a good amount of the purpose of using Scala in the first place.
+
+
+### API Parity
+
+Keep Java API and Scala API in sync in terms of functionality and code quality.
+
+The Scala API should cover all the features of the Java APIs as well.
+
+Scala APIs should have a “completeness test”, like the following example from the DataStream API: [https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala](https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala)
+
+
+### Language Features
+
+* **Avoid Scala implicits.**
+ * Scala’s implicits should only be used for user-facing API improvements such as the Table API expressions or type information extraction.
+ * Don’t use them for internal “magic”.
+* **Add explicit types for class members.**
+ * Don’t rely on implicit type inference for class fields and methods return types:
+
+ **Don’t:**
+ ```
+ var expressions = new java.util.ArrayList[String]()
+ ```
+
+ **Do:**
+ ```
+ var expressions: java.util.List[String] = new java.util.ArrayList[]()
+ ```
+
+ * Type inference for local variables on the stack is fine.
+* **Use strict visibility.**
+ * Avoid Scala’s package private features (such as private[flink]) and use regular private/protected instead.
+ * Keep in mind that `private[flink]` and `protected` members are public in Java.
+ * Keep in mind that `private[flink]` still exposes all members in Flink provided examples.
+
+
+### Coding Formatting
+
+**Use line wrapping to structure your code.**
+
+* Scala’s functional nature allows for long transformation chains (`x.map().map().foreach()`).
+* In order to force implementers to structure their code, the line length is therefore limited to 100 characters.
+* Use one line per transformation for better maintainability.
+
+
+## 8. Component Specific Guidelines
+
+_Additional guidelines about changes in specific components._
+
+
+### Configuration Changes
+
+Where should the config option go?
+
+* <span style="text-decoration:underline;">‘flink-conf.yaml’:</span> All configuration that pertains to execution behavior that one may want to standardize across jobs. Think of it as parameters someone would set wearing an “ops” hat, or someone that provides a stream processing platform to other teams.
+
+* <span style="text-decoration:underline;">‘ExecutionConfig’</span>: Parameters specific to an individual Flink application, needed by the operators during execution. Typical examples are watermark interval, serializer parameters, object reuse.
+* <span style="text-decoration:underline;">ExecutionEnvironment (in code)</span>: Everything that is specific to an individual Flink application and is only needed to build program / dataflow, not needed inside the operators during execution.
+
+How to name config keys:
+
+* Config key names should be hierarchical.
+ Think of the configuration as nested objects (JSON style)
+
+ ```
+ taskmanager: {
+ jvm-exit-on-oom: true,
+ network: {
+ detailed-metrics: false,
+ request-backoff: {
+ initial: 100,
+ max: 10000
+ },
+ memory: {
+ fraction: 0.1,
+ min: 64MB,
+ max: 1GB,
+ buffers-per-channel: 2,
+ floating-buffers-per-gate: 16
+ }
+ }
+ }
+ ```
+
+* The resulting config keys should hence be:
+
+ **NOT** `"taskmanager.detailed.network.metrics"`
+
+ **But rather** `"taskmanager.network.detailed-metrics"`
+
+
+### Connectors
+
+Connectors are historically hard to implement and need to deal with many aspects of threading, concurrency, and checkpointing.
+
+As part of [[FLIP-27]](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface) we are working on making this much simpler for sources. New sources should not have to deal with any aspect of concurrency/threading and checkpointing any more.
+
+A similar FLIP can be expected for sinks in the near future.
+
+
+### Examples
+
+Examples should be self-contained and not require systems other than Flink to run. Except for examples that show how to use specific connectors, like the Kafka connector. Sources/sinks that are ok to use are `StreamExecutionEnvironment.socketTextStream`, which should not be used in production but is quite handy for exploring how things work, and file-based sources/sinks. (For streaming, there is the continuous file source)
+
+Examples should also not be pure toy-examples but strike a balance between real-world code and purely abstract examples. The WordCount example is quite long in the tooth by now but it’s a good showcase of simple code that highlights functionality and can do useful things.
+
+Examples should also be heavy in comments. They should describe the general idea of the example in the class-level Javadoc and describe what is happening and what functionality is used throughout the code. The expected input data and output data should also be described.
+
+Examples should include parameter parsing, so that you can run an example (from the Jar that is created for each example using `bin/flink run path/to/myExample.jar --param1 … --param2`.
+
+
+### Table & SQL API
+
+
+#### Semantics
+
+**The SQL standard should be the main source of truth.**
+
+* Syntax, semantics, and features should be aligned with SQL!
+* We don’t need to reinvent the wheel. Most problems have already been discussed industry-wide and written down in the SQL standard.
+* We rely on the newest standard (SQL:2016 or ISO/IEC 9075:2016 when writing this document [[download]](https://standards.iso.org/ittf/PubliclyAvailableStandards/c065143_ISO_IEC_TR_19075-5_2016.zip) ). Not every part is available online but a quick web search might help here.
+
+Discuss divergence from the standard or vendor-specific interpretations.
+
+* Once a syntax or behavior is defined it cannot be undone easily.
+* Contributions that need to extent or interpret the standard need a thorough discussion with the community.
+* Please help committers by performing some initial research about how other vendors such as Postgres, Microsoft SQL Server, Oracle, Hive, Calcite, Beam are handling such cases.
+
+
+Consider the Table API as a bridge between the SQL and Java/Scala programming world.
+
+* The Table API is an Embedded Domain Specific Language for analytical programs following the relational model.
+It is not required to strictly follow the SQL standard in regards of syntax and names, but can be closer to the way a programming language would do/name functions and features, if that helps make it feel more intuitive.
+* The Table API might have some non-SQL features (e.g. map(), flatMap(), etc.) but should nevertheless “feel like SQL”. Functions and operations should have equal semantics and naming if possible.
+
+
+#### Common mistakes
+
+* Support SQL’s type system when adding a feature.
+ * A SQL function, connector, or format should natively support most SQL types from the very beginning.
+ * Unsupported types lead to confusion, limit the usability, and create overhead by touching the same code paths multiple times.
+ * For example, when adding a `SHIFT_LEFT` function, make sure that the contribution is general enough not only for `INT` but also `BIGINT` or `TINYINT`.
+
+
+#### Testing
+
+Test for nullability.
+
+* SQL natively supports `NULL` for almost every operation and has a 3-valued boolean logic.
+* Make sure to test every feature for nullability as well.
+
+
+Avoid full integration tests
+
+* Spawning a Flink mini-cluster and performing compilation of generated code for a SQL query is expensive.
+* Avoid integration tests for planner tests or variations of API calls.
+* Instead, use unit tests that validate the optimized plan which comes out of a planner. Or test the behavior of a runtime operator directly.
+
+
+#### Compatibility
+
+Don’t introduce physical plan changes in minor releases!
+
+* Backwards compatibility for state in streaming SQL relies on the fact that the physical execution plan remains stable. Otherwise the generated Operator Names/IDs change and state cannot be matched and restored.
+* Every bug fix that leads to changes in the optimized physical plan of a streaming pipeline hences breaks compatibility.
+* As a consequence, changes of the kind that lead to different optimizer plans can only be merged in major releases for now.
+
+
+#### Scala / Java interoperability (legacy code parts)
+
+Keep Java in mind when designing interfaces.
+
+* Consider whether a class will need to interact with a Java class in the future.
+* Use Java collections and Java Optional in interfaces for a smooth integration with Java code.
+* Don’t use features of case classes such as .copy() or apply() for construction if a class is subjected to be converted to Java.
+* Pure Scala user-facing APIs should use pure Scala collections/iterables/etc. for natural and idiomatic (“scalaesk”) integration with Scala.
+
+
+## 9. Java Code Formatting Style
+
+We recommend to set up the IDE to automatically check the code style. Please follow the [IDE setup guide](https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java) for that.
+
+
+### License
+
+* **Apache license headers.** Make sure you have Apache License headers in your files. The RAT plugin is checking for that when you build the code.
+
+### Imports
+
+* **Empty line before and after package declaration.**
+* **No unused imports.**
+* **No redundant imports.**
+* **No wildcard imports.** They can cause problems when adding to the code and in some cases even during refactoring.
+* **Import order.** Imports must be ordered alphabetically, grouped into the following blocks, with each block separated by an empty line:
+ * <imports from org.apache.flink.*>
+ * <imports from org.apache.flink.shaded.*>
+ * <imports from other libraries>
+ * <imports from javax.*>
+ * <imports from java.*>
+ * <imports from scala.*>
+ * <static imports>
+
+
+### Naming
+
+* **Package names must start with a letter, and must not contain upper-case letters or special characters.**
+ **Non-private static final fields must be upper-case, with words being separated by underscores.**(`MY_STATIC_VARIABLE`)
+* **Non-static fields/methods must be in lower camel case.** (`myNonStaticField`)
+
+
+### Whitespaces
+
+* **Tabs vs. spaces.** We are using tabs for indentation, not spaces.
+We are aware that spaces are a bit nicer; it just happened to be that we started with tabs a long time ago (because Eclipse’s default style used tabs then), and we tried to keep the code base homogeneous (not mix tabs and spaces).
+* **No trailing whitespace.**
+* **Spaces around operators/keywords.** Operators (`+`, `=`, `>`, …) and keywords (`if`, `for`, `catch`, …) must have a space before and after them, provided they are not at the start or end of the line.
+
+
+### Braces
+
+* **Left curly braces (<code>{</code>) must not be placed on a new line.**
+* <strong>Right curly braces (<code>}</code>) must always be placed at the beginning of the line.</strong>
+* <strong>Blocks.</strong> All statements after <code>if</code>, <code>for</code>, <code>while</code>, <code>do</code>, … must always be encapsulated in a block with curly braces (even if the block contains one statement).
+
+
+### Javadocs
+
+* **All public/protected methods and classes must have a Javadoc.**
+* **The first sentence of the Javadoc must end with a period.**
+* **Paragraphs must be separated with a new line, and started with <p>.**
+
+
+### Modifiers
+
+* **No redundant modifiers.** For example, public modifiers in interface methods.
+* **Follow JLS3 modifier order.** Modifiers must be ordered in the following order: public, protected, private, abstract, static, final, transient, volatile, synchronized, native, strictfp.
+
+
+### Files
+
+* **All files must end with <code>\n</code>.**
+* <strong>File length must not exceed 3000 lines.</strong>
+
+
+### Misc
+
+* **Arrays must be defined Java-style.** For example, `public String[] array`.
+* **Use Flink Preconditions.** To increase homogeneity, consistently use the `org.apache.flink.Preconditions` methods `checkNotNull` and `checkArgument` rather than Apache Commons Validate or Google Guava.
+
+
+
+<hr>
+
+[^1]:
+ We are keeping such frameworks out of Flink, to make debugging easier and avoid dependency clashes.
diff --git a/contributing/code-style-and-quality-preamble.md b/contributing/code-style-and-quality-preamble.md
new file mode 100644
index 0000000..bc2a9aa
--- /dev/null
+++ b/contributing/code-style-and-quality-preamble.md
@@ -0,0 +1,27 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Preamble"
+---
+
+{% include code-style-navbar.md %}
+
+
+_Preamble_
+
+This is an attempt to capture the code and quality standard that we want to maintain.
+
+A code contribution (or any piece of code) can be evaluated in various ways: One set of properties is whether the code is correct and efficient. This requires solving the _logical or algorithmic problem_ correctly and well.
+
+Another set of properties is whether the code follows an intuitive design and architecture, whether it is well structured with right separation of concerns, and whether the code is easily understandable and makes its assumptions explicit. That set of properties requires solving the _software engineering problem_ well. A good solution implies that the code is easily testable, maintainable also by other people than the original authors (because it is harder to accidentally break), and effi [...]
+
+While the first set of properties has rather objective approval criteria, the second set of properties is much harder to assess, but is of high importance for an open source project like Apache Flink. To make the code base inviting to many contributors, to make contributions easy to understand for developers that did not write the original code, and to make the code robust in the face of many contributions, well engineered code is crucial.[^1] For well engineered code, it is easier to ke [...]
+
+
+This is of course not a full guide on how to write well engineered code. There is a world of big books that try to capture that. This guide is meant as a checklist of best practices, patterns, anti-patterns, and common mistakes that we observed in the context of developing Flink.
+
+A big part of high-quality open source contributions is about helping the reviewer to understand the contribution and double-check the implications, so an important part of this guide is about how to structure a pull request for review.
+
+
+<hr>
+
+[^1]:
+ In earlier days, we (the Flink community) did not always pay sufficient attention to this, making some components of Flink harder to evolve and to contribute to.
diff --git a/contributing/code-style-and-quality-pull-requests.md b/contributing/code-style-and-quality-pull-requests.md
new file mode 100644
index 0000000..1ab82c9
--- /dev/null
+++ b/contributing/code-style-and-quality-pull-requests.md
@@ -0,0 +1,93 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Pull Requests & Changes"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+**Rationale:** We ask contributors to put in a little bit of extra effort to bring pull requests into a state that they can be more easily and more thoroughly reviewed. This helps the community in many ways:
+
+
+* Reviews are much faster and thus contributions get merged sooner.
+* We can ensure higher code quality by overlooking fewer issues in the contributions.
+* Committers can review more contributions in the same time, which helps to keep up with the high rate of contributions that Flink is experiencing
+
+Please understand that contributions that do not follow this guide will take longer to review and thus will typically be picked up with lower priority by the community. That is not ill intend, it is due to the added complexity of reviewing unstructured Pull Requests.
+
+
+## 1. JIRA issue and Naming
+
+Make sure that the pull request corresponds to a [JIRA issue]([https://issues.apache.org/jira/projects/FLINK/issues](https://issues.apache.org/jira/projects/FLINK/issues)).
+
+Exceptions are ****hotfixes****, like fixing typos in JavaDocs or documentation files.
+
+
+Name the pull request in the form `[FLINK-XXXX] [component] Title of the pull request`, where `FLINK-XXXX` should be replaced by the actual issue number. The components should be the same as used in the JIRA issue.
+
+Hotfixes should be named for example `[hotfix] [docs] Fix typo in event time introduction` or `[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
+
+
+## 2. Description
+
+Please fill out the pull request template to describe the contribution. Please describe it such that the reviewer understands the problem and solution from the description, not only from the code.
+
+A stellar example of a well-described pull request is [https://github.com/apache/flink/pull/7264](https://github.com/apache/flink/pull/7264)
+
+Make sure that the description is adequate for the problem solved by PR. Small changes do not need a wall of text. In ideal cases, the problem was described in the Jira issue and the description be mostly copied from there.
+
+If additional open questions / issues were discovered during the implementation and you made a choice regarding those, describe them in the pull request text so that reviewers can double check the assumptions. And example is in [https://github.com/apache/flink/pull/8290](https://github.com/apache/flink/pull/8290) (Section “Open Architecture Questions”).
+
+
+## 3. Separate Refactoring, Cleanup and Independent Changes
+
+****NOTE: This is not an optimization, this is a critical requirement.****
+
+Pull Requests must put cleanup, refactoring, and core changes into separate commits. That way, the reviewer can look independently at the cleanup and refactoring and ensure that those changes to not alter the behavior. Then the reviewer can look at the core changes in isolation (without the noise of other changes) and ensure that this is a clean and robust change.
+
+Examples for changes that strictly need to go into a separate commit include
+
+
+
+* Cleanup, fixing style and warnings in pre-existing code
+* Renaming packages, classes, or methods
+* Moving code (to other packages or classes)
+* Refactoring structure or changing design patterns
+* Consolidating related tests or utilities
+* Changing the assumptions in existing tests (add a commit message that describes why the changed assumptions make sense).
+
+There should be no cleanup commits that fix issues that have been introduced in previous commits of the same PR. Commits should be clean in themselves.
+
+In addition, any larger contributions should split the changes into a set of independent changes that can be independently reviewed.
+
+Two great examples of splitting issues into separate commits are:
+
+* [https://github.com/apache/flink/pull/6692](https://github.com/apache/flink/pull/6692) (splits cleanup and refactoring from main changes)
+* [https://github.com/apache/flink/pull/7264](https://github.com/apache/flink/pull/7264) (splits also main changes into independently reviewable pieces)
+
+If a pull request does still contain big commits (e.g. a commit with more than 1000 changed lines), it might be worth thinking about how to split the commit into multiple subproblems, as in the example above.
+
+
+## 4. Commit Naming Conventions
+
+Commit messages should follow a similar pattern as the pull request as a whole:
+`[FLINK-XXXX] [component] Commit description`.
+
+In some cases, the issue might be a subtask here, and the component may be different from the Pull Request’s main component. For example, when the commit introduces an end-to-end test for a runtime change, the PR would be tagged as `[runtime]`, but the individual commit would be tagged as `[e2e]`.
+
+Examples for commit messages:
+
+* `[hotfix] Fix update_branch_version.sh to allow version suffixes`
+* `[hotfix] [table] Remove unused geometry dependency`
+* `[FLINK-11704] [tests] Improve AbstractCheckpointStateOutputStreamTestBase`
+* `[FLINK-10569] [runtime] Remove Instance usage in ExecutionVertexCancelTest`
+* `[FLINK-11702] [table-planner-blink] Introduce a new table type system`
+
+
+## 5. Changes to the observable behavior of the system
+
+Contributors should be aware of changes in their PRs that break the observable behavior of Flink in any way because in many cases such changes can break existing setups. Red flags that should raise questions while coding or in reviews with respect to this problem are for example:
+
+* Assertions have been changed to make tests pass again with the breaking change.
+* Configuration setting that must suddenly be set to (non-default) values to keep existing tests passing. This can happen in particular for new settings with a breaking default.
+* Existing scripts or configurations have to be adjusted.
diff --git a/contributing/code-style-and-quality.md b/contributing/code-style-and-quality.md
deleted file mode 100644
index ed25b86..0000000
--- a/contributing/code-style-and-quality.md
+++ /dev/null
@@ -1,10 +0,0 @@
----
-title: "Code Style and Quality Guide (PLACEHOLDER)"
----
-
-<br />
-<br />
-
-This page is a placeholder for an updated Code Style and Quality Guide to be posted here, once the [discussion on the mailing list](https://lists.apache.org/thread.html/461d0922787c62447c3baff3177a0c8f34fee7c95e2896e5ddf59691@%3Cdev.flink.apache.org%3E) has concluded.
-
-
[flink-web] 03/04: Another round of addressing comments This closes
#224
Posted by rm...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
rmetzger pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit 8a173c9b6ed0f72da4828add8eb37abd3b7be7ee
Author: Robert Metzger <rm...@apache.org>
AuthorDate: Thu Aug 1 15:16:49 2019 +0200
Another round of addressing comments
This closes #224
---
README.md | 2 +-
_includes/code-style-navbar.md | 44 +-
contributing/code-style-and-quality-common.md | 305 ++++++++++
contributing/code-style-and-quality-common.zh.md | 305 ++++++++++
contributing/code-style-and-quality-components.md | 143 +++++
.../code-style-and-quality-components.zh.md | 143 +++++
contributing/code-style-and-quality-formatting.md | 87 +++
.../code-style-and-quality-formatting.zh.md | 87 +++
contributing/code-style-and-quality-general.md | 667 ---------------------
contributing/code-style-and-quality-java.md | 101 ++++
contributing/code-style-and-quality-java.zh.md | 101 ++++
contributing/code-style-and-quality-preamble.zh.md | 25 +
.../code-style-and-quality-pull-requests.md | 17 +-
contributing/code-style-and-quality-scala.md | 75 +++
contributing/code-style-and-quality-scala.zh.md | 75 +++
contributing/code-style-and-quality.zh.md | 10 -
contributing/contribute-code.md | 4 +-
contributing/reviewing-prs.md | 2 +-
18 files changed, 1490 insertions(+), 703 deletions(-)
diff --git a/README.md b/README.md
index 15bdfe8..7d6845d 100644
--- a/README.md
+++ b/README.md
@@ -2,4 +2,4 @@
This repository contains the Flink website: https://flink.apache.org/.
-You find instructions for this repository here: https://flink.apache.org/improve-website.html.
\ No newline at end of file
+You find instructions for this repository here: https://flink.apache.org/contributing/improve-website.html.
\ No newline at end of file
diff --git a/_includes/code-style-navbar.md b/_includes/code-style-navbar.md
index 101925b..9fc9062 100644
--- a/_includes/code-style-navbar.md
+++ b/_includes/code-style-navbar.md
@@ -1,22 +1,42 @@
-<hr/>
-<div class="row">
- <div class="col-sm-12" style="background-color: #f8f8f8;">
- <h2>
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
<a href="{{ site.baseurl }}/contributing/code-style-and-quality-preamble.html">
Preamble
</a>
-
- <span class="glyphicon glyphicon-chevron-right"></span>
+ </li>
+ <li class="list-group-item">
<a href="{{ site.baseurl }}/contributing/code-style-and-quality-pull-requests.html">
Pull Requests & Changes
</a>
- <span class="glyphicon glyphicon-chevron-right"></span>
- <a href="{{ site.baseurl }}/contributing/code-style-and-quality-general.html">
- Code Style and Quality Guidelines
+ </li>
+ <li class="list-group-item">
+ <a href="{{ site.baseurl }}/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="{{ site.baseurl }}/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="{{ site.baseurl }}/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
</a>
- </h2>
- </div>
-</div>
+ </li>
+ <li class="list-group-item">
+ <a href="{{ site.baseurl }}/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="{{ site.baseurl }}/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
+
<hr/>
\ No newline at end of file
diff --git a/contributing/code-style-and-quality-common.md b/contributing/code-style-and-quality-common.md
new file mode 100644
index 0000000..05f703b
--- /dev/null
+++ b/contributing/code-style-and-quality-common.md
@@ -0,0 +1,305 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Common Rules"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+## 1. Tools
+
+We recommend to follow the [IDE Setup Guide](https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java) to get IDE tooling configured.
+
+
+<!---
+### Use inspections in IntelliJ
+
+* Import the inspections settings into the IDE (see IDE setup guide)
+ * TODO: Need to agree on a profile and export it (like checkstyle)
+* Write the code such that inspection warnings are addressed
+ * There are few exceptions where an inspection warning is not meaningful. In that case, suppress the inspection warning.
+-->
+
+### Warnings
+
+* We strive for zero warnings
+* Even though there are many warnings in existing code, new changes should not add any additional compiler warnings
+* If it is not possible to address the warning in a sane way (in some cases when working with generics) add an annotation to suppress the warning
+* When deprecating methods, check that this does not introduce additional warnings
+
+
+
+## 2. Comments And Code Readability
+
+
+### Comments
+
+**Golden rule: Comment as much as necessary to support code understanding, but don’t add redundant information.**
+
+Think about
+
+* <span style="text-decoration:underline;">What</span> is the code doing?
+* <span style="text-decoration:underline;">How</span> does the code do this?
+* <span style="text-decoration:underline;">Why</span> is the code like that?
+
+The code alone should explain as much as possible the “<span style="text-decoration:underline;">what</span>” and the “<span style="text-decoration:underline;">how</span>”
+
+* Use JavaDocs to describe the roles of classes and the contracts of methods, in cases where the contract is not obvious or intuitive from the method name (the “what”).
+* The flow of the code should give a good description of the “how”.
+Think of variable and method names as part of the code documenting itself.
+* It often makes reading the code easier if larger blocks that form a unit are moved into a private method with a descriptive name of what that block is doing
+
+In-code comments help explain the <span style="text-decoration:underline;">“why”</span>
+
+* For example `// this specific code layout helps the JIT to better do this or that`
+* Or `// nulling out this field here means future write attempts are fail-fast`
+* Or `// for arguments with which this method is actually called, this seemingly naive approach works actually better than any optimized/smart version`
+
+In-code comments should not state redundant information about the “what” and “how” that is already obvious in the code itself.
+
+JavaDocs should not state meaningless information (just to satisfy the Checkstyle checker).
+
+__Don’t:__
+
+```
+/**
+ * The symbol expression.
+ */
+public class CommonSymbolExpression {}
+```
+__Do:__
+
+```
+/**
+ * An expression that wraps a single specific symbol.
+ * A symbol could be a unit, an alias, a variable, etc.
+ */
+public class CommonSymbolExpression {}
+```
+
+
+### Branches and Nesting
+
+Avoid deep nesting of scopes, by flipping the if condition and exiting early.
+
+__Don’t:__
+
+```
+if (a) {
+ if (b) {
+ if (c) {
+ the main path
+ }
+ }
+}
+```
+
+__Do__
+
+```
+if (!a) {
+ return ..
+}
+
+if (!b) {
+ return ...
+}
+
+if (!c) {
+ return ...
+}
+
+the main path
+```
+
+
+## 3. Design and Structure
+
+While it is hard to exactly specify what constitutes a good design, there are some properties that can serve as a _litmus test_ for a good design. If these properties are given, the chances are good that the design is going into a good direction. If these properties cannot be achieved, there is a high probability that the design is flawed.
+
+
+### Immutability and Eager Initialization
+
+1. Try to use immutable types where possible, especially for APIs, messages, identifiers, properties, configuration, etc.
+2. A good general approach is to try and make as many fields of a class `final` as possible.
+3. Classes that are used as keys in maps should be strictly immutable and only have `final` fields (except maybe auxiliary fields, like lazy cached hash codes).
+4. Eagerly initialize classes. There should be no `init()` or `setup()` methods. Once the constructor completes, the object should be usable.
+
+
+### Nullability of the Mutable Parts
+
+For nullability, the Flink codebase aims to follow these conventions:
+
+* Fields, parameters, and return types are always non-null, unless indicated otherwise
+* All fields, parameters and method types that can be null should be annotated with `@javax.annotation.Nullable`.
+That way, you get warnings from IntelliJ about all sections where you have to reason about potential null values.
+* For all mutable (non-final) fields that are not annotated, the assumption is that while the field value changes, there always is a value.
+ * This should be double check whether these can in fact not be null throughout the lifetime of the object.
+
+_Note: This means that `@Nonnull` annotations are usually not necessary, but can be used in certain cases to override a previous annotation, or to point non-nullability out in a context where one would expect a nullable value._
+
+`Optional` is a good solution as a return type for method that may or may not have a result, so nullable return types are good candidates to be replaced with `Optional`.
+For fields and parameters, `Optional` is disputed in Java and most parts of the Flink code case don’t use optional for fields.
+
+
+### Avoid Code Duplication
+
+1. Whenever you are about to copy/paste some code, or reproduce a similar type of functionality in a different place, think about the ways how to refactor/reuse/abstract the changes to avoid the duplication.
+2. Common behavior between different specializations should be shared in a common component (or a shared superclass).
+3. Always use “private static final” constants instead of duplicating strings or other special values at different locations. Constants should be declared in the top member area of a class.
+
+
+### Design for Testability
+
+Code that is easily testable typically has good separation of concerns and is structured to be reusable outside the original context (by being easily reusable in tests).
+
+A good summary or problems / symptoms and recommended refactoring is in the PDF linked below.
+Please note that while the examples in the PDF often use a dependency injection framework (Guice), it works in the same way without such a framework.[^1]
+
+[http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf](http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf)
+
+Here is a compact summary of the most important aspects.
+
+
+**Inject dependencies**
+
+Reusability becomes easier if constructors don’t create their dependencies (the objects assigned to the fields), but accept them as parameters.
+
+* Effectively, constructors should have no `new` keyword.
+* Exceptions are creating a new empty collection (`new ArrayList<>()`) or similar auxiliary fields (objects that have only primitive dependencies).
+
+To make instantiation easy / readable, add factory methods or additional convenience constructors to construct whole object with dependencies.
+
+In no case should it ever be required to use a reflection or a “Whitebox” util to change the fields of an object in a test, or to use PowerMock to intercept a “new” call and supply a mock.
+
+
+**Avoid “too many collaborators”**
+
+If you have to take a big set of other components into account during testing (“too many collaborators”), consider refactoring.
+
+The component/class you want to test probably depends on another broad component (and its implementation), rather than on the minimal interface (abstraction) required for its work.
+
+In that case, segregate the interfaces (factor out the minimal required interface) and supply a test stub in that case.
+
+* For example, if testing a S3RecoverableMultiPartUploader requires actual S3 access
+then the S3 access should be factored out into an interface and test should replace it by a test stub
+* This naturally requires to be able to inject dependencies (see above)
+
+⇒ Please note that these steps often require more effort in implementing tests (factoring out interfaces, creating dedicated test stubs), but make the tests more resilient to changes in other components, i.e., you do not need to touch the tests when making unrelated changes.
+
+
+**Write targeted tests**
+
+* <span style="text-decoration:underline;">Test contracts not implementations</span>: Test that after a sequence of actions, the components are in a certain state, rather than testing that the components followed a sequence of internal state modifications.
+ * For example, a typical antipattern is to check whether one specific method was called as part of the test
+* A way to enforce this is to try to follow the _Arrange_, _Act_, _Assert_ test structure when writing a unit test ([https://xp123.com/articles/3a-arrange-act-assert/](https://xp123.com/articles/3a-arrange-act-assert/))
+
+ This helps to communicate the intention of the test (what is the scenario under test) rather than the mechanics of the tests. The technical bits go to a static methods at the bottom of the test class.
+
+ Example of tests in Flink that follow this pattern are:
+
+ * [https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java](https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java)
+ * [https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java](https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java)
+
+
+**Avoid Mockito - Use reusable test implementations**
+
+* Mockito-based tests tend to be costly to maintain in the long run by encouraging duplication of functionality and testing for implementation rather than effect
+ * More details: [https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8](https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8)
+* Instead, create reusable test implementations and utilities
+ * That way, when some class changes, we only have to update a few test utils or mocks
+
+
+### Performance Awareness
+
+We can conceptually distinguish between code that “coordinates” and code that “processes data”. Code that coordinates should always favor simplicity and cleanness. Data processing code is highly performance critical and should optimize for performance.
+
+That means still applying the general idea of the sections above, but possibly forgoing some aspects in some place, in order to achieve higher performance.
+
+
+**Which code paths are Data Processing paths?**
+
+* <span style="text-decoration:underline;">Per-record code paths:</span> Methods and code paths that are called for each record. Found for example in Connectors, Serializers, State Backends, Formats, Tasks, Operators, Metrics, runtime data structures, etc.
+* <span style="text-decoration:underline;">I/O methods:</span> Transferring messages or chunks of data in buffers. Examples are in the RPC system, Network Stack, FileSystems, Encoders / Decoders, etc.
+
+
+**Things that performance critical code may do that we would otherwise avoid**
+
+* Using (and reusing) mutable objects to take pressure off the GC (and sometimes help with cache locality), thus forgoing the strive for immutability.
+* Using primitive types, arrays of primitive types, or MemorySegment/ByteBuffer and encoding meaning into the primitive types and byte sequences, rather than encapsulating the behavior in dedicated classes and using objects.
+* Structuring the code to amortize expensive work (allocations, lookups, virtual method calls, …) across multiple records, for example by doing the work once per buffer/bundle/batch.
+* Code layout optimized for the JIT rather than for readability. Examples are inlining fields from other classes (in cases where it is doubtful whether the JIT would do that optimization at runtime), or structuring code to help the JIT compiler with inlining, loop unrolling, vectorization, etc.
+
+
+
+## 4. Concurrency and Threading
+
+**Most code paths should not require any concurrency.** The right internal abstractions should obviate the need for concurrency in almost all cases.
+
+* The Flink core and runtime use concurrency to provide these building blocks.
+Examples are in the RPC system, Network Stack, in the Task’s mailbox model, or some predefined Source / Sink utilities.
+* We are not fully there, but any new addition that introduces implements its own concurrency should be under scrutiny, unless it falls into the above category of core system building blocks.
+* Contributors should reach out to committers if they feel they need to implement concurrent code to see if there is an existing abstraction/building-block, or if one should be added.
+
+
+**When developing a component think about threading model and synchronization points ahead.**
+
+* For example: single threaded, blocking, non-blocking, synchronous, asynchronous, multi threaded, thread pool, message queues, volatile, synchronized block/methods, mutexes, atomics, callbacks, …
+* Getting those things right and thinking about them ahead is even more important than designing classes interfaces/responsibilities, since it’s much harder to change later on.
+
+
+**Try to avoid using threads all together if possible in any way.**
+
+* If you feel you have a case for spawning a thread, point this out in the pull request as something to be explicitly reviewed.
+
+
+**Be aware that using threads is in fact much harder than it initially looks**
+
+* Clean shutdown of threads is very tricky.
+* Handling interruptions in a rock solid fashion (avoid both slow shutdown and live locks) requires almost a Java Wizard
+* Ensuring clean error propagation out of threads in all cases needs thorough design.
+* Complexity of multi-threaded application/component/class grows exponentially, with each additional synchronisation point/block/critical section. Your code initially might be easy enough to understand, but can quickly grow beyond that point.
+* Proper testing of multithreaded code is basically impossible, while alternative approaches (like asynchronous code, non-blocking code, actor model with message queues) are quite easy to test.
+* Usually multi-threaded code is often even less efficient compared to alternative approaches on modern hardware.
+
+
+**Be aware of the java.util.concurrent.CompletableFuture**
+
+* Like with other concurrent code, there should rarely be the need to use a CompletableFuture
+* Completing a future would also complete on the calling thread any chained futures that are waiting for the result to be completed, unless a completion executor specified explicitly.
+* This can be intentional, if the entire execution should be synchronous / single-threaded, as for example in parts of the Scheduler / ExecutionGraph.
+ * Flink even makes use of a “main-thread executor” to allow calling chained handlers in the same thread as a single-threaded RPC endpoint runs
+* This can be unexpected, if the thread that completes the future is a sensitive thread.
+ * It may be better to use `CompletableFuture.supplyAsync(value, executor)` in that case, instead of `future.complete(value)` when an executor is available
+* When blocking on a future awaiting completion, always supply a timeout for a result instead of waiting indefinitely, and handle timeouts explicitly.
+* Use `CompletableFuture.allOf()`/`anyOf()`, `ExecutorCompletionService`, or `org.apache.flink.runtime.concurrent.FutureUtils#waitForAll` if you need to wait for: all the results/any of the results/all the results but handled by (approximate) completion order.
+
+
+
+
+## 5. Dependencies and Modules
+
+* **Keep the dependency footprint small**
+ * The more dependencies the harder it gets for the community to manage them as a whole.
+ * Dependency management includes dependency conflicts, maintaining licenses and related notices, and handling security vulnerabilities.
+ * Discuss whether the dependency should be shaded/relocated to avoid future conflicts.
+* **Don’t add a dependency for just one method**
+ * Use Java built-in means if possible.
+ * If the method is Apache-licensed, you can copy the method into a Flink utility class with proper attribution.
+* **Declaration of dependencies**
+ * Declare dependencies that you explicitly rely on, whether it provides classes you directly import and use or it's something that provides a service you directly use, like Log4J.
+ * Transitive dependencies should only supply dependencies that are needed at runtime but that you don't use yourself.
+ * [[source](https://stackoverflow.com/questions/15177661/maven-transitive-dependencies)]
+* **Location of classes in the Maven modules**
+ * Whenever you create a new class, think about where to put it.
+ * A class might be used by multiple modules in the future and might belong into a `common` module in this case.
+
+
+
+
+
+<hr>
+
+[^1]:
+ We are keeping such frameworks out of Flink, to make debugging easier and avoid dependency clashes.
diff --git a/contributing/code-style-and-quality-common.zh.md b/contributing/code-style-and-quality-common.zh.md
new file mode 100644
index 0000000..05f703b
--- /dev/null
+++ b/contributing/code-style-and-quality-common.zh.md
@@ -0,0 +1,305 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Common Rules"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+## 1. Tools
+
+We recommend to follow the [IDE Setup Guide](https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java) to get IDE tooling configured.
+
+
+<!---
+### Use inspections in IntelliJ
+
+* Import the inspections settings into the IDE (see IDE setup guide)
+ * TODO: Need to agree on a profile and export it (like checkstyle)
+* Write the code such that inspection warnings are addressed
+ * There are few exceptions where an inspection warning is not meaningful. In that case, suppress the inspection warning.
+-->
+
+### Warnings
+
+* We strive for zero warnings
+* Even though there are many warnings in existing code, new changes should not add any additional compiler warnings
+* If it is not possible to address the warning in a sane way (in some cases when working with generics) add an annotation to suppress the warning
+* When deprecating methods, check that this does not introduce additional warnings
+
+
+
+## 2. Comments And Code Readability
+
+
+### Comments
+
+**Golden rule: Comment as much as necessary to support code understanding, but don’t add redundant information.**
+
+Think about
+
+* <span style="text-decoration:underline;">What</span> is the code doing?
+* <span style="text-decoration:underline;">How</span> does the code do this?
+* <span style="text-decoration:underline;">Why</span> is the code like that?
+
+The code alone should explain as much as possible the “<span style="text-decoration:underline;">what</span>” and the “<span style="text-decoration:underline;">how</span>”
+
+* Use JavaDocs to describe the roles of classes and the contracts of methods, in cases where the contract is not obvious or intuitive from the method name (the “what”).
+* The flow of the code should give a good description of the “how”.
+Think of variable and method names as part of the code documenting itself.
+* It often makes reading the code easier if larger blocks that form a unit are moved into a private method with a descriptive name of what that block is doing
+
+In-code comments help explain the <span style="text-decoration:underline;">“why”</span>
+
+* For example `// this specific code layout helps the JIT to better do this or that`
+* Or `// nulling out this field here means future write attempts are fail-fast`
+* Or `// for arguments with which this method is actually called, this seemingly naive approach works actually better than any optimized/smart version`
+
+In-code comments should not state redundant information about the “what” and “how” that is already obvious in the code itself.
+
+JavaDocs should not state meaningless information (just to satisfy the Checkstyle checker).
+
+__Don’t:__
+
+```
+/**
+ * The symbol expression.
+ */
+public class CommonSymbolExpression {}
+```
+__Do:__
+
+```
+/**
+ * An expression that wraps a single specific symbol.
+ * A symbol could be a unit, an alias, a variable, etc.
+ */
+public class CommonSymbolExpression {}
+```
+
+
+### Branches and Nesting
+
+Avoid deep nesting of scopes, by flipping the if condition and exiting early.
+
+__Don’t:__
+
+```
+if (a) {
+ if (b) {
+ if (c) {
+ the main path
+ }
+ }
+}
+```
+
+__Do__
+
+```
+if (!a) {
+ return ..
+}
+
+if (!b) {
+ return ...
+}
+
+if (!c) {
+ return ...
+}
+
+the main path
+```
+
+
+## 3. Design and Structure
+
+While it is hard to exactly specify what constitutes a good design, there are some properties that can serve as a _litmus test_ for a good design. If these properties are given, the chances are good that the design is going into a good direction. If these properties cannot be achieved, there is a high probability that the design is flawed.
+
+
+### Immutability and Eager Initialization
+
+1. Try to use immutable types where possible, especially for APIs, messages, identifiers, properties, configuration, etc.
+2. A good general approach is to try and make as many fields of a class `final` as possible.
+3. Classes that are used as keys in maps should be strictly immutable and only have `final` fields (except maybe auxiliary fields, like lazy cached hash codes).
+4. Eagerly initialize classes. There should be no `init()` or `setup()` methods. Once the constructor completes, the object should be usable.
+
+
+### Nullability of the Mutable Parts
+
+For nullability, the Flink codebase aims to follow these conventions:
+
+* Fields, parameters, and return types are always non-null, unless indicated otherwise
+* All fields, parameters and method types that can be null should be annotated with `@javax.annotation.Nullable`.
+That way, you get warnings from IntelliJ about all sections where you have to reason about potential null values.
+* For all mutable (non-final) fields that are not annotated, the assumption is that while the field value changes, there always is a value.
+ * This should be double check whether these can in fact not be null throughout the lifetime of the object.
+
+_Note: This means that `@Nonnull` annotations are usually not necessary, but can be used in certain cases to override a previous annotation, or to point non-nullability out in a context where one would expect a nullable value._
+
+`Optional` is a good solution as a return type for method that may or may not have a result, so nullable return types are good candidates to be replaced with `Optional`.
+For fields and parameters, `Optional` is disputed in Java and most parts of the Flink code case don’t use optional for fields.
+
+
+### Avoid Code Duplication
+
+1. Whenever you are about to copy/paste some code, or reproduce a similar type of functionality in a different place, think about the ways how to refactor/reuse/abstract the changes to avoid the duplication.
+2. Common behavior between different specializations should be shared in a common component (or a shared superclass).
+3. Always use “private static final” constants instead of duplicating strings or other special values at different locations. Constants should be declared in the top member area of a class.
+
+
+### Design for Testability
+
+Code that is easily testable typically has good separation of concerns and is structured to be reusable outside the original context (by being easily reusable in tests).
+
+A good summary or problems / symptoms and recommended refactoring is in the PDF linked below.
+Please note that while the examples in the PDF often use a dependency injection framework (Guice), it works in the same way without such a framework.[^1]
+
+[http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf](http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf)
+
+Here is a compact summary of the most important aspects.
+
+
+**Inject dependencies**
+
+Reusability becomes easier if constructors don’t create their dependencies (the objects assigned to the fields), but accept them as parameters.
+
+* Effectively, constructors should have no `new` keyword.
+* Exceptions are creating a new empty collection (`new ArrayList<>()`) or similar auxiliary fields (objects that have only primitive dependencies).
+
+To make instantiation easy / readable, add factory methods or additional convenience constructors to construct whole object with dependencies.
+
+In no case should it ever be required to use a reflection or a “Whitebox” util to change the fields of an object in a test, or to use PowerMock to intercept a “new” call and supply a mock.
+
+
+**Avoid “too many collaborators”**
+
+If you have to take a big set of other components into account during testing (“too many collaborators”), consider refactoring.
+
+The component/class you want to test probably depends on another broad component (and its implementation), rather than on the minimal interface (abstraction) required for its work.
+
+In that case, segregate the interfaces (factor out the minimal required interface) and supply a test stub in that case.
+
+* For example, if testing a S3RecoverableMultiPartUploader requires actual S3 access
+then the S3 access should be factored out into an interface and test should replace it by a test stub
+* This naturally requires to be able to inject dependencies (see above)
+
+⇒ Please note that these steps often require more effort in implementing tests (factoring out interfaces, creating dedicated test stubs), but make the tests more resilient to changes in other components, i.e., you do not need to touch the tests when making unrelated changes.
+
+
+**Write targeted tests**
+
+* <span style="text-decoration:underline;">Test contracts not implementations</span>: Test that after a sequence of actions, the components are in a certain state, rather than testing that the components followed a sequence of internal state modifications.
+ * For example, a typical antipattern is to check whether one specific method was called as part of the test
+* A way to enforce this is to try to follow the _Arrange_, _Act_, _Assert_ test structure when writing a unit test ([https://xp123.com/articles/3a-arrange-act-assert/](https://xp123.com/articles/3a-arrange-act-assert/))
+
+ This helps to communicate the intention of the test (what is the scenario under test) rather than the mechanics of the tests. The technical bits go to a static methods at the bottom of the test class.
+
+ Example of tests in Flink that follow this pattern are:
+
+ * [https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java](https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java)
+ * [https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java](https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java)
+
+
+**Avoid Mockito - Use reusable test implementations**
+
+* Mockito-based tests tend to be costly to maintain in the long run by encouraging duplication of functionality and testing for implementation rather than effect
+ * More details: [https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8](https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8)
+* Instead, create reusable test implementations and utilities
+ * That way, when some class changes, we only have to update a few test utils or mocks
+
+
+### Performance Awareness
+
+We can conceptually distinguish between code that “coordinates” and code that “processes data”. Code that coordinates should always favor simplicity and cleanness. Data processing code is highly performance critical and should optimize for performance.
+
+That means still applying the general idea of the sections above, but possibly forgoing some aspects in some place, in order to achieve higher performance.
+
+
+**Which code paths are Data Processing paths?**
+
+* <span style="text-decoration:underline;">Per-record code paths:</span> Methods and code paths that are called for each record. Found for example in Connectors, Serializers, State Backends, Formats, Tasks, Operators, Metrics, runtime data structures, etc.
+* <span style="text-decoration:underline;">I/O methods:</span> Transferring messages or chunks of data in buffers. Examples are in the RPC system, Network Stack, FileSystems, Encoders / Decoders, etc.
+
+
+**Things that performance critical code may do that we would otherwise avoid**
+
+* Using (and reusing) mutable objects to take pressure off the GC (and sometimes help with cache locality), thus forgoing the strive for immutability.
+* Using primitive types, arrays of primitive types, or MemorySegment/ByteBuffer and encoding meaning into the primitive types and byte sequences, rather than encapsulating the behavior in dedicated classes and using objects.
+* Structuring the code to amortize expensive work (allocations, lookups, virtual method calls, …) across multiple records, for example by doing the work once per buffer/bundle/batch.
+* Code layout optimized for the JIT rather than for readability. Examples are inlining fields from other classes (in cases where it is doubtful whether the JIT would do that optimization at runtime), or structuring code to help the JIT compiler with inlining, loop unrolling, vectorization, etc.
+
+
+
+## 4. Concurrency and Threading
+
+**Most code paths should not require any concurrency.** The right internal abstractions should obviate the need for concurrency in almost all cases.
+
+* The Flink core and runtime use concurrency to provide these building blocks.
+Examples are in the RPC system, Network Stack, in the Task’s mailbox model, or some predefined Source / Sink utilities.
+* We are not fully there, but any new addition that introduces implements its own concurrency should be under scrutiny, unless it falls into the above category of core system building blocks.
+* Contributors should reach out to committers if they feel they need to implement concurrent code to see if there is an existing abstraction/building-block, or if one should be added.
+
+
+**When developing a component think about threading model and synchronization points ahead.**
+
+* For example: single threaded, blocking, non-blocking, synchronous, asynchronous, multi threaded, thread pool, message queues, volatile, synchronized block/methods, mutexes, atomics, callbacks, …
+* Getting those things right and thinking about them ahead is even more important than designing classes interfaces/responsibilities, since it’s much harder to change later on.
+
+
+**Try to avoid using threads all together if possible in any way.**
+
+* If you feel you have a case for spawning a thread, point this out in the pull request as something to be explicitly reviewed.
+
+
+**Be aware that using threads is in fact much harder than it initially looks**
+
+* Clean shutdown of threads is very tricky.
+* Handling interruptions in a rock solid fashion (avoid both slow shutdown and live locks) requires almost a Java Wizard
+* Ensuring clean error propagation out of threads in all cases needs thorough design.
+* Complexity of multi-threaded application/component/class grows exponentially, with each additional synchronisation point/block/critical section. Your code initially might be easy enough to understand, but can quickly grow beyond that point.
+* Proper testing of multithreaded code is basically impossible, while alternative approaches (like asynchronous code, non-blocking code, actor model with message queues) are quite easy to test.
+* Usually multi-threaded code is often even less efficient compared to alternative approaches on modern hardware.
+
+
+**Be aware of the java.util.concurrent.CompletableFuture**
+
+* Like with other concurrent code, there should rarely be the need to use a CompletableFuture
+* Completing a future would also complete on the calling thread any chained futures that are waiting for the result to be completed, unless a completion executor specified explicitly.
+* This can be intentional, if the entire execution should be synchronous / single-threaded, as for example in parts of the Scheduler / ExecutionGraph.
+ * Flink even makes use of a “main-thread executor” to allow calling chained handlers in the same thread as a single-threaded RPC endpoint runs
+* This can be unexpected, if the thread that completes the future is a sensitive thread.
+ * It may be better to use `CompletableFuture.supplyAsync(value, executor)` in that case, instead of `future.complete(value)` when an executor is available
+* When blocking on a future awaiting completion, always supply a timeout for a result instead of waiting indefinitely, and handle timeouts explicitly.
+* Use `CompletableFuture.allOf()`/`anyOf()`, `ExecutorCompletionService`, or `org.apache.flink.runtime.concurrent.FutureUtils#waitForAll` if you need to wait for: all the results/any of the results/all the results but handled by (approximate) completion order.
+
+
+
+
+## 5. Dependencies and Modules
+
+* **Keep the dependency footprint small**
+ * The more dependencies the harder it gets for the community to manage them as a whole.
+ * Dependency management includes dependency conflicts, maintaining licenses and related notices, and handling security vulnerabilities.
+ * Discuss whether the dependency should be shaded/relocated to avoid future conflicts.
+* **Don’t add a dependency for just one method**
+ * Use Java built-in means if possible.
+ * If the method is Apache-licensed, you can copy the method into a Flink utility class with proper attribution.
+* **Declaration of dependencies**
+ * Declare dependencies that you explicitly rely on, whether it provides classes you directly import and use or it's something that provides a service you directly use, like Log4J.
+ * Transitive dependencies should only supply dependencies that are needed at runtime but that you don't use yourself.
+ * [[source](https://stackoverflow.com/questions/15177661/maven-transitive-dependencies)]
+* **Location of classes in the Maven modules**
+ * Whenever you create a new class, think about where to put it.
+ * A class might be used by multiple modules in the future and might belong into a `common` module in this case.
+
+
+
+
+
+<hr>
+
+[^1]:
+ We are keeping such frameworks out of Flink, to make debugging easier and avoid dependency clashes.
diff --git a/contributing/code-style-and-quality-components.md b/contributing/code-style-and-quality-components.md
new file mode 100644
index 0000000..0478b78
--- /dev/null
+++ b/contributing/code-style-and-quality-components.md
@@ -0,0 +1,143 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Components"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+
+
+## Component Specific Guidelines
+
+_Additional guidelines about changes in specific components._
+
+
+### Configuration Changes
+
+Where should the config option go?
+
+* <span style="text-decoration:underline;">‘flink-conf.yaml’:</span> All configuration that pertains to execution behavior that one may want to standardize across jobs. Think of it as parameters someone would set wearing an “ops” hat, or someone that provides a stream processing platform to other teams.
+
+* <span style="text-decoration:underline;">‘ExecutionConfig’</span>: Parameters specific to an individual Flink application, needed by the operators during execution. Typical examples are watermark interval, serializer parameters, object reuse.
+* <span style="text-decoration:underline;">ExecutionEnvironment (in code)</span>: Everything that is specific to an individual Flink application and is only needed to build program / dataflow, not needed inside the operators during execution.
+
+How to name config keys:
+
+* Config key names should be hierarchical.
+ Think of the configuration as nested objects (JSON style)
+
+ ```
+ taskmanager: {
+ jvm-exit-on-oom: true,
+ network: {
+ detailed-metrics: false,
+ request-backoff: {
+ initial: 100,
+ max: 10000
+ },
+ memory: {
+ fraction: 0.1,
+ min: 64MB,
+ max: 1GB,
+ buffers-per-channel: 2,
+ floating-buffers-per-gate: 16
+ }
+ }
+ }
+ ```
+
+* The resulting config keys should hence be:
+
+ **NOT** `"taskmanager.detailed.network.metrics"`
+
+ **But rather** `"taskmanager.network.detailed-metrics"`
+
+
+### Connectors
+
+Connectors are historically hard to implement and need to deal with many aspects of threading, concurrency, and checkpointing.
+
+As part of [FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface) we are working on making this much simpler for sources. New sources should not have to deal with any aspect of concurrency/threading and checkpointing any more.
+
+A similar FLIP can be expected for sinks in the near future.
+
+
+### Examples
+
+Examples should be self-contained and not require systems other than Flink to run. Except for examples that show how to use specific connectors, like the Kafka connector. Sources/sinks that are ok to use are `StreamExecutionEnvironment.socketTextStream`, which should not be used in production but is quite handy for exploring how things work, and file-based sources/sinks. (For streaming, there is the continuous file source)
+
+Examples should also not be pure toy-examples but strike a balance between real-world code and purely abstract examples. The WordCount example is quite long in the tooth by now but it’s a good showcase of simple code that highlights functionality and can do useful things.
+
+Examples should also be heavy in comments. They should describe the general idea of the example in the class-level Javadoc and describe what is happening and what functionality is used throughout the code. The expected input data and output data should also be described.
+
+Examples should include parameter parsing, so that you can run an example (from the Jar that is created for each example using `bin/flink run path/to/myExample.jar --param1 … --param2`.
+
+
+### Table & SQL API
+
+
+#### Semantics
+
+**The SQL standard should be the main source of truth.**
+
+* Syntax, semantics, and features should be aligned with SQL!
+* We don’t need to reinvent the wheel. Most problems have already been discussed industry-wide and written down in the SQL standard.
+* We rely on the newest standard (SQL:2016 or ISO/IEC 9075:2016 when writing this document [[download]](https://standards.iso.org/ittf/PubliclyAvailableStandards/c065143_ISO_IEC_TR_19075-5_2016.zip) ). Not every part is available online but a quick web search might help here.
+
+Discuss divergence from the standard or vendor-specific interpretations.
+
+* Once a syntax or behavior is defined it cannot be undone easily.
+* Contributions that need to extent or interpret the standard need a thorough discussion with the community.
+* Please help committers by performing some initial research about how other vendors such as Postgres, Microsoft SQL Server, Oracle, Hive, Calcite, Beam are handling such cases.
+
+
+Consider the Table API as a bridge between the SQL and Java/Scala programming world.
+
+* The Table API is an Embedded Domain Specific Language for analytical programs following the relational model.
+It is not required to strictly follow the SQL standard in regards of syntax and names, but can be closer to the way a programming language would do/name functions and features, if that helps make it feel more intuitive.
+* The Table API might have some non-SQL features (e.g. map(), flatMap(), etc.) but should nevertheless “feel like SQL”. Functions and operations should have equal semantics and naming if possible.
+
+
+#### Common mistakes
+
+* Support SQL’s type system when adding a feature.
+ * A SQL function, connector, or format should natively support most SQL types from the very beginning.
+ * Unsupported types lead to confusion, limit the usability, and create overhead by touching the same code paths multiple times.
+ * For example, when adding a `SHIFT_LEFT` function, make sure that the contribution is general enough not only for `INT` but also `BIGINT` or `TINYINT`.
+
+
+#### Testing
+
+Test for nullability.
+
+* SQL natively supports `NULL` for almost every operation and has a 3-valued boolean logic.
+* Make sure to test every feature for nullability as well.
+
+
+Avoid full integration tests
+
+* Spawning a Flink mini-cluster and performing compilation of generated code for a SQL query is expensive.
+* Avoid integration tests for planner tests or variations of API calls.
+* Instead, use unit tests that validate the optimized plan which comes out of a planner. Or test the behavior of a runtime operator directly.
+
+
+#### Compatibility
+
+Don’t introduce physical plan changes in minor releases!
+
+* Backwards compatibility for state in streaming SQL relies on the fact that the physical execution plan remains stable. Otherwise the generated Operator Names/IDs change and state cannot be matched and restored.
+* Every bug fix that leads to changes in the optimized physical plan of a streaming pipeline hences breaks compatibility.
+* As a consequence, changes of the kind that lead to different optimizer plans can only be merged in major releases for now.
+
+
+#### Scala / Java interoperability (legacy code parts)
+
+Keep Java in mind when designing interfaces.
+
+* Consider whether a class will need to interact with a Java class in the future.
+* Use Java collections and Java Optional in interfaces for a smooth integration with Java code.
+* Don’t use features of case classes such as .copy() or apply() for construction if a class is subjected to be converted to Java.
+* Pure Scala user-facing APIs should use pure Scala collections/iterables/etc. for natural and idiomatic (“scalaesk”) integration with Scala.
+
diff --git a/contributing/code-style-and-quality-components.zh.md b/contributing/code-style-and-quality-components.zh.md
new file mode 100644
index 0000000..0478b78
--- /dev/null
+++ b/contributing/code-style-and-quality-components.zh.md
@@ -0,0 +1,143 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Components"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+
+
+## Component Specific Guidelines
+
+_Additional guidelines about changes in specific components._
+
+
+### Configuration Changes
+
+Where should the config option go?
+
+* <span style="text-decoration:underline;">‘flink-conf.yaml’:</span> All configuration that pertains to execution behavior that one may want to standardize across jobs. Think of it as parameters someone would set wearing an “ops” hat, or someone that provides a stream processing platform to other teams.
+
+* <span style="text-decoration:underline;">‘ExecutionConfig’</span>: Parameters specific to an individual Flink application, needed by the operators during execution. Typical examples are watermark interval, serializer parameters, object reuse.
+* <span style="text-decoration:underline;">ExecutionEnvironment (in code)</span>: Everything that is specific to an individual Flink application and is only needed to build program / dataflow, not needed inside the operators during execution.
+
+How to name config keys:
+
+* Config key names should be hierarchical.
+ Think of the configuration as nested objects (JSON style)
+
+ ```
+ taskmanager: {
+ jvm-exit-on-oom: true,
+ network: {
+ detailed-metrics: false,
+ request-backoff: {
+ initial: 100,
+ max: 10000
+ },
+ memory: {
+ fraction: 0.1,
+ min: 64MB,
+ max: 1GB,
+ buffers-per-channel: 2,
+ floating-buffers-per-gate: 16
+ }
+ }
+ }
+ ```
+
+* The resulting config keys should hence be:
+
+ **NOT** `"taskmanager.detailed.network.metrics"`
+
+ **But rather** `"taskmanager.network.detailed-metrics"`
+
+
+### Connectors
+
+Connectors are historically hard to implement and need to deal with many aspects of threading, concurrency, and checkpointing.
+
+As part of [FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface) we are working on making this much simpler for sources. New sources should not have to deal with any aspect of concurrency/threading and checkpointing any more.
+
+A similar FLIP can be expected for sinks in the near future.
+
+
+### Examples
+
+Examples should be self-contained and not require systems other than Flink to run. Except for examples that show how to use specific connectors, like the Kafka connector. Sources/sinks that are ok to use are `StreamExecutionEnvironment.socketTextStream`, which should not be used in production but is quite handy for exploring how things work, and file-based sources/sinks. (For streaming, there is the continuous file source)
+
+Examples should also not be pure toy-examples but strike a balance between real-world code and purely abstract examples. The WordCount example is quite long in the tooth by now but it’s a good showcase of simple code that highlights functionality and can do useful things.
+
+Examples should also be heavy in comments. They should describe the general idea of the example in the class-level Javadoc and describe what is happening and what functionality is used throughout the code. The expected input data and output data should also be described.
+
+Examples should include parameter parsing, so that you can run an example (from the Jar that is created for each example using `bin/flink run path/to/myExample.jar --param1 … --param2`.
+
+
+### Table & SQL API
+
+
+#### Semantics
+
+**The SQL standard should be the main source of truth.**
+
+* Syntax, semantics, and features should be aligned with SQL!
+* We don’t need to reinvent the wheel. Most problems have already been discussed industry-wide and written down in the SQL standard.
+* We rely on the newest standard (SQL:2016 or ISO/IEC 9075:2016 when writing this document [[download]](https://standards.iso.org/ittf/PubliclyAvailableStandards/c065143_ISO_IEC_TR_19075-5_2016.zip) ). Not every part is available online but a quick web search might help here.
+
+Discuss divergence from the standard or vendor-specific interpretations.
+
+* Once a syntax or behavior is defined it cannot be undone easily.
+* Contributions that need to extent or interpret the standard need a thorough discussion with the community.
+* Please help committers by performing some initial research about how other vendors such as Postgres, Microsoft SQL Server, Oracle, Hive, Calcite, Beam are handling such cases.
+
+
+Consider the Table API as a bridge between the SQL and Java/Scala programming world.
+
+* The Table API is an Embedded Domain Specific Language for analytical programs following the relational model.
+It is not required to strictly follow the SQL standard in regards of syntax and names, but can be closer to the way a programming language would do/name functions and features, if that helps make it feel more intuitive.
+* The Table API might have some non-SQL features (e.g. map(), flatMap(), etc.) but should nevertheless “feel like SQL”. Functions and operations should have equal semantics and naming if possible.
+
+
+#### Common mistakes
+
+* Support SQL’s type system when adding a feature.
+ * A SQL function, connector, or format should natively support most SQL types from the very beginning.
+ * Unsupported types lead to confusion, limit the usability, and create overhead by touching the same code paths multiple times.
+ * For example, when adding a `SHIFT_LEFT` function, make sure that the contribution is general enough not only for `INT` but also `BIGINT` or `TINYINT`.
+
+
+#### Testing
+
+Test for nullability.
+
+* SQL natively supports `NULL` for almost every operation and has a 3-valued boolean logic.
+* Make sure to test every feature for nullability as well.
+
+
+Avoid full integration tests
+
+* Spawning a Flink mini-cluster and performing compilation of generated code for a SQL query is expensive.
+* Avoid integration tests for planner tests or variations of API calls.
+* Instead, use unit tests that validate the optimized plan which comes out of a planner. Or test the behavior of a runtime operator directly.
+
+
+#### Compatibility
+
+Don’t introduce physical plan changes in minor releases!
+
+* Backwards compatibility for state in streaming SQL relies on the fact that the physical execution plan remains stable. Otherwise the generated Operator Names/IDs change and state cannot be matched and restored.
+* Every bug fix that leads to changes in the optimized physical plan of a streaming pipeline hences breaks compatibility.
+* As a consequence, changes of the kind that lead to different optimizer plans can only be merged in major releases for now.
+
+
+#### Scala / Java interoperability (legacy code parts)
+
+Keep Java in mind when designing interfaces.
+
+* Consider whether a class will need to interact with a Java class in the future.
+* Use Java collections and Java Optional in interfaces for a smooth integration with Java code.
+* Don’t use features of case classes such as .copy() or apply() for construction if a class is subjected to be converted to Java.
+* Pure Scala user-facing APIs should use pure Scala collections/iterables/etc. for natural and idiomatic (“scalaesk”) integration with Scala.
+
diff --git a/contributing/code-style-and-quality-formatting.md b/contributing/code-style-and-quality-formatting.md
new file mode 100644
index 0000000..cb83717
--- /dev/null
+++ b/contributing/code-style-and-quality-formatting.md
@@ -0,0 +1,87 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Formatting"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+
+## Java Code Formatting Style
+
+We recommend to set up the IDE to automatically check the code style. Please follow the [IDE setup guide](https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java) for that.
+
+
+### License
+
+* **Apache license headers.** Make sure you have Apache License headers in your files. The RAT plugin is checking for that when you build the code.
+
+### Imports
+
+* **Empty line before and after package declaration.**
+* **No unused imports.**
+* **No redundant imports.**
+* **No wildcard imports.** They can cause problems when adding to the code and in some cases even during refactoring.
+* **Import order.** Imports must be ordered alphabetically, grouped into the following blocks, with each block separated by an empty line:
+ * <imports from org.apache.flink.*>
+ * <imports from org.apache.flink.shaded.*>
+ * <imports from other libraries>
+ * <imports from javax.*>
+ * <imports from java.*>
+ * <imports from scala.*>
+ * <static imports>
+
+
+### Naming
+
+* **Package names must start with a letter, and must not contain upper-case letters or special characters.**
+ **Non-private static final fields must be upper-case, with words being separated by underscores.**(`MY_STATIC_VARIABLE`)
+* **Non-static fields/methods must be in lower camel case.** (`myNonStaticField`)
+
+
+### Whitespaces
+
+* **Tabs vs. spaces.** We are using tabs for indentation, not spaces.
+We are aware that spaces are a bit nicer; it just happened to be that we started with tabs a long time ago (because Eclipse’s default style used tabs then), and we tried to keep the code base homogeneous (not mix tabs and spaces).
+* **No trailing whitespace.**
+* **Spaces around operators/keywords.** Operators (`+`, `=`, `>`, …) and keywords (`if`, `for`, `catch`, …) must have a space before and after them, provided they are not at the start or end of the line.
+
+
+### Braces
+
+* **Left curly braces (<code>{</code>) must not be placed on a new line.**
+* <strong>Right curly braces (<code>}</code>) must always be placed at the beginning of the line.</strong>
+* <strong>Blocks.</strong> All statements after <code>if</code>, <code>for</code>, <code>while</code>, <code>do</code>, … must always be encapsulated in a block with curly braces (even if the block contains one statement).
+
+
+### Javadocs
+
+* **All public/protected methods and classes must have a Javadoc.**
+* **The first sentence of the Javadoc must end with a period.**
+* **Paragraphs must be separated with a new line, and started with <p>.**
+
+
+### Modifiers
+
+* **No redundant modifiers.** For example, public modifiers in interface methods.
+* **Follow JLS3 modifier order.** Modifiers must be ordered in the following order: public, protected, private, abstract, static, final, transient, volatile, synchronized, native, strictfp.
+
+
+### Files
+
+* **All files must end with <code>\n</code>.**
+* <strong>File length must not exceed 3000 lines.</strong>
+
+
+### Misc
+
+* **Arrays must be defined Java-style.** For example, `public String[] array`.
+* **Use Flink Preconditions.** To increase homogeneity, consistently use the `org.apache.flink.Preconditions` methods `checkNotNull` and `checkArgument` rather than Apache Commons Validate or Google Guava.
+
+
+
+<hr>
+
+[^1]:
+ We are keeping such frameworks out of Flink, to make debugging easier and avoid dependency clashes.
diff --git a/contributing/code-style-and-quality-formatting.zh.md b/contributing/code-style-and-quality-formatting.zh.md
new file mode 100644
index 0000000..cb83717
--- /dev/null
+++ b/contributing/code-style-and-quality-formatting.zh.md
@@ -0,0 +1,87 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Formatting"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+
+## Java Code Formatting Style
+
+We recommend to set up the IDE to automatically check the code style. Please follow the [IDE setup guide](https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java) for that.
+
+
+### License
+
+* **Apache license headers.** Make sure you have Apache License headers in your files. The RAT plugin is checking for that when you build the code.
+
+### Imports
+
+* **Empty line before and after package declaration.**
+* **No unused imports.**
+* **No redundant imports.**
+* **No wildcard imports.** They can cause problems when adding to the code and in some cases even during refactoring.
+* **Import order.** Imports must be ordered alphabetically, grouped into the following blocks, with each block separated by an empty line:
+ * <imports from org.apache.flink.*>
+ * <imports from org.apache.flink.shaded.*>
+ * <imports from other libraries>
+ * <imports from javax.*>
+ * <imports from java.*>
+ * <imports from scala.*>
+ * <static imports>
+
+
+### Naming
+
+* **Package names must start with a letter, and must not contain upper-case letters or special characters.**
+ **Non-private static final fields must be upper-case, with words being separated by underscores.**(`MY_STATIC_VARIABLE`)
+* **Non-static fields/methods must be in lower camel case.** (`myNonStaticField`)
+
+
+### Whitespaces
+
+* **Tabs vs. spaces.** We are using tabs for indentation, not spaces.
+We are aware that spaces are a bit nicer; it just happened to be that we started with tabs a long time ago (because Eclipse’s default style used tabs then), and we tried to keep the code base homogeneous (not mix tabs and spaces).
+* **No trailing whitespace.**
+* **Spaces around operators/keywords.** Operators (`+`, `=`, `>`, …) and keywords (`if`, `for`, `catch`, …) must have a space before and after them, provided they are not at the start or end of the line.
+
+
+### Braces
+
+* **Left curly braces (<code>{</code>) must not be placed on a new line.**
+* <strong>Right curly braces (<code>}</code>) must always be placed at the beginning of the line.</strong>
+* <strong>Blocks.</strong> All statements after <code>if</code>, <code>for</code>, <code>while</code>, <code>do</code>, … must always be encapsulated in a block with curly braces (even if the block contains one statement).
+
+
+### Javadocs
+
+* **All public/protected methods and classes must have a Javadoc.**
+* **The first sentence of the Javadoc must end with a period.**
+* **Paragraphs must be separated with a new line, and started with <p>.**
+
+
+### Modifiers
+
+* **No redundant modifiers.** For example, public modifiers in interface methods.
+* **Follow JLS3 modifier order.** Modifiers must be ordered in the following order: public, protected, private, abstract, static, final, transient, volatile, synchronized, native, strictfp.
+
+
+### Files
+
+* **All files must end with <code>\n</code>.**
+* <strong>File length must not exceed 3000 lines.</strong>
+
+
+### Misc
+
+* **Arrays must be defined Java-style.** For example, `public String[] array`.
+* **Use Flink Preconditions.** To increase homogeneity, consistently use the `org.apache.flink.Preconditions` methods `checkNotNull` and `checkArgument` rather than Apache Commons Validate or Google Guava.
+
+
+
+<hr>
+
+[^1]:
+ We are keeping such frameworks out of Flink, to make debugging easier and avoid dependency clashes.
diff --git a/contributing/code-style-and-quality-general.md b/contributing/code-style-and-quality-general.md
deleted file mode 100644
index 0d02733..0000000
--- a/contributing/code-style-and-quality-general.md
+++ /dev/null
@@ -1,667 +0,0 @@
----
-title: "Apache Flink Code Style and Quality Guide"
----
-
-{% include code-style-navbar.md %}
-
-{% toc %}
-
-
-## 1. Tools
-
-We recommend to follow the [IDE Setup Guide](https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java) to get IDE tooling configured.
-
-
-### Use inspections in IntelliJ
-
-* Import the inspections settings into the IDE (see IDE setup guide)
- * TODO: Need to agree on a profile and export it (like checkstyle)
-* Write the code such that inspection warnings are addressed
- * There are few exceptions where an inspection warning is not meaningful. In that case, suppress the inspection warning.
-
-
-### Warnings
-
-* We strive for zero warnings
-* Even though there are many warnings in existing code, new changes should not add any additional compiler warnings
-* If it is not possible to address the warning in a sane way (in some cases when working with generics) add an annotation to suppress the warning
-* When deprecating methods, check that this does not introduce additional warnings
-
-
-
-## 2. Comments And Code Readability
-
-
-### Comments
-
-**Golden rule: Comment as much as necessary to support code understanding, but don’t add redundant information.**
-
-Think about
-
-* <span style="text-decoration:underline;">What</span> is the code doing?
-* <span style="text-decoration:underline;">How</span> does the code do this?
-* <span style="text-decoration:underline;">Why</span> is the code like that?
-
-The code alone should explain as much as possible the “<span style="text-decoration:underline;">what</span>” and the “<span style="text-decoration:underline;">how</span>”
-
-* Use JavaDocs to describe the roles of classes and the contracts of methods, in cases where the contract is not obvious or intuitive from the method name (the “what”).
-* The flow of the code should give a good description of the “how”.
-Think of variable and method names as part of the code documenting itself.
-* It often makes reading the code easier if larger blocks that form a unit are moved into a private method with a descriptive name of what that block is doing
-
-In-code comments help explain the <span style="text-decoration:underline;">“why”</span>
-
-* For example `// this specific code layout helps the JIT to better do this or that`
-* Or `// nulling out this field here means future write attempts are fail-fast`
-* Or `// for arguments with which this method is actually called, this seemingly naive approach works actually better than any optimized/smart version`
-
-In-code comments should not state redundant information about the “what” and “how” that is already obvious in the code itself.
-
-JavaDocs should not state meaningless information (just to satisfy the Checkstyle checker).
-
-__Don’t:__
-
-```
-/**
- * The symbol expression.
- */
-public class CommonSymbolExpression {}
-```
-__Do:__
-
-```
-/**
- * An expression that wraps a single specific symbol.
- * A symbol could be a unit, an alias, a variable, etc.
- */
-public class CommonSymbolExpression {}
-```
-
-
-### Branches and Nesting
-
-Avoid deep nesting of scopes, by flipping the if condition and exiting early.
-
-__Don’t:__
-
-```
-if (a) {
- if (b) {
- if (c) {
- the main path
- }
- }
-}
-```
-
-__Do__
-
-```
-if (!a) {
- return ..
-}
-
-if (!b) {
- return ...
-}
-
-if (!c) {
- return ...
-}
-
-the main path
-```
-
-
-## 3. Design and Structure
-
-While it is hard to exactly specify what constitutes a good design, there are some properties that can serve as a _litmus test_ for a good design. If these properties are given, the chances are good that the design is going into a good direction. If these properties cannot be achieved, there is a high probability that the design is flawed.
-
-
-### Immutability and Eager Initialization
-
-1. Try to use immutable types where possible, especially for APIs, messages, identifiers, properties, configuration, etc.
-2. A good general approach is to try and make as many fields of a class `final` as possible.
-3. Classes that are used as keys in maps should be strictly immutable and only have `final` fields (except maybe auxiliary fields, like lazy cached hash codes).
-4. Eagerly initialize classes. There should be no `init()` or `setup()` methods. Once the constructor completes, the object should be usable.
-
-
-### Nullability of the Mutable Parts
-
-For nullability, the Flink codebase aims to follow these conventions:
-
-* Fields, parameters, and return types are always non-null, unless indicated otherwise
-* All fields, parameters and method types that can be null should be annotated with `@javax.annotation.Nullable`.
-That way, you get warnings from IntelliJ about all sections where you have to reason about potential null values.
-* For all mutable (non-final) fields that are not annotated, the assumption is that while the field value changes, there always is a value.
- * This should be double check whether these can in fact not be null throughout the lifetime of the object.
-
-_Note: This means that `@Nonnull` annotations are usually not necessary, but can be used in certain cases to override a previous annotation, or to point non-nullability out in a context where one would expect a nullable value._
-
-`Optional` is a good solution as a return type for method that may or may not have a result, so nullable return types are good candidates to be replaced with `Optional`.
-For fields and parameters, `Optional` is disputed in Java and most parts of the Flink code case don’t use optional for fields.
-
-
-### Avoid Code Duplication
-
-1. Whenever you are about to copy/paste some code, or reproduce a similar type of functionality in a different place, think about the ways how to refactor/reuse/abstract the changes to avoid the duplication.
-2. Common behavior between different specializations should be shared in a common component (or a shared superclass).
-3. Always use “private static final” constants instead of duplicating strings or other special values at different locations. Constants should be declared in the top member area of a class.
-
-
-### Design for Testability
-
-Code that is easily testable typically has good separation of concerns and is structured to be reusable outside the original context (by being easily reusable in tests).
-
-A good summary or problems / symptoms and recommended refactoring is in the PDF linked below.
-Please note that while the examples in the PDF often use a dependency injection framework (Guice), it works in the same way without such a framework.[^1]
-
-[http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf](http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf)
-
-Here is a compact summary of the most important aspects.
-
-
-**Inject dependencies**
-
-Reusability becomes easier if constructors don’t create their dependencies (the objects assigned to the fields), but accept them as parameters.
-
-* Effectively, constructors should have no `new` keyword.
-* Exceptions are creating a new empty collection (`new ArrayList<>()`) or similar auxiliary fields (objects that have only primitive dependencies).
-
-To make instantiation easy / readable, add factory methods or additional convenience constructors to construct whole object with dependencies.
-
-In no case should it ever be required to use a reflection or a “Whitebox” util to change the fields of an object in a test, or to use PowerMock to intercept a “new” call and supply a mock.
-
-
-**Avoid “too many collaborators”**
-
-If you have to take a big set of other components into account during testing (“too many collaborators”), consider refactoring.
-
-The component/class you want to test probably depends on another broad component (and its implementation), rather than on the minimal interface (abstraction) required for its work.
-
-In that case, segregate the interfaces (factor out the minimal required interface) and supply a test stub in that case.
-
-
-* For example, if testing a S3RecoverableMultiPartUploader requires actual S3 access
-then the S3 access should be factored out into an interface and test should replace it by a test stub
-* This naturally requires to be able to inject dependencies (see above)
-
-⇒ Please note that these steps often require more effort in implementing tests (factoring out interfaces, creating dedicated test stubs), but make the tests more resilient to changes in other components, i.e., you do not need to touch the tests when making unrelated changes.
-
-
-**Write targeted tests**
-
-* <span style="text-decoration:underline;">Test contracts not implementations</span>: Test that after a sequence of actions, the components are in a certain state, rather than testing that the components followed a sequence of internal state modifications.
- * For example, a typical antipattern is to check whether one specific method was called as part of the test
-* A way to enforce this is to try to follow the _Arrange_, _Act_, _Assert_ test structure when writing a unit test ([https://xp123.com/articles/3a-arrange-act-assert/](https://xp123.com/articles/3a-arrange-act-assert/))
-
- This helps to communicate the intention of the test (what is the scenario under test) rather than the mechanics of the tests. The technical bits go to a static methods at the bottom of the test class.
-
- Example of tests in Flink that follow this pattern are:
-
- * [https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java](https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java)
- * [https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java](https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java)
-
-
-**Avoid Mockito - Use reusable test implementations**
-
-* Mockito-based tests tend to be costly to maintain in the long run by encouraging duplication of functionality and testing for implementation rather than effect
- * More details: [https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8](https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8)
-* Instead, create reusable test implementations and utilities
- * That way, when some class changes, we only have to update a few test utils or mocks
-
-
-### Performance Awareness
-
-We can conceptually distinguish between code that “coordinates” and code that “processes data”. Code that coordinates should always favor simplicity and cleanness. Data processing code is highly performance critical and should optimize for performance.
-
-That means still applying the general idea of the sections above, but possibly forgoing some aspects in some place, in order to achieve higher performance.
-
-
-**Which code paths are Data Processing paths?**
-
-* <span style="text-decoration:underline;">Per-record code paths:</span> Methods and code paths that are called for each record. Found for example in Connectors, Serializers, State Backends, Formats, Tasks, Operators, Metrics, runtime data structures, etc.
-* <span style="text-decoration:underline;">I/O methods:</span> Transferring messages or chunks of data in buffers. Examples are in the RPC system, Network Stack, FileSystems, Encoders / Decoders, etc.
-
-
-**Things that performance critical code may do that we would otherwise avoid**
-
-* Using (and reusing) mutable objects to take pressure off the GC (and sometimes help with cache locality), thus forgoing the strive for immutability.
-* Using primitive types, arrays of primitive types, or MemorySegment/ByteBuffer and encoding meaning into the primitive types and byte sequences, rather than encapsulating the behavior in dedicated classes and using objects.
-* Structuring the code to amortize expensive work (allocations, lookups, virtual method calls, …) across multiple records, for example by doing the work once per buffer/bundle/batch.
-* Code layout optimized for the JIT rather than for readability. Examples are inlining fields from other classes (in cases where it is doubtful whether the JIT would do that optimization at runtime), or structuring code to help the JIT compiler with inlining, loop unrolling, vectorization, etc.
-
-
-
-## 4. Concurrency and Threading
-
-**Most code paths should not require any concurrency.** The right internal abstractions should obviate the need for concurrency in almost all cases.
-
-* The Flink core and runtime use concurrency to provide these building blocks.
-Examples are in the RPC system, Network Stack, in the Task’s mailbox model, or some predefined Source / Sink utilities.
-* We are not fully there, but any new addition that introduces implements its own concurrency should be under scrutiny, unless it falls into the above category of core system building blocks.
-* Contributors should reach out to committers if they feel they need to implement concurrent code to see if there is an existing abstraction/building-block, or if one should be added.
-
-
-**When developing a component think about threading model and synchronization points ahead.**
-
-* For example: single threaded, blocking, non-blocking, synchronous, asynchronous, multi threaded, thread pool, message queues, volatile, synchronized block/methods, mutexes, atomics, callbacks, …
-* Getting those things right and thinking about them ahead is even more important than designing classes interfaces/responsibilities, since it’s much harder to change later on.
-
-
-**Try to avoid using threads all together if possible in any way.**
-
-* If you feel you have a case for spawning a thread, point this out in the pull request as something to be explicitly reviewed.
-
-
-**Be aware that using threads is in fact much harder than it initially looks**
-
-* Clean shutdown of threads is very tricky.
-* Handling interruptions in a rock solid fashion (avoid both slow shutdown and live locks) requires almost a Java Wizard
-* Ensuring clean error propagation out of threads in all cases needs thorough design.
-* Complexity of multi-threaded application/component/class grows exponentially, with each additional synchronisation point/block/critical section. Your code initially might be easy enough to understand, but can quickly grow beyond that point.
-* Proper testing of multithreaded code is basically impossible, while alternative approaches (like asynchronous code, non-blocking code, actor model with message queues) are quite easy to test.
-* Usually multi-threaded code is often even less efficient compared to alternative approaches on modern hardware.
-
-
-**Be aware of the java.util.concurrent.CompletableFuture**
-
-* Like with other concurrent code, there should rarely be the need to use a CompletableFuture
-* Completing a future would also complete on the calling thread any chained futures that are waiting for the result to be completed, unless a completion executor specified explicitly.
-* This can be intentional, if the entire execution should be synchronous / single-threaded, as for example in parts of the Scheduler / ExecutionGraph.
- * Flink even makes use of a “main-thread executor” to allow calling chained handlers in the same thread as a single-threaded RPC endpoint runs
-* This can be unexpected, if the thread that completes the future is a sensitive thread.
- * It may be better to use `CompletableFuture.supplyAsync(value, executor)` in that case, instead of `future.complete(value)` when an executor is available
-* When blocking on a future awaiting completion, always supply a timeout for a result instead of waiting indefinitely, and handle timeouts explicitly.
-* Use `CompletableFuture.allOf()`/`anyOf()`, `ExecutorCompletionService`, or `org.apache.flink.runtime.concurrent.FutureUtils#waitForAll` if you need to wait for: all the results/any of the results/all the results but handled by (approximate) completion order.
-
-
-
-
-## 5. Dependencies and Modules
-
-* **Keep the dependency footprint small**
- * The more dependencies the harder it gets for the community to manage them as a whole.
- * Dependency management includes dependency conflicts, maintaining licenses and related notices, and handling security vulnerabilities.
- * Discuss whether the dependency should be shaded/relocated to avoid future conflicts.
-* **Don’t add a dependency for just one method**
- * Use Java built-in means if possible.
- * If the method is Apache-licensed, you can copy the method into a Flink utility class with proper attribution.
-* **Declaration of dependencies**
- * Declare dependencies that you explicitly rely on, whether it provides classes you directly import and use or it's something that provides a service you directly use, like Log4J.
- * Transitive dependencies should only supply dependencies that are needed at runtime but that you don't use yourself.
- * [[source](https://stackoverflow.com/questions/15177661/maven-transitive-dependencies)]
-* **Location of classes in the Maven modules**
- * Whenever you create a new class, think about where to put it.
- * A class might be used by multiple modules in the future and might belong into a `common` module in this case.
-
-
-## 6. Java Language Features and Libraries
-
-
-### Preconditions and Log Statements
-
-* Never concatenate strings in the parameters
- * <span style="text-decoration:underline;">Don’t:</span> `Preconditions.checkState(value <= threshold, "value must be below " + threshold)`
- * <span style="text-decoration:underline;">Don’t:</span> `LOG.debug("value is " + value)`
- * <span style="text-decoration:underline;">Do:</span> `Preconditions.checkState(value <= threshold, "value must be below %s", threshold)`
- * <span style="text-decoration:underline;">Do:</span> `LOG.debug("value is {}", value)`
-
-
-### Generics
-
-* **No raw types:** Do not use raw types, unless strictly necessary (sometimes necessary for signature matches, arrays).
-* **Suppress warnings for unchecked conversions:** Add annotations to suppress warnings, if they cannot be avoided (such as “unchecked”, or “serial”). Otherwise warnings about generics flood the build and drown relevant warnings.
-
-
-### equals() / hashCode()
-
-* **equals() / hashCode() should be added when they are well defined only.**
-* They should **not be added to enable a simpler assertion in tests** when they are not well defined. Use hamcrest matchers in that case: [https://github.com/junit-team/junit4/wiki/matchers-and-assertthat](https://github.com/junit-team/junit4/wiki/matchers-and-assertthat)
-* A common indicator that the methods are not well defined is when they take a subset of the fields into account (other than fields that are purely auxiliary).
-* When the methods take mutable fields into account, you often have a design issue. The `equals()`/`hashCode()` methods suggest to use the type as a key, but the signatures suggest it is safe to keep mutating the type.
-
-
-### Java Serialization
-
-* **Do not use Java Serialization for anything !!!**
-* **Do not use Java Serialization for anything !!! !!!**
-* **Do not use Java Serialization for anything !!! !!! !!!**
-* Internal to Flink, Java serialization is used to transport messages and programs through RPC. This is the only case where we use Java serialization. Because of that, some classes need to be serializable (if they are transported via RPC).
-* **Serializable classes must define a Serial Version UID:**
-
- `private static final long serialVersionUID = 1L;`
-* **The Serial Version UID for new classes should start at 1** and should generally be bumped on every incompatible change to the class according to the Java serialization compatibility definition (i.e: changing the type of a field, or moving the position of a class in the class hierarchy).
-
-
-### Java Reflection
-
-**Avoid using Java’s Reflection API**
-
-* Java’s Reflection API can be a very useful tool in certain cases but in all cases it is a hack and one should research for alternatives. The only cases where Flink should use reflection are
- * Dynamically loading implementations from another module (like webUI, additional serializers, pluggable query processors).
- * Extracting types inside the TypeExtractor class. This is fragile enough and should not be done outside the TypeExtractor class.
- * Some cases of cross-JDK version features, where we need to use reflection because we cannot assume a class/method to be present in all versions.
-* If you need reflection for accessing methods or fields in tests, it usually indicates some deeper architectural issues, like wrong scoping, bad separation of concerns, or that there is no clean way to provide components / dependencies to the class that is tested
-
-
-### Collections
-
-* **ArrayList and ArrayDeque are almost always superior to LinkedList**, except when frequently insert and deleting in the middle of the list
-* **For Maps, avoid patterns that require multiple lookups**
- * `contains()` before `get()` → `get()` and check null
- * `contains()` before `put()` → `putIfAbsent()` or `computeIfAbsent()`
- * Iterating over keys, getting values → iterate over `entrySet()`
-
-
-### Lambdas
-
-* Prefer non-capturing lambdas (lambdas that do not contain references to the outer scope). Capturing lambdas need to create a new object instance for every call. Non-capturing lambdas can use the same instance for each invocation.
-
- **don’t:**
- ```
- map.computeIfAbsent(key, x -> key.toLowerCase())
- ```
-
- **do:**
- ```
- map.computeIfAbsent(key, k -> k.toLowerCase());
- ```
-
-* Consider method references instead of inline lambdas
-
- **don’t**:
- ```
- map.computeIfAbsent(key, k-> Loader.load(k));
- ```
-
- **do:**
- ```
- map.computeIfAbsent(key, Loader::load);
- ```
-
-
-### Java Streams
-
-* Avoid Java Streams in any performance critical code.
-* The main motivation to use Java Streams would be to improve code readability. As such, they can be a good match in parts of the code that are not data-intensive, but deal with coordination..
-* Even in the latter case, try to limit the scope to a method, or a few private methods within an internal class.
-
-
-## 7. Scala Language Features
-
-### Where to use (and not use) Scala
-
-**We use Scala for Scala APIs or pure Scala Libraries.**
-
-**We do not use Scala in the core APIs and runtime components. We aim to remove existing Scala use (code and dependencies) from those components.**
-
-⇒ This is not because we do not like Scala, it is a consequence of “the right tool for the right job” approach (see below).
-
-For APIs, we develop the foundation in Java, and layer Scala on top.
-
-* This has traditionally given the best interoperability for both Java and Scala
-* It does mean dedicated effort to keep the Scala API up to date
-
-Why don’t we use Scala in the core APIs and runtime?
-
-* The past has shown that Scala evolves too quickly with tricky changes in functionality. Each Scala version upgrade was a rather big effort process for the Flink community.
-* Scala does not always interact nicely with Java classes, e.g. Scala’s visibility scopes work differently and often expose more to Java consumers than desired
-* Scala adds an additional layer of complexity to artifact/dependency management.
- * We may want to keep Scala dependent libraries like Akka in the runtime, but abstract them via an interface and load them in a separate classloader, to keep them shielded and avoid version conflicts.
-* Scala makes it very easy for knowledgeable Scala programmers to write code that is very hard to understand for programmers that are less knowledgeable in Scala. That is especially tricky for an open source project with a broad community of diverse experience levels. Working around this means restricting the Scala feature set by a lot, which defeats a good amount of the purpose of using Scala in the first place.
-
-
-### API Parity
-
-Keep Java API and Scala API in sync in terms of functionality and code quality.
-
-The Scala API should cover all the features of the Java APIs as well.
-
-Scala APIs should have a “completeness test”, like the following example from the DataStream API: [https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala](https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala)
-
-
-### Language Features
-
-* **Avoid Scala implicits.**
- * Scala’s implicits should only be used for user-facing API improvements such as the Table API expressions or type information extraction.
- * Don’t use them for internal “magic”.
-* **Add explicit types for class members.**
- * Don’t rely on implicit type inference for class fields and methods return types:
-
- **Don’t:**
- ```
- var expressions = new java.util.ArrayList[String]()
- ```
-
- **Do:**
- ```
- var expressions: java.util.List[String] = new java.util.ArrayList[]()
- ```
-
- * Type inference for local variables on the stack is fine.
-* **Use strict visibility.**
- * Avoid Scala’s package private features (such as private[flink]) and use regular private/protected instead.
- * Keep in mind that `private[flink]` and `protected` members are public in Java.
- * Keep in mind that `private[flink]` still exposes all members in Flink provided examples.
-
-
-### Coding Formatting
-
-**Use line wrapping to structure your code.**
-
-* Scala’s functional nature allows for long transformation chains (`x.map().map().foreach()`).
-* In order to force implementers to structure their code, the line length is therefore limited to 100 characters.
-* Use one line per transformation for better maintainability.
-
-
-## 8. Component Specific Guidelines
-
-_Additional guidelines about changes in specific components._
-
-
-### Configuration Changes
-
-Where should the config option go?
-
-* <span style="text-decoration:underline;">‘flink-conf.yaml’:</span> All configuration that pertains to execution behavior that one may want to standardize across jobs. Think of it as parameters someone would set wearing an “ops” hat, or someone that provides a stream processing platform to other teams.
-
-* <span style="text-decoration:underline;">‘ExecutionConfig’</span>: Parameters specific to an individual Flink application, needed by the operators during execution. Typical examples are watermark interval, serializer parameters, object reuse.
-* <span style="text-decoration:underline;">ExecutionEnvironment (in code)</span>: Everything that is specific to an individual Flink application and is only needed to build program / dataflow, not needed inside the operators during execution.
-
-How to name config keys:
-
-* Config key names should be hierarchical.
- Think of the configuration as nested objects (JSON style)
-
- ```
- taskmanager: {
- jvm-exit-on-oom: true,
- network: {
- detailed-metrics: false,
- request-backoff: {
- initial: 100,
- max: 10000
- },
- memory: {
- fraction: 0.1,
- min: 64MB,
- max: 1GB,
- buffers-per-channel: 2,
- floating-buffers-per-gate: 16
- }
- }
- }
- ```
-
-* The resulting config keys should hence be:
-
- **NOT** `"taskmanager.detailed.network.metrics"`
-
- **But rather** `"taskmanager.network.detailed-metrics"`
-
-
-### Connectors
-
-Connectors are historically hard to implement and need to deal with many aspects of threading, concurrency, and checkpointing.
-
-As part of [FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface) we are working on making this much simpler for sources. New sources should not have to deal with any aspect of concurrency/threading and checkpointing any more.
-
-A similar FLIP can be expected for sinks in the near future.
-
-
-### Examples
-
-Examples should be self-contained and not require systems other than Flink to run. Except for examples that show how to use specific connectors, like the Kafka connector. Sources/sinks that are ok to use are `StreamExecutionEnvironment.socketTextStream`, which should not be used in production but is quite handy for exploring how things work, and file-based sources/sinks. (For streaming, there is the continuous file source)
-
-Examples should also not be pure toy-examples but strike a balance between real-world code and purely abstract examples. The WordCount example is quite long in the tooth by now but it’s a good showcase of simple code that highlights functionality and can do useful things.
-
-Examples should also be heavy in comments. They should describe the general idea of the example in the class-level Javadoc and describe what is happening and what functionality is used throughout the code. The expected input data and output data should also be described.
-
-Examples should include parameter parsing, so that you can run an example (from the Jar that is created for each example using `bin/flink run path/to/myExample.jar --param1 … --param2`.
-
-
-### Table & SQL API
-
-
-#### Semantics
-
-**The SQL standard should be the main source of truth.**
-
-* Syntax, semantics, and features should be aligned with SQL!
-* We don’t need to reinvent the wheel. Most problems have already been discussed industry-wide and written down in the SQL standard.
-* We rely on the newest standard (SQL:2016 or ISO/IEC 9075:2016 when writing this document [[download]](https://standards.iso.org/ittf/PubliclyAvailableStandards/c065143_ISO_IEC_TR_19075-5_2016.zip) ). Not every part is available online but a quick web search might help here.
-
-Discuss divergence from the standard or vendor-specific interpretations.
-
-* Once a syntax or behavior is defined it cannot be undone easily.
-* Contributions that need to extent or interpret the standard need a thorough discussion with the community.
-* Please help committers by performing some initial research about how other vendors such as Postgres, Microsoft SQL Server, Oracle, Hive, Calcite, Beam are handling such cases.
-
-
-Consider the Table API as a bridge between the SQL and Java/Scala programming world.
-
-* The Table API is an Embedded Domain Specific Language for analytical programs following the relational model.
-It is not required to strictly follow the SQL standard in regards of syntax and names, but can be closer to the way a programming language would do/name functions and features, if that helps make it feel more intuitive.
-* The Table API might have some non-SQL features (e.g. map(), flatMap(), etc.) but should nevertheless “feel like SQL”. Functions and operations should have equal semantics and naming if possible.
-
-
-#### Common mistakes
-
-* Support SQL’s type system when adding a feature.
- * A SQL function, connector, or format should natively support most SQL types from the very beginning.
- * Unsupported types lead to confusion, limit the usability, and create overhead by touching the same code paths multiple times.
- * For example, when adding a `SHIFT_LEFT` function, make sure that the contribution is general enough not only for `INT` but also `BIGINT` or `TINYINT`.
-
-
-#### Testing
-
-Test for nullability.
-
-* SQL natively supports `NULL` for almost every operation and has a 3-valued boolean logic.
-* Make sure to test every feature for nullability as well.
-
-
-Avoid full integration tests
-
-* Spawning a Flink mini-cluster and performing compilation of generated code for a SQL query is expensive.
-* Avoid integration tests for planner tests or variations of API calls.
-* Instead, use unit tests that validate the optimized plan which comes out of a planner. Or test the behavior of a runtime operator directly.
-
-
-#### Compatibility
-
-Don’t introduce physical plan changes in minor releases!
-
-* Backwards compatibility for state in streaming SQL relies on the fact that the physical execution plan remains stable. Otherwise the generated Operator Names/IDs change and state cannot be matched and restored.
-* Every bug fix that leads to changes in the optimized physical plan of a streaming pipeline hences breaks compatibility.
-* As a consequence, changes of the kind that lead to different optimizer plans can only be merged in major releases for now.
-
-
-#### Scala / Java interoperability (legacy code parts)
-
-Keep Java in mind when designing interfaces.
-
-* Consider whether a class will need to interact with a Java class in the future.
-* Use Java collections and Java Optional in interfaces for a smooth integration with Java code.
-* Don’t use features of case classes such as .copy() or apply() for construction if a class is subjected to be converted to Java.
-* Pure Scala user-facing APIs should use pure Scala collections/iterables/etc. for natural and idiomatic (“scalaesk”) integration with Scala.
-
-
-## 9. Java Code Formatting Style
-
-We recommend to set up the IDE to automatically check the code style. Please follow the [IDE setup guide](https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java) for that.
-
-
-### License
-
-* **Apache license headers.** Make sure you have Apache License headers in your files. The RAT plugin is checking for that when you build the code.
-
-### Imports
-
-* **Empty line before and after package declaration.**
-* **No unused imports.**
-* **No redundant imports.**
-* **No wildcard imports.** They can cause problems when adding to the code and in some cases even during refactoring.
-* **Import order.** Imports must be ordered alphabetically, grouped into the following blocks, with each block separated by an empty line:
- * <imports from org.apache.flink.*>
- * <imports from org.apache.flink.shaded.*>
- * <imports from other libraries>
- * <imports from javax.*>
- * <imports from java.*>
- * <imports from scala.*>
- * <static imports>
-
-
-### Naming
-
-* **Package names must start with a letter, and must not contain upper-case letters or special characters.**
- **Non-private static final fields must be upper-case, with words being separated by underscores.**(`MY_STATIC_VARIABLE`)
-* **Non-static fields/methods must be in lower camel case.** (`myNonStaticField`)
-
-
-### Whitespaces
-
-* **Tabs vs. spaces.** We are using tabs for indentation, not spaces.
-We are aware that spaces are a bit nicer; it just happened to be that we started with tabs a long time ago (because Eclipse’s default style used tabs then), and we tried to keep the code base homogeneous (not mix tabs and spaces).
-* **No trailing whitespace.**
-* **Spaces around operators/keywords.** Operators (`+`, `=`, `>`, …) and keywords (`if`, `for`, `catch`, …) must have a space before and after them, provided they are not at the start or end of the line.
-
-
-### Braces
-
-* **Left curly braces (<code>{</code>) must not be placed on a new line.**
-* <strong>Right curly braces (<code>}</code>) must always be placed at the beginning of the line.</strong>
-* <strong>Blocks.</strong> All statements after <code>if</code>, <code>for</code>, <code>while</code>, <code>do</code>, … must always be encapsulated in a block with curly braces (even if the block contains one statement).
-
-
-### Javadocs
-
-* **All public/protected methods and classes must have a Javadoc.**
-* **The first sentence of the Javadoc must end with a period.**
-* **Paragraphs must be separated with a new line, and started with <p>.**
-
-
-### Modifiers
-
-* **No redundant modifiers.** For example, public modifiers in interface methods.
-* **Follow JLS3 modifier order.** Modifiers must be ordered in the following order: public, protected, private, abstract, static, final, transient, volatile, synchronized, native, strictfp.
-
-
-### Files
-
-* **All files must end with <code>\n</code>.**
-* <strong>File length must not exceed 3000 lines.</strong>
-
-
-### Misc
-
-* **Arrays must be defined Java-style.** For example, `public String[] array`.
-* **Use Flink Preconditions.** To increase homogeneity, consistently use the `org.apache.flink.Preconditions` methods `checkNotNull` and `checkArgument` rather than Apache Commons Validate or Google Guava.
-
-
-
-<hr>
-
-[^1]:
- We are keeping such frameworks out of Flink, to make debugging easier and avoid dependency clashes.
diff --git a/contributing/code-style-and-quality-java.md b/contributing/code-style-and-quality-java.md
new file mode 100644
index 0000000..d1e7dea
--- /dev/null
+++ b/contributing/code-style-and-quality-java.md
@@ -0,0 +1,101 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Java"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+## Java Language Features and Libraries
+
+
+### Preconditions and Log Statements
+
+* Never concatenate strings in the parameters
+ * <span style="text-decoration:underline;">Don’t:</span> `Preconditions.checkState(value <= threshold, "value must be below " + threshold)`
+ * <span style="text-decoration:underline;">Don’t:</span> `LOG.debug("value is " + value)`
+ * <span style="text-decoration:underline;">Do:</span> `Preconditions.checkState(value <= threshold, "value must be below %s", threshold)`
+ * <span style="text-decoration:underline;">Do:</span> `LOG.debug("value is {}", value)`
+
+
+### Generics
+
+* **No raw types:** Do not use raw types, unless strictly necessary (sometimes necessary for signature matches, arrays).
+* **Suppress warnings for unchecked conversions:** Add annotations to suppress warnings, if they cannot be avoided (such as “unchecked”, or “serial”). Otherwise warnings about generics flood the build and drown relevant warnings.
+
+
+### equals() / hashCode()
+
+* **equals() / hashCode() should be added when they are well defined only.**
+* They should **not be added to enable a simpler assertion in tests** when they are not well defined. Use hamcrest matchers in that case: [https://github.com/junit-team/junit4/wiki/matchers-and-assertthat](https://github.com/junit-team/junit4/wiki/matchers-and-assertthat)
+* A common indicator that the methods are not well defined is when they take a subset of the fields into account (other than fields that are purely auxiliary).
+* When the methods take mutable fields into account, you often have a design issue. The `equals()`/`hashCode()` methods suggest to use the type as a key, but the signatures suggest it is safe to keep mutating the type.
+
+
+### Java Serialization
+
+* **Do not use Java Serialization for anything !!!**
+* **Do not use Java Serialization for anything !!! !!!**
+* **Do not use Java Serialization for anything !!! !!! !!!**
+* Internal to Flink, Java serialization is used to transport messages and programs through RPC. This is the only case where we use Java serialization. Because of that, some classes need to be serializable (if they are transported via RPC).
+* **Serializable classes must define a Serial Version UID:**
+
+ `private static final long serialVersionUID = 1L;`
+* **The Serial Version UID for new classes should start at 1** and should generally be bumped on every incompatible change to the class according to the Java serialization compatibility definition (i.e: changing the type of a field, or moving the position of a class in the class hierarchy).
+
+
+### Java Reflection
+
+**Avoid using Java’s Reflection API**
+
+* Java’s Reflection API can be a very useful tool in certain cases but in all cases it is a hack and one should research for alternatives. The only cases where Flink should use reflection are
+ * Dynamically loading implementations from another module (like webUI, additional serializers, pluggable query processors).
+ * Extracting types inside the TypeExtractor class. This is fragile enough and should not be done outside the TypeExtractor class.
+ * Some cases of cross-JDK version features, where we need to use reflection because we cannot assume a class/method to be present in all versions.
+* If you need reflection for accessing methods or fields in tests, it usually indicates some deeper architectural issues, like wrong scoping, bad separation of concerns, or that there is no clean way to provide components / dependencies to the class that is tested
+
+
+### Collections
+
+* **ArrayList and ArrayDeque are almost always superior to LinkedList**, except when frequently insert and deleting in the middle of the list
+* **For Maps, avoid patterns that require multiple lookups**
+ * `contains()` before `get()` → `get()` and check null
+ * `contains()` before `put()` → `putIfAbsent()` or `computeIfAbsent()`
+ * Iterating over keys, getting values → iterate over `entrySet()`
+
+
+### Lambdas
+
+* Prefer non-capturing lambdas (lambdas that do not contain references to the outer scope). Capturing lambdas need to create a new object instance for every call. Non-capturing lambdas can use the same instance for each invocation.
+
+ **don’t:**
+ ```
+ map.computeIfAbsent(key, x -> key.toLowerCase())
+ ```
+
+ **do:**
+ ```
+ map.computeIfAbsent(key, k -> k.toLowerCase());
+ ```
+
+* Consider method references instead of inline lambdas
+
+ **don’t**:
+ ```
+ map.computeIfAbsent(key, k-> Loader.load(k));
+ ```
+
+ **do:**
+ ```
+ map.computeIfAbsent(key, Loader::load);
+ ```
+
+
+### Java Streams
+
+* Avoid Java Streams in any performance critical code.
+* The main motivation to use Java Streams would be to improve code readability. As such, they can be a good match in parts of the code that are not data-intensive, but deal with coordination..
+* Even in the latter case, try to limit the scope to a method, or a few private methods within an internal class.
+
+
diff --git a/contributing/code-style-and-quality-java.zh.md b/contributing/code-style-and-quality-java.zh.md
new file mode 100644
index 0000000..d1e7dea
--- /dev/null
+++ b/contributing/code-style-and-quality-java.zh.md
@@ -0,0 +1,101 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Java"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+## Java Language Features and Libraries
+
+
+### Preconditions and Log Statements
+
+* Never concatenate strings in the parameters
+ * <span style="text-decoration:underline;">Don’t:</span> `Preconditions.checkState(value <= threshold, "value must be below " + threshold)`
+ * <span style="text-decoration:underline;">Don’t:</span> `LOG.debug("value is " + value)`
+ * <span style="text-decoration:underline;">Do:</span> `Preconditions.checkState(value <= threshold, "value must be below %s", threshold)`
+ * <span style="text-decoration:underline;">Do:</span> `LOG.debug("value is {}", value)`
+
+
+### Generics
+
+* **No raw types:** Do not use raw types, unless strictly necessary (sometimes necessary for signature matches, arrays).
+* **Suppress warnings for unchecked conversions:** Add annotations to suppress warnings, if they cannot be avoided (such as “unchecked”, or “serial”). Otherwise warnings about generics flood the build and drown relevant warnings.
+
+
+### equals() / hashCode()
+
+* **equals() / hashCode() should be added when they are well defined only.**
+* They should **not be added to enable a simpler assertion in tests** when they are not well defined. Use hamcrest matchers in that case: [https://github.com/junit-team/junit4/wiki/matchers-and-assertthat](https://github.com/junit-team/junit4/wiki/matchers-and-assertthat)
+* A common indicator that the methods are not well defined is when they take a subset of the fields into account (other than fields that are purely auxiliary).
+* When the methods take mutable fields into account, you often have a design issue. The `equals()`/`hashCode()` methods suggest to use the type as a key, but the signatures suggest it is safe to keep mutating the type.
+
+
+### Java Serialization
+
+* **Do not use Java Serialization for anything !!!**
+* **Do not use Java Serialization for anything !!! !!!**
+* **Do not use Java Serialization for anything !!! !!! !!!**
+* Internal to Flink, Java serialization is used to transport messages and programs through RPC. This is the only case where we use Java serialization. Because of that, some classes need to be serializable (if they are transported via RPC).
+* **Serializable classes must define a Serial Version UID:**
+
+ `private static final long serialVersionUID = 1L;`
+* **The Serial Version UID for new classes should start at 1** and should generally be bumped on every incompatible change to the class according to the Java serialization compatibility definition (i.e: changing the type of a field, or moving the position of a class in the class hierarchy).
+
+
+### Java Reflection
+
+**Avoid using Java’s Reflection API**
+
+* Java’s Reflection API can be a very useful tool in certain cases but in all cases it is a hack and one should research for alternatives. The only cases where Flink should use reflection are
+ * Dynamically loading implementations from another module (like webUI, additional serializers, pluggable query processors).
+ * Extracting types inside the TypeExtractor class. This is fragile enough and should not be done outside the TypeExtractor class.
+ * Some cases of cross-JDK version features, where we need to use reflection because we cannot assume a class/method to be present in all versions.
+* If you need reflection for accessing methods or fields in tests, it usually indicates some deeper architectural issues, like wrong scoping, bad separation of concerns, or that there is no clean way to provide components / dependencies to the class that is tested
+
+
+### Collections
+
+* **ArrayList and ArrayDeque are almost always superior to LinkedList**, except when frequently insert and deleting in the middle of the list
+* **For Maps, avoid patterns that require multiple lookups**
+ * `contains()` before `get()` → `get()` and check null
+ * `contains()` before `put()` → `putIfAbsent()` or `computeIfAbsent()`
+ * Iterating over keys, getting values → iterate over `entrySet()`
+
+
+### Lambdas
+
+* Prefer non-capturing lambdas (lambdas that do not contain references to the outer scope). Capturing lambdas need to create a new object instance for every call. Non-capturing lambdas can use the same instance for each invocation.
+
+ **don’t:**
+ ```
+ map.computeIfAbsent(key, x -> key.toLowerCase())
+ ```
+
+ **do:**
+ ```
+ map.computeIfAbsent(key, k -> k.toLowerCase());
+ ```
+
+* Consider method references instead of inline lambdas
+
+ **don’t**:
+ ```
+ map.computeIfAbsent(key, k-> Loader.load(k));
+ ```
+
+ **do:**
+ ```
+ map.computeIfAbsent(key, Loader::load);
+ ```
+
+
+### Java Streams
+
+* Avoid Java Streams in any performance critical code.
+* The main motivation to use Java Streams would be to improve code readability. As such, they can be a good match in parts of the code that are not data-intensive, but deal with coordination..
+* Even in the latter case, try to limit the scope to a method, or a few private methods within an internal class.
+
+
diff --git a/contributing/code-style-and-quality-preamble.zh.md b/contributing/code-style-and-quality-preamble.zh.md
new file mode 100644
index 0000000..3b672fc
--- /dev/null
+++ b/contributing/code-style-and-quality-preamble.zh.md
@@ -0,0 +1,25 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Preamble"
+---
+
+{% include code-style-navbar.md %}
+
+
+This is an attempt to capture the code and quality standard that we want to maintain.
+
+A code contribution (or any piece of code) can be evaluated in various ways: One set of properties is whether the code is correct and efficient. This requires solving the _logical or algorithmic problem_ correctly and well.
+
+Another set of properties is whether the code follows an intuitive design and architecture, whether it is well structured with right separation of concerns, and whether the code is easily understandable and makes its assumptions explicit. That set of properties requires solving the _software engineering problem_ well. A good solution implies that the code is easily testable, maintainable also by other people than the original authors (because it is harder to accidentally break), and effi [...]
+
+While the first set of properties has rather objective approval criteria, the second set of properties is much harder to assess, but is of high importance for an open source project like Apache Flink. To make the code base inviting to many contributors, to make contributions easy to understand for developers that did not write the original code, and to make the code robust in the face of many contributions, well engineered code is crucial.[^1] For well engineered code, it is easier to ke [...]
+
+
+This is of course not a full guide on how to write well engineered code. There is a world of big books that try to capture that. This guide is meant as a checklist of best practices, patterns, anti-patterns, and common mistakes that we observed in the context of developing Flink.
+
+A big part of high-quality open source contributions is about helping the reviewer to understand the contribution and double-check the implications, so an important part of this guide is about how to structure a pull request for review.
+
+
+<hr>
+
+[^1]:
+ In earlier days, we (the Flink community) did not always pay sufficient attention to this, making some components of Flink harder to evolve and to contribute to.
diff --git a/contributing/code-style-and-quality-pull-requests.md b/contributing/code-style-and-quality-pull-requests.md
index 1ab82c9..e297bbd 100644
--- a/contributing/code-style-and-quality-pull-requests.md
+++ b/contributing/code-style-and-quality-pull-requests.md
@@ -8,7 +8,6 @@ title: "Apache Flink Code Style and Quality Guide — Pull Requests & Changes"
**Rationale:** We ask contributors to put in a little bit of extra effort to bring pull requests into a state that they can be more easily and more thoroughly reviewed. This helps the community in many ways:
-
* Reviews are much faster and thus contributions get merged sooner.
* We can ensure higher code quality by overlooking fewer issues in the contributions.
* Committers can review more contributions in the same time, which helps to keep up with the high rate of contributions that Flink is experiencing
@@ -18,14 +17,14 @@ Please understand that contributions that do not follow this guide will take lon
## 1. JIRA issue and Naming
-Make sure that the pull request corresponds to a [JIRA issue]([https://issues.apache.org/jira/projects/FLINK/issues](https://issues.apache.org/jira/projects/FLINK/issues)).
+Make sure that the pull request corresponds to a [JIRA issue]([https://issues.apache.org/jira/projects/FLINK/issues).
Exceptions are ****hotfixes****, like fixing typos in JavaDocs or documentation files.
-Name the pull request in the form `[FLINK-XXXX] [component] Title of the pull request`, where `FLINK-XXXX` should be replaced by the actual issue number. The components should be the same as used in the JIRA issue.
+Name the pull request in the form `[FLINK-XXXX][component] Title of the pull request`, where `FLINK-XXXX` should be replaced by the actual issue number. The components should be the same as used in the JIRA issue.
-Hotfixes should be named for example `[hotfix] [docs] Fix typo in event time introduction` or `[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
+Hotfixes should be named for example `[hotfix][docs] Fix typo in event time introduction` or `[hotfix][javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
## 2. Description
@@ -47,8 +46,6 @@ Pull Requests must put cleanup, refactoring, and core changes into separate comm
Examples for changes that strictly need to go into a separate commit include
-
-
* Cleanup, fixing style and warnings in pre-existing code
* Renaming packages, classes, or methods
* Moving code (to other packages or classes)
@@ -71,7 +68,7 @@ If a pull request does still contain big commits (e.g. a commit with more than 1
## 4. Commit Naming Conventions
Commit messages should follow a similar pattern as the pull request as a whole:
-`[FLINK-XXXX] [component] Commit description`.
+`[FLINK-XXXX][component] Commit description`.
In some cases, the issue might be a subtask here, and the component may be different from the Pull Request’s main component. For example, when the commit introduces an end-to-end test for a runtime change, the PR would be tagged as `[runtime]`, but the individual commit would be tagged as `[e2e]`.
@@ -79,9 +76,9 @@ Examples for commit messages:
* `[hotfix] Fix update_branch_version.sh to allow version suffixes`
* `[hotfix] [table] Remove unused geometry dependency`
-* `[FLINK-11704] [tests] Improve AbstractCheckpointStateOutputStreamTestBase`
-* `[FLINK-10569] [runtime] Remove Instance usage in ExecutionVertexCancelTest`
-* `[FLINK-11702] [table-planner-blink] Introduce a new table type system`
+* `[FLINK-11704][tests] Improve AbstractCheckpointStateOutputStreamTestBase`
+* `[FLINK-10569][runtime] Remove Instance usage in ExecutionVertexCancelTest`
+* `[FLINK-11702][table-planner-blink] Introduce a new table type system`
## 5. Changes to the observable behavior of the system
diff --git a/contributing/code-style-and-quality-scala.md b/contributing/code-style-and-quality-scala.md
new file mode 100644
index 0000000..7c5276e
--- /dev/null
+++ b/contributing/code-style-and-quality-scala.md
@@ -0,0 +1,75 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Scala"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+
+## Scala Language Features
+
+### Where to use (and not use) Scala
+
+**We use Scala for Scala APIs or pure Scala Libraries.**
+
+**We do not use Scala in the core APIs and runtime components. We aim to remove existing Scala use (code and dependencies) from those components.**
+
+⇒ This is not because we do not like Scala, it is a consequence of “the right tool for the right job” approach (see below).
+
+For APIs, we develop the foundation in Java, and layer Scala on top.
+
+* This has traditionally given the best interoperability for both Java and Scala
+* It does mean dedicated effort to keep the Scala API up to date
+
+Why don’t we use Scala in the core APIs and runtime?
+
+* The past has shown that Scala evolves too quickly with tricky changes in functionality. Each Scala version upgrade was a rather big effort process for the Flink community.
+* Scala does not always interact nicely with Java classes, e.g. Scala’s visibility scopes work differently and often expose more to Java consumers than desired
+* Scala adds an additional layer of complexity to artifact/dependency management.
+ * We may want to keep Scala dependent libraries like Akka in the runtime, but abstract them via an interface and load them in a separate classloader, to keep them shielded and avoid version conflicts.
+* Scala makes it very easy for knowledgeable Scala programmers to write code that is very hard to understand for programmers that are less knowledgeable in Scala. That is especially tricky for an open source project with a broad community of diverse experience levels. Working around this means restricting the Scala feature set by a lot, which defeats a good amount of the purpose of using Scala in the first place.
+
+
+### API Parity
+
+Keep Java API and Scala API in sync in terms of functionality and code quality.
+
+The Scala API should cover all the features of the Java APIs as well.
+
+Scala APIs should have a “completeness test”, like the following example from the DataStream API: [https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala](https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala)
+
+
+### Language Features
+
+* **Avoid Scala implicits.**
+ * Scala’s implicits should only be used for user-facing API improvements such as the Table API expressions or type information extraction.
+ * Don’t use them for internal “magic”.
+* **Add explicit types for class members.**
+ * Don’t rely on implicit type inference for class fields and methods return types:
+
+ **Don’t:**
+ ```
+ var expressions = new java.util.ArrayList[String]()
+ ```
+
+ **Do:**
+ ```
+ var expressions: java.util.List[String] = new java.util.ArrayList[]()
+ ```
+
+ * Type inference for local variables on the stack is fine.
+* **Use strict visibility.**
+ * Avoid Scala’s package private features (such as private[flink]) and use regular private/protected instead.
+ * Keep in mind that `private[flink]` and `protected` members are public in Java.
+ * Keep in mind that `private[flink]` still exposes all members in Flink provided examples.
+
+
+### Coding Formatting
+
+**Use line wrapping to structure your code.**
+
+* Scala’s functional nature allows for long transformation chains (`x.map().map().foreach()`).
+* In order to force implementers to structure their code, the line length is therefore limited to 100 characters.
+* Use one line per transformation for better maintainability.
diff --git a/contributing/code-style-and-quality-scala.zh.md b/contributing/code-style-and-quality-scala.zh.md
new file mode 100644
index 0000000..7c5276e
--- /dev/null
+++ b/contributing/code-style-and-quality-scala.zh.md
@@ -0,0 +1,75 @@
+---
+title: "Apache Flink Code Style and Quality Guide — Scala"
+---
+
+{% include code-style-navbar.md %}
+
+{% toc %}
+
+
+
+## Scala Language Features
+
+### Where to use (and not use) Scala
+
+**We use Scala for Scala APIs or pure Scala Libraries.**
+
+**We do not use Scala in the core APIs and runtime components. We aim to remove existing Scala use (code and dependencies) from those components.**
+
+⇒ This is not because we do not like Scala, it is a consequence of “the right tool for the right job” approach (see below).
+
+For APIs, we develop the foundation in Java, and layer Scala on top.
+
+* This has traditionally given the best interoperability for both Java and Scala
+* It does mean dedicated effort to keep the Scala API up to date
+
+Why don’t we use Scala in the core APIs and runtime?
+
+* The past has shown that Scala evolves too quickly with tricky changes in functionality. Each Scala version upgrade was a rather big effort process for the Flink community.
+* Scala does not always interact nicely with Java classes, e.g. Scala’s visibility scopes work differently and often expose more to Java consumers than desired
+* Scala adds an additional layer of complexity to artifact/dependency management.
+ * We may want to keep Scala dependent libraries like Akka in the runtime, but abstract them via an interface and load them in a separate classloader, to keep them shielded and avoid version conflicts.
+* Scala makes it very easy for knowledgeable Scala programmers to write code that is very hard to understand for programmers that are less knowledgeable in Scala. That is especially tricky for an open source project with a broad community of diverse experience levels. Working around this means restricting the Scala feature set by a lot, which defeats a good amount of the purpose of using Scala in the first place.
+
+
+### API Parity
+
+Keep Java API and Scala API in sync in terms of functionality and code quality.
+
+The Scala API should cover all the features of the Java APIs as well.
+
+Scala APIs should have a “completeness test”, like the following example from the DataStream API: [https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala](https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala)
+
+
+### Language Features
+
+* **Avoid Scala implicits.**
+ * Scala’s implicits should only be used for user-facing API improvements such as the Table API expressions or type information extraction.
+ * Don’t use them for internal “magic”.
+* **Add explicit types for class members.**
+ * Don’t rely on implicit type inference for class fields and methods return types:
+
+ **Don’t:**
+ ```
+ var expressions = new java.util.ArrayList[String]()
+ ```
+
+ **Do:**
+ ```
+ var expressions: java.util.List[String] = new java.util.ArrayList[]()
+ ```
+
+ * Type inference for local variables on the stack is fine.
+* **Use strict visibility.**
+ * Avoid Scala’s package private features (such as private[flink]) and use regular private/protected instead.
+ * Keep in mind that `private[flink]` and `protected` members are public in Java.
+ * Keep in mind that `private[flink]` still exposes all members in Flink provided examples.
+
+
+### Coding Formatting
+
+**Use line wrapping to structure your code.**
+
+* Scala’s functional nature allows for long transformation chains (`x.map().map().foreach()`).
+* In order to force implementers to structure their code, the line length is therefore limited to 100 characters.
+* Use one line per transformation for better maintainability.
diff --git a/contributing/code-style-and-quality.zh.md b/contributing/code-style-and-quality.zh.md
deleted file mode 100644
index ed25b86..0000000
--- a/contributing/code-style-and-quality.zh.md
+++ /dev/null
@@ -1,10 +0,0 @@
----
-title: "Code Style and Quality Guide (PLACEHOLDER)"
----
-
-<br />
-<br />
-
-This page is a placeholder for an updated Code Style and Quality Guide to be posted here, once the [discussion on the mailing list](https://lists.apache.org/thread.html/461d0922787c62447c3baff3177a0c8f34fee7c95e2896e5ddf59691@%3Cdev.flink.apache.org%3E) has concluded.
-
-
diff --git a/contributing/contribute-code.md b/contributing/contribute-code.md
index 0b06f99..ac25524 100644
--- a/contributing/contribute-code.md
+++ b/contributing/contribute-code.md
@@ -98,7 +98,7 @@ Apache Flink is maintained, improved, and extended by code contributions of volu
<div class="panel panel-default">
<div class="panel-body">
<h2><span class="number">2</span><a href="#implement">Implement</a></h2>
- <p>Implement the change according to the <a href="{{ site.base }}/contributing/code-style-and-quality.html">Code Style and Quality Guide</a> and the approach agreed upon in the Jira ticket.</p> <br />
+ <p>Implement the change according to the <a href="{{ site.base }}/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a> and the approach agreed upon in the Jira ticket.</p> <br />
<p><b>Only start working on the implementation if there is consensus on the approach (e.g. you are assigned to the ticket)</b></p>
</div>
</div>
@@ -196,7 +196,7 @@ Once you've been assigned to a Jira issue, you may start to implement the requir
Here are some further points to keep in mind while implementing:
- [Set up a Flink development environment](https://cwiki.apache.org/confluence/display/FLINK/Setting+up+a+Flink+development+environment)
-- Follow the [Code Style and Quality Guide]({{ site.base }}/contributing/code-style-and-quality.html) of Flink
+- Follow the [Code Style and Quality Guide]({{ site.base }}/contributing/code-style-and-quality-preamble.html) of Flink
- Take any discussions and requirements from the Jira issue or design document into account.
- Do not mix unrelated issues into one contribution.
diff --git a/contributing/reviewing-prs.md b/contributing/reviewing-prs.md
index 308f785..2ef4812 100644
--- a/contributing/reviewing-prs.md
+++ b/contributing/reviewing-prs.md
@@ -82,7 +82,7 @@ This is the detailed code review of the actual changes, covering:
* Does the code avoid to introduce additional compiler warnings?
* If dependencies have been changed, were the NOTICE files updated?
-Code guidelines can be found in the [Flink Code Style and Quality Guide]({{ site.baseurl }}/contributing/code-style-and-quality.html).
+Code guidelines can be found in the [Flink Code Style and Quality Guide]({{ site.baseurl }}/contributing/code-style-and-quality-preamble.html).
----
[flink-web] 02/04: Address PR comments
Posted by rm...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
rmetzger pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit d6ad8618388a57faca6cd82ecc43ee26fb71d857
Author: Robert Metzger <rm...@apache.org>
AuthorDate: Mon Jul 15 10:36:26 2019 +0200
Address PR comments
---
contributing/code-style-and-quality-general.md | 6 +++---
contributing/code-style-and-quality-preamble.md | 2 --
2 files changed, 3 insertions(+), 5 deletions(-)
diff --git a/contributing/code-style-and-quality-general.md b/contributing/code-style-and-quality-general.md
index e782723..0d02733 100644
--- a/contributing/code-style-and-quality-general.md
+++ b/contributing/code-style-and-quality-general.md
@@ -153,7 +153,7 @@ For fields and parameters, `Optional` is disputed in Java and most parts of the
Code that is easily testable typically has good separation of concerns and is structured to be reusable outside the original context (by being easily reusable in tests).
-A good summary or problems / symptoms and recommended refactoring is in this PDF.
+A good summary or problems / symptoms and recommended refactoring is in the PDF linked below.
Please note that while the examples in the PDF often use a dependency injection framework (Guice), it works in the same way without such a framework.[^1]
[http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf](http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf)
@@ -340,7 +340,7 @@ Examples are in the RPC system, Network Stack, in the Task’s mailbox model, or
* Java’s Reflection API can be a very useful tool in certain cases but in all cases it is a hack and one should research for alternatives. The only cases where Flink should use reflection are
* Dynamically loading implementations from another module (like webUI, additional serializers, pluggable query processors).
- * Extracting types inside the TypeExtractor class. This is fragile enough any should not be done outside the TypeExtractor class.
+ * Extracting types inside the TypeExtractor class. This is fragile enough and should not be done outside the TypeExtractor class.
* Some cases of cross-JDK version features, where we need to use reflection because we cannot assume a class/method to be present in all versions.
* If you need reflection for accessing methods or fields in tests, it usually indicates some deeper architectural issues, like wrong scoping, bad separation of concerns, or that there is no clean way to provide components / dependencies to the class that is tested
@@ -505,7 +505,7 @@ How to name config keys:
Connectors are historically hard to implement and need to deal with many aspects of threading, concurrency, and checkpointing.
-As part of [[FLIP-27]](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface) we are working on making this much simpler for sources. New sources should not have to deal with any aspect of concurrency/threading and checkpointing any more.
+As part of [FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface) we are working on making this much simpler for sources. New sources should not have to deal with any aspect of concurrency/threading and checkpointing any more.
A similar FLIP can be expected for sinks in the near future.
diff --git a/contributing/code-style-and-quality-preamble.md b/contributing/code-style-and-quality-preamble.md
index bc2a9aa..3b672fc 100644
--- a/contributing/code-style-and-quality-preamble.md
+++ b/contributing/code-style-and-quality-preamble.md
@@ -5,8 +5,6 @@ title: "Apache Flink Code Style and Quality Guide — Preamble"
{% include code-style-navbar.md %}
-_Preamble_
-
This is an attempt to capture the code and quality standard that we want to maintain.
A code contribution (or any piece of code) can be evaluated in various ways: One set of properties is whether the code is correct and efficient. This requires solving the _logical or algorithmic problem_ correctly and well.
[flink-web] 04/04: rebuild site
Posted by rm...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
rmetzger pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/flink-web.git
commit 4c53ac026967abed7ea778bd3553ea46dcdc1b35
Author: Robert Metzger <rm...@apache.org>
AuthorDate: Wed Aug 7 14:25:07 2019 +0200
rebuild site
---
.../code-style-and-quality-common.html | 620 +++++++++++++++++++++
.../code-style-and-quality-components.html | 415 ++++++++++++++
...html => code-style-and-quality-formatting.html} | 176 ++++--
.../contributing/code-style-and-quality-java.html | 385 +++++++++++++
...y.html => code-style-and-quality-preamble.html} | 78 ++-
.../code-style-and-quality-pull-requests.html | 352 ++++++++++++
...tion.html => code-style-and-quality-scala.html} | 174 ++++--
content/contributing/contribute-code.html | 6 +-
content/contributing/contribute-documentation.html | 2 +-
content/contributing/how-to-contribute.html | 2 +-
content/contributing/improve-website.html | 2 +-
content/contributing/reviewing-prs.html | 4 +-
.../code-style-and-quality-common.html | 618 ++++++++++++++++++++
.../code-style-and-quality-components.html | 413 ++++++++++++++
...html => code-style-and-quality-formatting.html} | 176 ++++--
.../contributing/code-style-and-quality-java.html | 383 +++++++++++++
...y.html => code-style-and-quality-preamble.html} | 78 ++-
...tion.html => code-style-and-quality-scala.html} | 174 ++++--
content/zh/contributing/contribute-code.html | 2 +-
.../zh/contributing/contribute-documentation.html | 2 +-
content/zh/contributing/how-to-contribute.html | 2 +-
content/zh/contributing/improve-website.html | 2 +-
content/zh/contributing/reviewing-prs.html | 2 +-
23 files changed, 3819 insertions(+), 249 deletions(-)
diff --git a/content/contributing/code-style-and-quality-common.html b/content/contributing/code-style-and-quality-common.html
new file mode 100644
index 0000000..58dd44c
--- /dev/null
+++ b/content/contributing/code-style-and-quality-common.html
@@ -0,0 +1,620 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Common Rules</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/flink-architecture.html">What is Apache Flink?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/usecases.html">Use Cases</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/poweredby.html">Powered By</a></li>
+
+ <!-- FAQ -->
+ <li><a href="/faq.html">FAQ</a></li>
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/downloads.html">Downloads</a></li>
+
+ <!-- Quickstart -->
+ <li>
+ <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8/quickstart/setup_quickstart.html" target="_blank">Tutorials <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8" target="_blank">1.8 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/gettinghelp.html">Getting Help</a></li>
+
+ <!-- Blog -->
+ <li><a href="/blog/"><b>Flink Blog</b></a></li>
+
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/community.html">Community & Project Info</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/roadmap.html">Roadmap</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/contributing/how-to-contribute.html">How to Contribute</a></li>
+
+ <ul class="nav navbar-nav navbar-subnav">
+ <li >
+ <a href="/contributing/contribute-code.html">Contribute Code</a>
+ </li>
+ <li >
+ <a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
+ </li>
+ <li >
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
+ </li>
+ <li >
+ <a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
+ </li>
+ <li >
+ <a href="/contributing/improve-website.html">Contribute to the Website</a>
+ </li>
+ </ul>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+
+ <a href="/zh/contributing/code-style-and-quality-common.html">中文版</a>
+
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <h1>Apache Flink Code Style and Quality Guide — Common Rules</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
+
+<hr />
+
+<div class="page-toc">
+<ul id="markdown-toc">
+ <li><a href="#tools" id="markdown-toc-tools">1. Tools</a> <ul>
+ <li><a href="#warnings" id="markdown-toc-warnings">Warnings</a></li>
+ </ul>
+ </li>
+ <li><a href="#comments-and-code-readability" id="markdown-toc-comments-and-code-readability">2. Comments And Code Readability</a> <ul>
+ <li><a href="#comments" id="markdown-toc-comments">Comments</a></li>
+ <li><a href="#branches-and-nesting" id="markdown-toc-branches-and-nesting">Branches and Nesting</a></li>
+ </ul>
+ </li>
+ <li><a href="#design-and-structure" id="markdown-toc-design-and-structure">3. Design and Structure</a> <ul>
+ <li><a href="#immutability-and-eager-initialization" id="markdown-toc-immutability-and-eager-initialization">Immutability and Eager Initialization</a></li>
+ <li><a href="#nullability-of-the-mutable-parts" id="markdown-toc-nullability-of-the-mutable-parts">Nullability of the Mutable Parts</a></li>
+ <li><a href="#avoid-code-duplication" id="markdown-toc-avoid-code-duplication">Avoid Code Duplication</a></li>
+ <li><a href="#design-for-testability" id="markdown-toc-design-for-testability">Design for Testability</a></li>
+ <li><a href="#performance-awareness" id="markdown-toc-performance-awareness">Performance Awareness</a></li>
+ </ul>
+ </li>
+ <li><a href="#concurrency-and-threading" id="markdown-toc-concurrency-and-threading">4. Concurrency and Threading</a></li>
+ <li><a href="#dependencies-and-modules" id="markdown-toc-dependencies-and-modules">5. Dependencies and Modules</a></li>
+</ul>
+
+</div>
+
+<h2 id="tools">1. Tools</h2>
+
+<p>We recommend to follow the <a href="https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java">IDE Setup Guide</a> to get IDE tooling configured.</p>
+
+<!---
+### Use inspections in IntelliJ
+
+* Import the inspections settings into the IDE (see IDE setup guide)
+ * TODO: Need to agree on a profile and export it (like checkstyle)
+* Write the code such that inspection warnings are addressed
+ * There are few exceptions where an inspection warning is not meaningful. In that case, suppress the inspection warning.
+-->
+
+<h3 id="warnings">Warnings</h3>
+
+<ul>
+ <li>We strive for zero warnings</li>
+ <li>Even though there are many warnings in existing code, new changes should not add any additional compiler warnings</li>
+ <li>If it is not possible to address the warning in a sane way (in some cases when working with generics) add an annotation to suppress the warning</li>
+ <li>When deprecating methods, check that this does not introduce additional warnings</li>
+</ul>
+
+<h2 id="comments-and-code-readability">2. Comments And Code Readability</h2>
+
+<h3 id="comments">Comments</h3>
+
+<p><strong>Golden rule: Comment as much as necessary to support code understanding, but don’t add redundant information.</strong></p>
+
+<p>Think about</p>
+
+<ul>
+ <li><span style="text-decoration:underline;">What</span> is the code doing?</li>
+ <li><span style="text-decoration:underline;">How</span> does the code do this?</li>
+ <li><span style="text-decoration:underline;">Why</span> is the code like that?</li>
+</ul>
+
+<p>The code alone should explain as much as possible the “<span style="text-decoration:underline;">what</span>” and the “<span style="text-decoration:underline;">how</span>”</p>
+
+<ul>
+ <li>Use JavaDocs to describe the roles of classes and the contracts of methods, in cases where the contract is not obvious or intuitive from the method name (the “what”).</li>
+ <li>The flow of the code should give a good description of the “how”.
+Think of variable and method names as part of the code documenting itself.</li>
+ <li>It often makes reading the code easier if larger blocks that form a unit are moved into a private method with a descriptive name of what that block is doing</li>
+</ul>
+
+<p>In-code comments help explain the <span style="text-decoration:underline;">“why”</span></p>
+
+<ul>
+ <li>For example <code>// this specific code layout helps the JIT to better do this or that</code></li>
+ <li>Or <code>// nulling out this field here means future write attempts are fail-fast</code></li>
+ <li>Or <code>// for arguments with which this method is actually called, this seemingly naive approach works actually better than any optimized/smart version</code></li>
+</ul>
+
+<p>In-code comments should not state redundant information about the “what” and “how” that is already obvious in the code itself.</p>
+
+<p>JavaDocs should not state meaningless information (just to satisfy the Checkstyle checker).</p>
+
+<p><strong>Don’t:</strong></p>
+
+<div class="highlight"><pre><code>/**
+ * The symbol expression.
+ */
+public class CommonSymbolExpression {}
+</code></pre></div>
+<p><strong>Do:</strong></p>
+
+<div class="highlight"><pre><code>/**
+ * An expression that wraps a single specific symbol.
+ * A symbol could be a unit, an alias, a variable, etc.
+ */
+public class CommonSymbolExpression {}
+</code></pre></div>
+
+<h3 id="branches-and-nesting">Branches and Nesting</h3>
+
+<p>Avoid deep nesting of scopes, by flipping the if condition and exiting early.</p>
+
+<p><strong>Don’t:</strong></p>
+
+<div class="highlight"><pre><code>if (a) {
+ if (b) {
+ if (c) {
+ the main path
+ }
+ }
+}
+</code></pre></div>
+
+<p><strong>Do</strong></p>
+
+<div class="highlight"><pre><code>if (!a) {
+ return ..
+}
+
+if (!b) {
+ return ...
+}
+
+if (!c) {
+ return ...
+}
+
+the main path
+</code></pre></div>
+
+<h2 id="design-and-structure">3. Design and Structure</h2>
+
+<p>While it is hard to exactly specify what constitutes a good design, there are some properties that can serve as a <em>litmus test</em> for a good design. If these properties are given, the chances are good that the design is going into a good direction. If these properties cannot be achieved, there is a high probability that the design is flawed.</p>
+
+<h3 id="immutability-and-eager-initialization">Immutability and Eager Initialization</h3>
+
+<ol>
+ <li>Try to use immutable types where possible, especially for APIs, messages, identifiers, properties, configuration, etc.</li>
+ <li>A good general approach is to try and make as many fields of a class <code>final</code> as possible.</li>
+ <li>Classes that are used as keys in maps should be strictly immutable and only have <code>final</code> fields (except maybe auxiliary fields, like lazy cached hash codes).</li>
+ <li>Eagerly initialize classes. There should be no <code>init()</code> or <code>setup()</code> methods. Once the constructor completes, the object should be usable.</li>
+</ol>
+
+<h3 id="nullability-of-the-mutable-parts">Nullability of the Mutable Parts</h3>
+
+<p>For nullability, the Flink codebase aims to follow these conventions:</p>
+
+<ul>
+ <li>Fields, parameters, and return types are always non-null, unless indicated otherwise</li>
+ <li>All fields, parameters and method types that can be null should be annotated with <code>@javax.annotation.Nullable</code>.
+That way, you get warnings from IntelliJ about all sections where you have to reason about potential null values.</li>
+ <li>For all mutable (non-final) fields that are not annotated, the assumption is that while the field value changes, there always is a value.
+ <ul>
+ <li>This should be double check whether these can in fact not be null throughout the lifetime of the object.</li>
+ </ul>
+ </li>
+</ul>
+
+<p><em>Note: This means that <code>@Nonnull</code> annotations are usually not necessary, but can be used in certain cases to override a previous annotation, or to point non-nullability out in a context where one would expect a nullable value.</em></p>
+
+<p><code>Optional</code> is a good solution as a return type for method that may or may not have a result, so nullable return types are good candidates to be replaced with <code>Optional</code>.
+For fields and parameters, <code>Optional</code> is disputed in Java and most parts of the Flink code case don’t use optional for fields.</p>
+
+<h3 id="avoid-code-duplication">Avoid Code Duplication</h3>
+
+<ol>
+ <li>Whenever you are about to copy/paste some code, or reproduce a similar type of functionality in a different place, think about the ways how to refactor/reuse/abstract the changes to avoid the duplication.</li>
+ <li>Common behavior between different specializations should be shared in a common component (or a shared superclass).</li>
+ <li>Always use “private static final” constants instead of duplicating strings or other special values at different locations. Constants should be declared in the top member area of a class.</li>
+</ol>
+
+<h3 id="design-for-testability">Design for Testability</h3>
+
+<p>Code that is easily testable typically has good separation of concerns and is structured to be reusable outside the original context (by being easily reusable in tests).</p>
+
+<p>A good summary or problems / symptoms and recommended refactoring is in the PDF linked below.
+Please note that while the examples in the PDF often use a dependency injection framework (Guice), it works in the same way without such a framework.<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup></p>
+
+<p><a href="http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf">http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf</a></p>
+
+<p>Here is a compact summary of the most important aspects.</p>
+
+<p><strong>Inject dependencies</strong></p>
+
+<p>Reusability becomes easier if constructors don’t create their dependencies (the objects assigned to the fields), but accept them as parameters.</p>
+
+<ul>
+ <li>Effectively, constructors should have no <code>new</code> keyword.</li>
+ <li>Exceptions are creating a new empty collection (<code>new ArrayList<>()</code>) or similar auxiliary fields (objects that have only primitive dependencies).</li>
+</ul>
+
+<p>To make instantiation easy / readable, add factory methods or additional convenience constructors to construct whole object with dependencies.</p>
+
+<p>In no case should it ever be required to use a reflection or a “Whitebox” util to change the fields of an object in a test, or to use PowerMock to intercept a “new” call and supply a mock.</p>
+
+<p><strong>Avoid “too many collaborators”</strong></p>
+
+<p>If you have to take a big set of other components into account during testing (“too many collaborators”), consider refactoring.</p>
+
+<p>The component/class you want to test probably depends on another broad component (and its implementation), rather than on the minimal interface (abstraction) required for its work.</p>
+
+<p>In that case, segregate the interfaces (factor out the minimal required interface) and supply a test stub in that case.</p>
+
+<ul>
+ <li>For example, if testing a S3RecoverableMultiPartUploader requires actual S3 access
+then the S3 access should be factored out into an interface and test should replace it by a test stub</li>
+ <li>This naturally requires to be able to inject dependencies (see above)</li>
+</ul>
+
+<p>⇒ Please note that these steps often require more effort in implementing tests (factoring out interfaces, creating dedicated test stubs), but make the tests more resilient to changes in other components, i.e., you do not need to touch the tests when making unrelated changes.</p>
+
+<p><strong>Write targeted tests</strong></p>
+
+<ul>
+ <li><span style="text-decoration:underline;">Test contracts not implementations</span>: Test that after a sequence of actions, the components are in a certain state, rather than testing that the components followed a sequence of internal state modifications.
+ <ul>
+ <li>For example, a typical antipattern is to check whether one specific method was called as part of the test</li>
+ </ul>
+ </li>
+ <li>
+ <p>A way to enforce this is to try to follow the <em>Arrange</em>, <em>Act</em>, <em>Assert</em> test structure when writing a unit test (<a href="https://xp123.com/articles/3a-arrange-act-assert/">https://xp123.com/articles/3a-arrange-act-assert/</a>)</p>
+
+ <p>This helps to communicate the intention of the test (what is the scenario under test) rather than the mechanics of the tests. The technical bits go to a static methods at the bottom of the test class.</p>
+
+ <p>Example of tests in Flink that follow this pattern are:</p>
+
+ <ul>
+ <li><a href="https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java">https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java</a></li>
+ <li><a href="https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java">https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java</a></li>
+ </ul>
+ </li>
+</ul>
+
+<p><strong>Avoid Mockito - Use reusable test implementations</strong></p>
+
+<ul>
+ <li>Mockito-based tests tend to be costly to maintain in the long run by encouraging duplication of functionality and testing for implementation rather than effect
+ <ul>
+ <li>More details: <a href="https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8">https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8</a></li>
+ </ul>
+ </li>
+ <li>Instead, create reusable test implementations and utilities
+ <ul>
+ <li>That way, when some class changes, we only have to update a few test utils or mocks</li>
+ </ul>
+ </li>
+</ul>
+
+<h3 id="performance-awareness">Performance Awareness</h3>
+
+<p>We can conceptually distinguish between code that “coordinates” and code that “processes data”. Code that coordinates should always favor simplicity and cleanness. Data processing code is highly performance critical and should optimize for performance.</p>
+
+<p>That means still applying the general idea of the sections above, but possibly forgoing some aspects in some place, in order to achieve higher performance.</p>
+
+<p><strong>Which code paths are Data Processing paths?</strong></p>
+
+<ul>
+ <li><span style="text-decoration:underline;">Per-record code paths:</span> Methods and code paths that are called for each record. Found for example in Connectors, Serializers, State Backends, Formats, Tasks, Operators, Metrics, runtime data structures, etc.</li>
+ <li><span style="text-decoration:underline;">I/O methods:</span> Transferring messages or chunks of data in buffers. Examples are in the RPC system, Network Stack, FileSystems, Encoders / Decoders, etc.</li>
+</ul>
+
+<p><strong>Things that performance critical code may do that we would otherwise avoid</strong></p>
+
+<ul>
+ <li>Using (and reusing) mutable objects to take pressure off the GC (and sometimes help with cache locality), thus forgoing the strive for immutability.</li>
+ <li>Using primitive types, arrays of primitive types, or MemorySegment/ByteBuffer and encoding meaning into the primitive types and byte sequences, rather than encapsulating the behavior in dedicated classes and using objects.</li>
+ <li>Structuring the code to amortize expensive work (allocations, lookups, virtual method calls, …) across multiple records, for example by doing the work once per buffer/bundle/batch.</li>
+ <li>Code layout optimized for the JIT rather than for readability. Examples are inlining fields from other classes (in cases where it is doubtful whether the JIT would do that optimization at runtime), or structuring code to help the JIT compiler with inlining, loop unrolling, vectorization, etc.</li>
+</ul>
+
+<h2 id="concurrency-and-threading">4. Concurrency and Threading</h2>
+
+<p><strong>Most code paths should not require any concurrency.</strong> The right internal abstractions should obviate the need for concurrency in almost all cases.</p>
+
+<ul>
+ <li>The Flink core and runtime use concurrency to provide these building blocks.
+Examples are in the RPC system, Network Stack, in the Task’s mailbox model, or some predefined Source / Sink utilities.</li>
+ <li>We are not fully there, but any new addition that introduces implements its own concurrency should be under scrutiny, unless it falls into the above category of core system building blocks.</li>
+ <li>Contributors should reach out to committers if they feel they need to implement concurrent code to see if there is an existing abstraction/building-block, or if one should be added.</li>
+</ul>
+
+<p><strong>When developing a component think about threading model and synchronization points ahead.</strong></p>
+
+<ul>
+ <li>For example: single threaded, blocking, non-blocking, synchronous, asynchronous, multi threaded, thread pool, message queues, volatile, synchronized block/methods, mutexes, atomics, callbacks, …</li>
+ <li>Getting those things right and thinking about them ahead is even more important than designing classes interfaces/responsibilities, since it’s much harder to change later on.</li>
+</ul>
+
+<p><strong>Try to avoid using threads all together if possible in any way.</strong></p>
+
+<ul>
+ <li>If you feel you have a case for spawning a thread, point this out in the pull request as something to be explicitly reviewed.</li>
+</ul>
+
+<p><strong>Be aware that using threads is in fact much harder than it initially looks</strong></p>
+
+<ul>
+ <li>Clean shutdown of threads is very tricky.</li>
+ <li>Handling interruptions in a rock solid fashion (avoid both slow shutdown and live locks) requires almost a Java Wizard</li>
+ <li>Ensuring clean error propagation out of threads in all cases needs thorough design.</li>
+ <li>Complexity of multi-threaded application/component/class grows exponentially, with each additional synchronisation point/block/critical section. Your code initially might be easy enough to understand, but can quickly grow beyond that point.</li>
+ <li>Proper testing of multithreaded code is basically impossible, while alternative approaches (like asynchronous code, non-blocking code, actor model with message queues) are quite easy to test.</li>
+ <li>Usually multi-threaded code is often even less efficient compared to alternative approaches on modern hardware.</li>
+</ul>
+
+<p><strong>Be aware of the java.util.concurrent.CompletableFuture</strong></p>
+
+<ul>
+ <li>Like with other concurrent code, there should rarely be the need to use a CompletableFuture</li>
+ <li>Completing a future would also complete on the calling thread any chained futures that are waiting for the result to be completed, unless a completion executor specified explicitly.</li>
+ <li>This can be intentional, if the entire execution should be synchronous / single-threaded, as for example in parts of the Scheduler / ExecutionGraph.
+ <ul>
+ <li>Flink even makes use of a “main-thread executor” to allow calling chained handlers in the same thread as a single-threaded RPC endpoint runs</li>
+ </ul>
+ </li>
+ <li>This can be unexpected, if the thread that completes the future is a sensitive thread.
+ <ul>
+ <li>It may be better to use <code>CompletableFuture.supplyAsync(value, executor)</code> in that case, instead of <code>future.complete(value)</code> when an executor is available</li>
+ </ul>
+ </li>
+ <li>When blocking on a future awaiting completion, always supply a timeout for a result instead of waiting indefinitely, and handle timeouts explicitly.</li>
+ <li>Use <code>CompletableFuture.allOf()</code>/<code>anyOf()</code>, <code>ExecutorCompletionService</code>, or <code>org.apache.flink.runtime.concurrent.FutureUtils#waitForAll</code> if you need to wait for: all the results/any of the results/all the results but handled by (approximate) completion order.</li>
+</ul>
+
+<h2 id="dependencies-and-modules">5. Dependencies and Modules</h2>
+
+<ul>
+ <li><strong>Keep the dependency footprint small</strong>
+ <ul>
+ <li>The more dependencies the harder it gets for the community to manage them as a whole.</li>
+ <li>Dependency management includes dependency conflicts, maintaining licenses and related notices, and handling security vulnerabilities.</li>
+ <li>Discuss whether the dependency should be shaded/relocated to avoid future conflicts.</li>
+ </ul>
+ </li>
+ <li><strong>Don’t add a dependency for just one method</strong>
+ <ul>
+ <li>Use Java built-in means if possible.</li>
+ <li>If the method is Apache-licensed, you can copy the method into a Flink utility class with proper attribution.</li>
+ </ul>
+ </li>
+ <li><strong>Declaration of dependencies</strong>
+ <ul>
+ <li>Declare dependencies that you explicitly rely on, whether it provides classes you directly import and use or it’s something that provides a service you directly use, like Log4J.</li>
+ <li>Transitive dependencies should only supply dependencies that are needed at runtime but that you don’t use yourself.</li>
+ <li>[<a href="https://stackoverflow.com/questions/15177661/maven-transitive-dependencies">source</a>]</li>
+ </ul>
+ </li>
+ <li><strong>Location of classes in the Maven modules</strong>
+ <ul>
+ <li>Whenever you create a new class, think about where to put it.</li>
+ <li>A class might be used by multiple modules in the future and might belong into a <code>common</code> module in this case.</li>
+ </ul>
+ </li>
+</ul>
+
+<hr />
+
+<div class="footnotes">
+ <ol>
+ <li id="fn:1">
+
+ <p>We are keeping such frameworks out of Flink, to make debugging easier and avoid dependency clashes. <a href="#fnref:1" class="reversefootnote">↩</a></p>
+ </li>
+ </ol>
+</div>
+
+
+ </div>
+</div>
+
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/contributing/code-style-and-quality-components.html b/content/contributing/code-style-and-quality-components.html
new file mode 100644
index 0000000..1d8521d
--- /dev/null
+++ b/content/contributing/code-style-and-quality-components.html
@@ -0,0 +1,415 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Components</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/flink-architecture.html">What is Apache Flink?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/usecases.html">Use Cases</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/poweredby.html">Powered By</a></li>
+
+ <!-- FAQ -->
+ <li><a href="/faq.html">FAQ</a></li>
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/downloads.html">Downloads</a></li>
+
+ <!-- Quickstart -->
+ <li>
+ <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8/quickstart/setup_quickstart.html" target="_blank">Tutorials <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8" target="_blank">1.8 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/gettinghelp.html">Getting Help</a></li>
+
+ <!-- Blog -->
+ <li><a href="/blog/"><b>Flink Blog</b></a></li>
+
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/community.html">Community & Project Info</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/roadmap.html">Roadmap</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/contributing/how-to-contribute.html">How to Contribute</a></li>
+
+ <ul class="nav navbar-nav navbar-subnav">
+ <li >
+ <a href="/contributing/contribute-code.html">Contribute Code</a>
+ </li>
+ <li >
+ <a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
+ </li>
+ <li >
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
+ </li>
+ <li >
+ <a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
+ </li>
+ <li >
+ <a href="/contributing/improve-website.html">Contribute to the Website</a>
+ </li>
+ </ul>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+
+ <a href="/zh/contributing/code-style-and-quality-components.html">中文版</a>
+
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <h1>Apache Flink Code Style and Quality Guide — Components</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
+
+<hr />
+
+<div class="page-toc">
+<ul id="markdown-toc">
+ <li><a href="#component-specific-guidelines" id="markdown-toc-component-specific-guidelines">Component Specific Guidelines</a> <ul>
+ <li><a href="#configuration-changes" id="markdown-toc-configuration-changes">Configuration Changes</a></li>
+ <li><a href="#connectors" id="markdown-toc-connectors">Connectors</a></li>
+ <li><a href="#examples" id="markdown-toc-examples">Examples</a></li>
+ <li><a href="#table--sql-api" id="markdown-toc-table--sql-api">Table & SQL API</a></li>
+ </ul>
+ </li>
+</ul>
+
+</div>
+
+<h2 id="component-specific-guidelines">Component Specific Guidelines</h2>
+
+<p><em>Additional guidelines about changes in specific components.</em></p>
+
+<h3 id="configuration-changes">Configuration Changes</h3>
+
+<p>Where should the config option go?</p>
+
+<ul>
+ <li>
+ <p><span style="text-decoration:underline;">‘flink-conf.yaml’:</span> All configuration that pertains to execution behavior that one may want to standardize across jobs. Think of it as parameters someone would set wearing an “ops” hat, or someone that provides a stream processing platform to other teams.</p>
+ </li>
+ <li><span style="text-decoration:underline;">‘ExecutionConfig’</span>: Parameters specific to an individual Flink application, needed by the operators during execution. Typical examples are watermark interval, serializer parameters, object reuse.</li>
+ <li><span style="text-decoration:underline;">ExecutionEnvironment (in code)</span>: Everything that is specific to an individual Flink application and is only needed to build program / dataflow, not needed inside the operators during execution.</li>
+</ul>
+
+<p>How to name config keys:</p>
+
+<ul>
+ <li>
+ <p>Config key names should be hierarchical.
+Think of the configuration as nested objects (JSON style)</p>
+
+ <div class="highlight"><pre><code>taskmanager: {
+ jvm-exit-on-oom: true,
+ network: {
+ detailed-metrics: false,
+ request-backoff: {
+ initial: 100,
+ max: 10000
+ },
+ memory: {
+ fraction: 0.1,
+ min: 64MB,
+ max: 1GB,
+ buffers-per-channel: 2,
+ floating-buffers-per-gate: 16
+ }
+ }
+}
+</code></pre></div>
+ </li>
+ <li>
+ <p>The resulting config keys should hence be:</p>
+
+ <p><strong>NOT</strong> <code>"taskmanager.detailed.network.metrics"</code></p>
+
+ <p><strong>But rather</strong> <code>"taskmanager.network.detailed-metrics"</code></p>
+ </li>
+</ul>
+
+<h3 id="connectors">Connectors</h3>
+
+<p>Connectors are historically hard to implement and need to deal with many aspects of threading, concurrency, and checkpointing.</p>
+
+<p>As part of <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface">FLIP-27</a> we are working on making this much simpler for sources. New sources should not have to deal with any aspect of concurrency/threading and checkpointing any more.</p>
+
+<p>A similar FLIP can be expected for sinks in the near future.</p>
+
+<h3 id="examples">Examples</h3>
+
+<p>Examples should be self-contained and not require systems other than Flink to run. Except for examples that show how to use specific connectors, like the Kafka connector. Sources/sinks that are ok to use are <code>StreamExecutionEnvironment.socketTextStream</code>, which should not be used in production but is quite handy for exploring how things work, and file-based sources/sinks. (For streaming, there is the continuous file source)</p>
+
+<p>Examples should also not be pure toy-examples but strike a balance between real-world code and purely abstract examples. The WordCount example is quite long in the tooth by now but it’s a good showcase of simple code that highlights functionality and can do useful things.</p>
+
+<p>Examples should also be heavy in comments. They should describe the general idea of the example in the class-level Javadoc and describe what is happening and what functionality is used throughout the code. The expected input data and output data should also be described.</p>
+
+<p>Examples should include parameter parsing, so that you can run an example (from the Jar that is created for each example using <code>bin/flink run path/to/myExample.jar --param1 … --param2</code>.</p>
+
+<h3 id="table--sql-api">Table & SQL API</h3>
+
+<h4 id="semantics">Semantics</h4>
+
+<p><strong>The SQL standard should be the main source of truth.</strong></p>
+
+<ul>
+ <li>Syntax, semantics, and features should be aligned with SQL!</li>
+ <li>We don’t need to reinvent the wheel. Most problems have already been discussed industry-wide and written down in the SQL standard.</li>
+ <li>We rely on the newest standard (SQL:2016 or ISO/IEC 9075:2016 when writing this document <a href="https://standards.iso.org/ittf/PubliclyAvailableStandards/c065143_ISO_IEC_TR_19075-5_2016.zip">[download]</a> ). Not every part is available online but a quick web search might help here.</li>
+</ul>
+
+<p>Discuss divergence from the standard or vendor-specific interpretations.</p>
+
+<ul>
+ <li>Once a syntax or behavior is defined it cannot be undone easily.</li>
+ <li>Contributions that need to extent or interpret the standard need a thorough discussion with the community.</li>
+ <li>Please help committers by performing some initial research about how other vendors such as Postgres, Microsoft SQL Server, Oracle, Hive, Calcite, Beam are handling such cases.</li>
+</ul>
+
+<p>Consider the Table API as a bridge between the SQL and Java/Scala programming world.</p>
+
+<ul>
+ <li>The Table API is an Embedded Domain Specific Language for analytical programs following the relational model.
+It is not required to strictly follow the SQL standard in regards of syntax and names, but can be closer to the way a programming language would do/name functions and features, if that helps make it feel more intuitive.</li>
+ <li>The Table API might have some non-SQL features (e.g. map(), flatMap(), etc.) but should nevertheless “feel like SQL”. Functions and operations should have equal semantics and naming if possible.</li>
+</ul>
+
+<h4 id="common-mistakes">Common mistakes</h4>
+
+<ul>
+ <li>Support SQL’s type system when adding a feature.
+ <ul>
+ <li>A SQL function, connector, or format should natively support most SQL types from the very beginning.</li>
+ <li>Unsupported types lead to confusion, limit the usability, and create overhead by touching the same code paths multiple times.</li>
+ <li>For example, when adding a <code>SHIFT_LEFT</code> function, make sure that the contribution is general enough not only for <code>INT</code> but also <code>BIGINT</code> or <code>TINYINT</code>.</li>
+ </ul>
+ </li>
+</ul>
+
+<h4 id="testing">Testing</h4>
+
+<p>Test for nullability.</p>
+
+<ul>
+ <li>SQL natively supports <code>NULL</code> for almost every operation and has a 3-valued boolean logic.</li>
+ <li>Make sure to test every feature for nullability as well.</li>
+</ul>
+
+<p>Avoid full integration tests</p>
+
+<ul>
+ <li>Spawning a Flink mini-cluster and performing compilation of generated code for a SQL query is expensive.</li>
+ <li>Avoid integration tests for planner tests or variations of API calls.</li>
+ <li>Instead, use unit tests that validate the optimized plan which comes out of a planner. Or test the behavior of a runtime operator directly.</li>
+</ul>
+
+<h4 id="compatibility">Compatibility</h4>
+
+<p>Don’t introduce physical plan changes in minor releases!</p>
+
+<ul>
+ <li>Backwards compatibility for state in streaming SQL relies on the fact that the physical execution plan remains stable. Otherwise the generated Operator Names/IDs change and state cannot be matched and restored.</li>
+ <li>Every bug fix that leads to changes in the optimized physical plan of a streaming pipeline hences breaks compatibility.</li>
+ <li>As a consequence, changes of the kind that lead to different optimizer plans can only be merged in major releases for now.</li>
+</ul>
+
+<h4 id="scala--java-interoperability-legacy-code-parts">Scala / Java interoperability (legacy code parts)</h4>
+
+<p>Keep Java in mind when designing interfaces.</p>
+
+<ul>
+ <li>Consider whether a class will need to interact with a Java class in the future.</li>
+ <li>Use Java collections and Java Optional in interfaces for a smooth integration with Java code.</li>
+ <li>Don’t use features of case classes such as .copy() or apply() for construction if a class is subjected to be converted to Java.</li>
+ <li>Pure Scala user-facing APIs should use pure Scala collections/iterables/etc. for natural and idiomatic (“scalaesk”) integration with Scala.</li>
+</ul>
+
+
+
+ </div>
+</div>
+
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/contributing/contribute-documentation.html b/content/contributing/code-style-and-quality-formatting.html
similarity index 55%
copy from content/contributing/contribute-documentation.html
copy to content/contributing/code-style-and-quality-formatting.html
index 9552e41..3088bec 100644
--- a/content/contributing/contribute-documentation.html
+++ b/content/contributing/code-style-and-quality-formatting.html
@@ -5,7 +5,7 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
- <title>Apache Flink: Contribute Documentation</title>
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Formatting</title>
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
<link rel="icon" href="/favicon.ico" type="image/x-icon">
@@ -127,9 +127,9 @@
<a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
</li>
<li >
- <a href="/contributing/code-style-and-quality.html">Code Style and Quality Guide</a>
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
</li>
- <li class="active">
+ <li >
<a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
</li>
<li >
@@ -149,7 +149,7 @@
<li>
- <a href="/zh/contributing/contribute-documentation.html">中文版</a>
+ <a href="/zh/contributing/code-style-and-quality-formatting.html">中文版</a>
</li>
@@ -173,83 +173,153 @@
<div class="col-sm-9">
<div class="row-fluid">
<div class="col-sm-12">
- <h1>Contribute Documentation</h1>
+ <h1>Apache Flink Code Style and Quality Guide — Formatting</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
- <p>Good documentation is crucial for any kind of software. This is especially true for sophisticated software systems such as distributed data processing engines like Apache Flink. The Apache Flink community aims to provide concise, precise, and complete documentation and welcomes any contribution to improve Apache Flink’s documentation.</p>
+<hr />
<div class="page-toc">
<ul id="markdown-toc">
- <li><a href="#obtain-the-documentation-sources" id="markdown-toc-obtain-the-documentation-sources">Obtain the documentation sources</a></li>
- <li><a href="#before-you-start-working-on-the-documentation-" id="markdown-toc-before-you-start-working-on-the-documentation-">Before you start working on the documentation …</a></li>
- <li><a href="#update-or-extend-the-documentation" id="markdown-toc-update-or-extend-the-documentation">Update or extend the documentation</a></li>
- <li><a href="#chinese-documentation-translation" id="markdown-toc-chinese-documentation-translation">Chinese documentation translation</a></li>
- <li><a href="#submit-your-contribution" id="markdown-toc-submit-your-contribution">Submit your contribution</a></li>
+ <li><a href="#java-code-formatting-style" id="markdown-toc-java-code-formatting-style">Java Code Formatting Style</a> <ul>
+ <li><a href="#license" id="markdown-toc-license">License</a></li>
+ <li><a href="#imports" id="markdown-toc-imports">Imports</a></li>
+ <li><a href="#naming" id="markdown-toc-naming">Naming</a></li>
+ <li><a href="#whitespaces" id="markdown-toc-whitespaces">Whitespaces</a></li>
+ <li><a href="#braces" id="markdown-toc-braces">Braces</a></li>
+ <li><a href="#javadocs" id="markdown-toc-javadocs">Javadocs</a></li>
+ <li><a href="#modifiers" id="markdown-toc-modifiers">Modifiers</a></li>
+ <li><a href="#files" id="markdown-toc-files">Files</a></li>
+ <li><a href="#misc" id="markdown-toc-misc">Misc</a></li>
+ </ul>
+ </li>
</ul>
</div>
-<h2 id="obtain-the-documentation-sources">Obtain the documentation sources</h2>
-
-<p>Apache Flink’s documentation is maintained in the same <a href="http://git-scm.com/">git</a> repository as the code base. This is done to ensure that code and documentation can be easily kept in sync.</p>
-
-<p>The easiest way to contribute documentation is to fork <a href="https://github.com/apache/flink">Flink’s mirrored repository on GitHub</a> into your own GitHub account by clicking on the fork button at the top right. If you have no GitHub account, you can create one for free.</p>
+<h2 id="java-code-formatting-style">Java Code Formatting Style</h2>
-<p>Next, clone your fork to your local machine.</p>
+<p>We recommend to set up the IDE to automatically check the code style. Please follow the <a href="https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java">IDE setup guide</a> for that.</p>
-<div class="highlight"><pre><code>git clone https://github.com/<your-user-name>/flink.git
-</code></pre></div>
+<h3 id="license">License</h3>
-<p>The documentation is located in the <code>docs/</code> subdirectory of the Flink code base.</p>
-
-<h2 id="before-you-start-working-on-the-documentation-">Before you start working on the documentation …</h2>
+<ul>
+ <li><strong>Apache license headers.</strong> Make sure you have Apache License headers in your files. The RAT plugin is checking for that when you build the code.</li>
+</ul>
-<p>… please make sure there exists a <a href="https://issues.apache.org/jira/browse/FLINK">Jira</a> issue that corresponds to your contribution. We require all documentation changes to refer to a Jira issue, except for trivial fixes such as typos.</p>
+<h3 id="imports">Imports</h3>
+
+<ul>
+ <li><strong>Empty line before and after package declaration.</strong></li>
+ <li><strong>No unused imports.</strong></li>
+ <li><strong>No redundant imports.</strong></li>
+ <li><strong>No wildcard imports.</strong> They can cause problems when adding to the code and in some cases even during refactoring.</li>
+ <li><strong>Import order.</strong> Imports must be ordered alphabetically, grouped into the following blocks, with each block separated by an empty line:
+ <ul>
+ <li><imports from org.apache.flink.*></li>
+ <li><imports from org.apache.flink.shaded.*></li>
+ <li><imports from other libraries></li>
+ <li><imports from javax.*></li>
+ <li><imports from java.*></li>
+ <li><imports from scala.*></li>
+ <li><static imports></li>
+ </ul>
+ </li>
+</ul>
-<h2 id="update-or-extend-the-documentation">Update or extend the documentation</h2>
+<h3 id="naming">Naming</h3>
-<p>The Flink documentation is written in <a href="http://daringfireball.net/projects/markdown/">Markdown</a>. Markdown is a lightweight markup language which can be translated to HTML.</p>
+<ul>
+ <li><strong>Package names must start with a letter, and must not contain upper-case letters or special characters.</strong>
+ <strong>Non-private static final fields must be upper-case, with words being separated by underscores.</strong>(<code>MY_STATIC_VARIABLE</code>)</li>
+ <li><strong>Non-static fields/methods must be in lower camel case.</strong> (<code>myNonStaticField</code>)</li>
+</ul>
-<p>In order to update or extend the documentation you have to modify the Markdown (<code>.md</code>) files. Please verify your changes by starting the build script in preview mode.</p>
+<h3 id="whitespaces">Whitespaces</h3>
-<div class="highlight"><pre><code>cd docs
-./build_docs.sh -p
-</code></pre></div>
+<ul>
+ <li><strong>Tabs vs. spaces.</strong> We are using tabs for indentation, not spaces.
+We are aware that spaces are a bit nicer; it just happened to be that we started with tabs a long time ago (because Eclipse’s default style used tabs then), and we tried to keep the code base homogeneous (not mix tabs and spaces).</li>
+ <li><strong>No trailing whitespace.</strong></li>
+ <li><strong>Spaces around operators/keywords.</strong> Operators (<code>+</code>, <code>=</code>, <code>></code>, …) and keywords (<code>if</code>, <code>for</code>, <code>catch</code>, …) must have a space before and after them, provided they are not at the start or end of the line.</li>
+</ul>
-<p>The script compiles the Markdown files into static HTML pages and starts a local webserver. Open your browser at <code>http://localhost:4000</code> to view the compiled documentation including your changes. The served documentation is automatically re-compiled and updated when you modify and save Markdown files and refresh your browser.</p>
+<h3 id="braces">Braces</h3>
-<p>Please feel free to ask any questions you have on the developer mailing list.</p>
+<ul>
+ <li><strong>Left curly braces (<code>{</code>) must not be placed on a new line.</strong></li>
+ <li><strong>Right curly braces (<code>}</code>) must always be placed at the beginning of the line.</strong></li>
+ <li><strong>Blocks.</strong> All statements after <code>if</code>, <code>for</code>, <code>while</code>, <code>do</code>, … must always be encapsulated in a block with curly braces (even if the block contains one statement).</li>
+</ul>
-<h2 id="chinese-documentation-translation">Chinese documentation translation</h2>
+<h3 id="javadocs">Javadocs</h3>
-<p>The Flink community is maintaining both English and Chinese documentation. If you want to update or extend the documentation, both English and Chinese documentation should be updated. If you are not familiar with Chinese language, please open a JIRA tagged with the <code>chinese-translation</code> component for Chinese documentation translation and link it with the current JIRA issue. If you are familiar with Chinese language, you are encouraged to update both sides in one pull request.</p>
+<ul>
+ <li><strong>All public/protected methods and classes must have a Javadoc.</strong></li>
+ <li><strong>The first sentence of the Javadoc must end with a period.</strong></li>
+ <li><strong>Paragraphs must be separated with a new line, and started with <p>.</strong></li>
+</ul>
-<p><em>NOTE: The Flink community is still in the process of translating Chinese documentations, some documents may not have been translated yet. If the document you are updating is not translated yet, you can just copy the English changes to the Chinese document.</em></p>
+<h3 id="modifiers">Modifiers</h3>
-<p>The Chinese documents are located in the same folders as the corresponding English documents. The files for both languages have the same names except that the Chinese version have the <code>.zh.md</code> suffix. You can update or extend the <code>.zh.md</code> file according to the English documents changes. If the corresponding <code>.zh.md</code> file doesn’t exited, just copy the existed English file and rename to <code>.zh.md</code> suffix. It will generate under <code>/zh</code> [...]
+<ul>
+ <li><strong>No redundant modifiers.</strong> For example, public modifiers in interface methods.</li>
+ <li><strong>Follow JLS3 modifier order.</strong> Modifiers must be ordered in the following order: public, protected, private, abstract, static, final, transient, volatile, synchronized, native, strictfp.</li>
+</ul>
-<h2 id="submit-your-contribution">Submit your contribution</h2>
+<h3 id="files">Files</h3>
-<p>The Flink project accepts documentation contributions through the <a href="https://github.com/apache/flink">GitHub Mirror</a> as <a href="https://help.github.com/articles/using-pull-requests">Pull Requests</a>. Pull requests are a simple way of offering a patch by providing a pointer to a code branch that contains the changes.</p>
+<ul>
+ <li><strong>All files must end with <code>\n</code>.</strong></li>
+ <li><strong>File length must not exceed 3000 lines.</strong></li>
+</ul>
-<p>To prepare and submit a pull request follow these steps.</p>
+<h3 id="misc">Misc</h3>
-<ol>
- <li>
- <p>Commit your changes to your local git repository. The commit message should point to the corresponding Jira issue by starting with <code>[FLINK-XXXX]</code>.</p>
- </li>
- <li>
- <p>Push your committed contribution to your fork of the Flink repository at GitHub.</p>
+<ul>
+ <li><strong>Arrays must be defined Java-style.</strong> For example, <code>public String[] array</code>.</li>
+ <li><strong>Use Flink Preconditions.</strong> To increase homogeneity, consistently use the <code>org.apache.flink.Preconditions</code> methods <code>checkNotNull</code> and <code>checkArgument</code> rather than Apache Commons Validate or Google Guava.</li>
+</ul>
- <p><code>
- git push origin myBranch
-</code></p>
- </li>
- <li>
- <p>Go to the website of your repository fork (<code>https://github.com/<your-user-name>/flink</code>) and use the “Create Pull Request” button to start creating a pull request. Make sure that the base fork is <code>apache/flink master</code> and the head fork selects the branch with your changes. Give the pull request a meaningful description and submit it.</p>
- </li>
-</ol>
+<hr />
-<p>It is also possible to attach a patch to a <a href="https://issues.apache.org/jira/browse/FLINK">Jira</a> issue.</p>
</div>
diff --git a/content/contributing/code-style-and-quality-java.html b/content/contributing/code-style-and-quality-java.html
new file mode 100644
index 0000000..20f1dc9
--- /dev/null
+++ b/content/contributing/code-style-and-quality-java.html
@@ -0,0 +1,385 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Java</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/flink-architecture.html">What is Apache Flink?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/usecases.html">Use Cases</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/poweredby.html">Powered By</a></li>
+
+ <!-- FAQ -->
+ <li><a href="/faq.html">FAQ</a></li>
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/downloads.html">Downloads</a></li>
+
+ <!-- Quickstart -->
+ <li>
+ <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8/quickstart/setup_quickstart.html" target="_blank">Tutorials <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8" target="_blank">1.8 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/gettinghelp.html">Getting Help</a></li>
+
+ <!-- Blog -->
+ <li><a href="/blog/"><b>Flink Blog</b></a></li>
+
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/community.html">Community & Project Info</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/roadmap.html">Roadmap</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/contributing/how-to-contribute.html">How to Contribute</a></li>
+
+ <ul class="nav navbar-nav navbar-subnav">
+ <li >
+ <a href="/contributing/contribute-code.html">Contribute Code</a>
+ </li>
+ <li >
+ <a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
+ </li>
+ <li >
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
+ </li>
+ <li >
+ <a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
+ </li>
+ <li >
+ <a href="/contributing/improve-website.html">Contribute to the Website</a>
+ </li>
+ </ul>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+
+ <a href="/zh/contributing/code-style-and-quality-java.html">中文版</a>
+
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <h1>Apache Flink Code Style and Quality Guide — Java</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
+
+<hr />
+
+<div class="page-toc">
+<ul id="markdown-toc">
+ <li><a href="#java-language-features-and-libraries" id="markdown-toc-java-language-features-and-libraries">Java Language Features and Libraries</a> <ul>
+ <li><a href="#preconditions-and-log-statements" id="markdown-toc-preconditions-and-log-statements">Preconditions and Log Statements</a></li>
+ <li><a href="#generics" id="markdown-toc-generics">Generics</a></li>
+ <li><a href="#equals--hashcode" id="markdown-toc-equals--hashcode">equals() / hashCode()</a></li>
+ <li><a href="#java-serialization" id="markdown-toc-java-serialization">Java Serialization</a></li>
+ <li><a href="#java-reflection" id="markdown-toc-java-reflection">Java Reflection</a></li>
+ <li><a href="#collections" id="markdown-toc-collections">Collections</a></li>
+ <li><a href="#lambdas" id="markdown-toc-lambdas">Lambdas</a></li>
+ <li><a href="#java-streams" id="markdown-toc-java-streams">Java Streams</a></li>
+ </ul>
+ </li>
+</ul>
+
+</div>
+
+<h2 id="java-language-features-and-libraries">Java Language Features and Libraries</h2>
+
+<h3 id="preconditions-and-log-statements">Preconditions and Log Statements</h3>
+
+<ul>
+ <li>Never concatenate strings in the parameters
+ <ul>
+ <li><span style="text-decoration:underline;">Don’t:</span> <code>Preconditions.checkState(value <= threshold, "value must be below " + threshold)</code></li>
+ <li><span style="text-decoration:underline;">Don’t:</span> <code>LOG.debug("value is " + value)</code></li>
+ <li><span style="text-decoration:underline;">Do:</span> <code>Preconditions.checkState(value <= threshold, "value must be below %s", threshold)</code></li>
+ <li><span style="text-decoration:underline;">Do:</span> <code>LOG.debug("value is {}", value)</code></li>
+ </ul>
+ </li>
+</ul>
+
+<h3 id="generics">Generics</h3>
+
+<ul>
+ <li><strong>No raw types:</strong> Do not use raw types, unless strictly necessary (sometimes necessary for signature matches, arrays).</li>
+ <li><strong>Suppress warnings for unchecked conversions:</strong> Add annotations to suppress warnings, if they cannot be avoided (such as “unchecked”, or “serial”). Otherwise warnings about generics flood the build and drown relevant warnings.</li>
+</ul>
+
+<h3 id="equals--hashcode">equals() / hashCode()</h3>
+
+<ul>
+ <li><strong>equals() / hashCode() should be added when they are well defined only.</strong></li>
+ <li>They should <strong>not be added to enable a simpler assertion in tests</strong> when they are not well defined. Use hamcrest matchers in that case: <a href="https://github.com/junit-team/junit4/wiki/matchers-and-assertthat">https://github.com/junit-team/junit4/wiki/matchers-and-assertthat</a></li>
+ <li>A common indicator that the methods are not well defined is when they take a subset of the fields into account (other than fields that are purely auxiliary).</li>
+ <li>When the methods take mutable fields into account, you often have a design issue. The <code>equals()</code>/<code>hashCode()</code> methods suggest to use the type as a key, but the signatures suggest it is safe to keep mutating the type.</li>
+</ul>
+
+<h3 id="java-serialization">Java Serialization</h3>
+
+<ul>
+ <li><strong>Do not use Java Serialization for anything !!!</strong></li>
+ <li><strong>Do not use Java Serialization for anything !!! !!!</strong></li>
+ <li><strong>Do not use Java Serialization for anything !!! !!! !!!</strong></li>
+ <li>Internal to Flink, Java serialization is used to transport messages and programs through RPC. This is the only case where we use Java serialization. Because of that, some classes need to be serializable (if they are transported via RPC).</li>
+ <li>
+ <p><strong>Serializable classes must define a Serial Version UID:</strong></p>
+
+ <p><code>private static final long serialVersionUID = 1L;</code></p>
+ </li>
+ <li><strong>The Serial Version UID for new classes should start at 1</strong> and should generally be bumped on every incompatible change to the class according to the Java serialization compatibility definition (i.e: changing the type of a field, or moving the position of a class in the class hierarchy).</li>
+</ul>
+
+<h3 id="java-reflection">Java Reflection</h3>
+
+<p><strong>Avoid using Java’s Reflection API</strong></p>
+
+<ul>
+ <li>Java’s Reflection API can be a very useful tool in certain cases but in all cases it is a hack and one should research for alternatives. The only cases where Flink should use reflection are
+ <ul>
+ <li>Dynamically loading implementations from another module (like webUI, additional serializers, pluggable query processors).</li>
+ <li>Extracting types inside the TypeExtractor class. This is fragile enough and should not be done outside the TypeExtractor class.</li>
+ <li>Some cases of cross-JDK version features, where we need to use reflection because we cannot assume a class/method to be present in all versions.</li>
+ </ul>
+ </li>
+ <li>If you need reflection for accessing methods or fields in tests, it usually indicates some deeper architectural issues, like wrong scoping, bad separation of concerns, or that there is no clean way to provide components / dependencies to the class that is tested</li>
+</ul>
+
+<h3 id="collections">Collections</h3>
+
+<ul>
+ <li><strong>ArrayList and ArrayDeque are almost always superior to LinkedList</strong>, except when frequently insert and deleting in the middle of the list</li>
+ <li><strong>For Maps, avoid patterns that require multiple lookups</strong>
+ <ul>
+ <li><code>contains()</code> before <code>get()</code> → <code>get()</code> and check null</li>
+ <li><code>contains()</code> before <code>put()</code> → <code>putIfAbsent()</code> or <code>computeIfAbsent()</code></li>
+ <li>Iterating over keys, getting values → iterate over <code>entrySet()</code></li>
+ </ul>
+ </li>
+</ul>
+
+<h3 id="lambdas">Lambdas</h3>
+
+<ul>
+ <li>
+ <p>Prefer non-capturing lambdas (lambdas that do not contain references to the outer scope). Capturing lambdas need to create a new object instance for every call. Non-capturing lambdas can use the same instance for each invocation.</p>
+
+ <p><strong>don’t:</strong>
+<code>
+map.computeIfAbsent(key, x -> key.toLowerCase())
+</code></p>
+
+ <p><strong>do:</strong>
+<code>
+map.computeIfAbsent(key, k -> k.toLowerCase());
+</code></p>
+ </li>
+ <li>
+ <p>Consider method references instead of inline lambdas</p>
+
+ <p><strong>don’t</strong>:
+<code>
+map.computeIfAbsent(key, k-> Loader.load(k));
+</code></p>
+
+ <p><strong>do:</strong>
+<code>
+map.computeIfAbsent(key, Loader::load);
+</code></p>
+ </li>
+</ul>
+
+<h3 id="java-streams">Java Streams</h3>
+
+<ul>
+ <li>Avoid Java Streams in any performance critical code.</li>
+ <li>The main motivation to use Java Streams would be to improve code readability. As such, they can be a good match in parts of the code that are not data-intensive, but deal with coordination..</li>
+ <li>Even in the latter case, try to limit the scope to a method, or a few private methods within an internal class.</li>
+</ul>
+
+
+
+ </div>
+</div>
+
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/contributing/code-style-and-quality.html b/content/contributing/code-style-and-quality-preamble.html
similarity index 69%
rename from content/contributing/code-style-and-quality.html
rename to content/contributing/code-style-and-quality-preamble.html
index d41bf40..aa7295a 100644
--- a/content/contributing/code-style-and-quality.html
+++ b/content/contributing/code-style-and-quality-preamble.html
@@ -5,7 +5,7 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
- <title>Apache Flink: Code Style and Quality Guide (PLACEHOLDER)</title>
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Preamble</title>
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
<link rel="icon" href="/favicon.ico" type="image/x-icon">
@@ -127,7 +127,7 @@
<a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
</li>
<li class="active">
- <a href="/contributing/code-style-and-quality.html">Code Style and Quality Guide</a>
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
</li>
<li >
<a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
@@ -149,7 +149,7 @@
<li>
- <a href="/zh/contributing/code-style-and-quality.html">中文版</a>
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">中文版</a>
</li>
@@ -173,13 +173,71 @@
<div class="col-sm-9">
<div class="row-fluid">
<div class="col-sm-12">
- <h1>Code Style and Quality Guide (PLACEHOLDER)</h1>
-
- <p><br />
-<br /></p>
-
-<p>This page is a placeholder for an updated Code Style and Quality Guide to be posted here, once the <a href="https://lists.apache.org/thread.html/461d0922787c62447c3baff3177a0c8f34fee7c95e2896e5ddf59691@%3Cdev.flink.apache.org%3E">discussion on the mailing list</a> has concluded.</p>
-
+ <h1>Apache Flink Code Style and Quality Guide — Preamble</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
+
+<hr />
+
+<p>This is an attempt to capture the code and quality standard that we want to maintain.</p>
+
+<p>A code contribution (or any piece of code) can be evaluated in various ways: One set of properties is whether the code is correct and efficient. This requires solving the <em>logical or algorithmic problem</em> correctly and well.</p>
+
+<p>Another set of properties is whether the code follows an intuitive design and architecture, whether it is well structured with right separation of concerns, and whether the code is easily understandable and makes its assumptions explicit. That set of properties requires solving the <em>software engineering problem</em> well. A good solution implies that the code is easily testable, maintainable also by other people than the original authors (because it is harder to accidentally break) [...]
+
+<p>While the first set of properties has rather objective approval criteria, the second set of properties is much harder to assess, but is of high importance for an open source project like Apache Flink. To make the code base inviting to many contributors, to make contributions easy to understand for developers that did not write the original code, and to make the code robust in the face of many contributions, well engineered code is crucial.<sup id="fnref:1"><a href="#fn:1" class="footn [...]
+
+<p>This is of course not a full guide on how to write well engineered code. There is a world of big books that try to capture that. This guide is meant as a checklist of best practices, patterns, anti-patterns, and common mistakes that we observed in the context of developing Flink.</p>
+
+<p>A big part of high-quality open source contributions is about helping the reviewer to understand the contribution and double-check the implications, so an important part of this guide is about how to structure a pull request for review.</p>
+
+<hr />
+
+<div class="footnotes">
+ <ol>
+ <li id="fn:1">
+
+ <p>In earlier days, we (the Flink community) did not always pay sufficient attention to this, making some components of Flink harder to evolve and to contribute to. <a href="#fnref:1" class="reversefootnote">↩</a></p>
+ </li>
+ </ol>
+</div>
</div>
diff --git a/content/contributing/code-style-and-quality-pull-requests.html b/content/contributing/code-style-and-quality-pull-requests.html
new file mode 100644
index 0000000..d38451c
--- /dev/null
+++ b/content/contributing/code-style-and-quality-pull-requests.html
@@ -0,0 +1,352 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Pull Requests & Changes</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/flink-architecture.html">What is Apache Flink?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/usecases.html">Use Cases</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/poweredby.html">Powered By</a></li>
+
+ <!-- FAQ -->
+ <li><a href="/faq.html">FAQ</a></li>
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/downloads.html">Downloads</a></li>
+
+ <!-- Quickstart -->
+ <li>
+ <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8/quickstart/setup_quickstart.html" target="_blank">Tutorials <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8" target="_blank">1.8 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/gettinghelp.html">Getting Help</a></li>
+
+ <!-- Blog -->
+ <li><a href="/blog/"><b>Flink Blog</b></a></li>
+
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/community.html">Community & Project Info</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/roadmap.html">Roadmap</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/contributing/how-to-contribute.html">How to Contribute</a></li>
+
+ <ul class="nav navbar-nav navbar-subnav">
+ <li >
+ <a href="/contributing/contribute-code.html">Contribute Code</a>
+ </li>
+ <li >
+ <a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
+ </li>
+ <li >
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
+ </li>
+ <li >
+ <a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
+ </li>
+ <li >
+ <a href="/contributing/improve-website.html">Contribute to the Website</a>
+ </li>
+ </ul>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+
+ <a href="/zh/contributing/code-style-and-quality-pull-requests.html">中文版</a>
+
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <h1>Apache Flink Code Style and Quality Guide — Pull Requests & Changes</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
+
+<hr />
+
+<div class="page-toc">
+<ul id="markdown-toc">
+ <li><a href="#jira-issue-and-naming" id="markdown-toc-jira-issue-and-naming">1. JIRA issue and Naming</a></li>
+ <li><a href="#description" id="markdown-toc-description">2. Description</a></li>
+ <li><a href="#separate-refactoring-cleanup-and-independent-changes" id="markdown-toc-separate-refactoring-cleanup-and-independent-changes">3. Separate Refactoring, Cleanup and Independent Changes</a></li>
+ <li><a href="#commit-naming-conventions" id="markdown-toc-commit-naming-conventions">4. Commit Naming Conventions</a></li>
+ <li><a href="#changes-to-the-observable-behavior-of-the-system" id="markdown-toc-changes-to-the-observable-behavior-of-the-system">5. Changes to the observable behavior of the system</a></li>
+</ul>
+
+</div>
+
+<p><strong>Rationale:</strong> We ask contributors to put in a little bit of extra effort to bring pull requests into a state that they can be more easily and more thoroughly reviewed. This helps the community in many ways:</p>
+
+<ul>
+ <li>Reviews are much faster and thus contributions get merged sooner.</li>
+ <li>We can ensure higher code quality by overlooking fewer issues in the contributions.</li>
+ <li>Committers can review more contributions in the same time, which helps to keep up with the high rate of contributions that Flink is experiencing</li>
+</ul>
+
+<p>Please understand that contributions that do not follow this guide will take longer to review and thus will typically be picked up with lower priority by the community. That is not ill intend, it is due to the added complexity of reviewing unstructured Pull Requests.</p>
+
+<h2 id="jira-issue-and-naming">1. JIRA issue and Naming</h2>
+
+<p>Make sure that the pull request corresponds to a <a href="[https://issues.apache.org/jira/projects/FLINK/issues">JIRA issue</a>.</p>
+
+<p>Exceptions are <strong>**hotfixes</strong>**, like fixing typos in JavaDocs or documentation files.</p>
+
+<p>Name the pull request in the form <code>[FLINK-XXXX][component] Title of the pull request</code>, where <code>FLINK-XXXX</code> should be replaced by the actual issue number. The components should be the same as used in the JIRA issue.</p>
+
+<p>Hotfixes should be named for example <code>[hotfix][docs] Fix typo in event time introduction</code> or <code>[hotfix][javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator</code>.</p>
+
+<h2 id="description">2. Description</h2>
+
+<p>Please fill out the pull request template to describe the contribution. Please describe it such that the reviewer understands the problem and solution from the description, not only from the code.</p>
+
+<p>A stellar example of a well-described pull request is <a href="https://github.com/apache/flink/pull/7264">https://github.com/apache/flink/pull/7264</a></p>
+
+<p>Make sure that the description is adequate for the problem solved by PR. Small changes do not need a wall of text. In ideal cases, the problem was described in the Jira issue and the description be mostly copied from there.</p>
+
+<p>If additional open questions / issues were discovered during the implementation and you made a choice regarding those, describe them in the pull request text so that reviewers can double check the assumptions. And example is in <a href="https://github.com/apache/flink/pull/8290">https://github.com/apache/flink/pull/8290</a> (Section “Open Architecture Questions”).</p>
+
+<h2 id="separate-refactoring-cleanup-and-independent-changes">3. Separate Refactoring, Cleanup and Independent Changes</h2>
+
+<p><strong>**NOTE: This is not an optimization, this is a critical requirement.</strong>**</p>
+
+<p>Pull Requests must put cleanup, refactoring, and core changes into separate commits. That way, the reviewer can look independently at the cleanup and refactoring and ensure that those changes to not alter the behavior. Then the reviewer can look at the core changes in isolation (without the noise of other changes) and ensure that this is a clean and robust change.</p>
+
+<p>Examples for changes that strictly need to go into a separate commit include</p>
+
+<ul>
+ <li>Cleanup, fixing style and warnings in pre-existing code</li>
+ <li>Renaming packages, classes, or methods</li>
+ <li>Moving code (to other packages or classes)</li>
+ <li>Refactoring structure or changing design patterns</li>
+ <li>Consolidating related tests or utilities</li>
+ <li>Changing the assumptions in existing tests (add a commit message that describes why the changed assumptions make sense).</li>
+</ul>
+
+<p>There should be no cleanup commits that fix issues that have been introduced in previous commits of the same PR. Commits should be clean in themselves.</p>
+
+<p>In addition, any larger contributions should split the changes into a set of independent changes that can be independently reviewed.</p>
+
+<p>Two great examples of splitting issues into separate commits are:</p>
+
+<ul>
+ <li><a href="https://github.com/apache/flink/pull/6692">https://github.com/apache/flink/pull/6692</a> (splits cleanup and refactoring from main changes)</li>
+ <li><a href="https://github.com/apache/flink/pull/7264">https://github.com/apache/flink/pull/7264</a> (splits also main changes into independently reviewable pieces)</li>
+</ul>
+
+<p>If a pull request does still contain big commits (e.g. a commit with more than 1000 changed lines), it might be worth thinking about how to split the commit into multiple subproblems, as in the example above.</p>
+
+<h2 id="commit-naming-conventions">4. Commit Naming Conventions</h2>
+
+<p>Commit messages should follow a similar pattern as the pull request as a whole:
+<code>[FLINK-XXXX][component] Commit description</code>.</p>
+
+<p>In some cases, the issue might be a subtask here, and the component may be different from the Pull Request’s main component. For example, when the commit introduces an end-to-end test for a runtime change, the PR would be tagged as <code>[runtime]</code>, but the individual commit would be tagged as <code>[e2e]</code>.</p>
+
+<p>Examples for commit messages:</p>
+
+<ul>
+ <li><code>[hotfix] Fix update_branch_version.sh to allow version suffixes</code></li>
+ <li><code>[hotfix] [table] Remove unused geometry dependency</code></li>
+ <li><code>[FLINK-11704][tests] Improve AbstractCheckpointStateOutputStreamTestBase</code></li>
+ <li><code>[FLINK-10569][runtime] Remove Instance usage in ExecutionVertexCancelTest</code></li>
+ <li><code>[FLINK-11702][table-planner-blink] Introduce a new table type system</code></li>
+</ul>
+
+<h2 id="changes-to-the-observable-behavior-of-the-system">5. Changes to the observable behavior of the system</h2>
+
+<p>Contributors should be aware of changes in their PRs that break the observable behavior of Flink in any way because in many cases such changes can break existing setups. Red flags that should raise questions while coding or in reviews with respect to this problem are for example:</p>
+
+<ul>
+ <li>Assertions have been changed to make tests pass again with the breaking change.</li>
+ <li>Configuration setting that must suddenly be set to (non-default) values to keep existing tests passing. This can happen in particular for new settings with a breaking default.</li>
+ <li>Existing scripts or configurations have to be adjusted.</li>
+</ul>
+
+
+ </div>
+</div>
+
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/contributing/contribute-documentation.html b/content/contributing/code-style-and-quality-scala.html
similarity index 56%
copy from content/contributing/contribute-documentation.html
copy to content/contributing/code-style-and-quality-scala.html
index 9552e41..94c5b70 100644
--- a/content/contributing/contribute-documentation.html
+++ b/content/contributing/code-style-and-quality-scala.html
@@ -5,7 +5,7 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
- <title>Apache Flink: Contribute Documentation</title>
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Scala</title>
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
<link rel="icon" href="/favicon.ico" type="image/x-icon">
@@ -127,9 +127,9 @@
<a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
</li>
<li >
- <a href="/contributing/code-style-and-quality.html">Code Style and Quality Guide</a>
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
</li>
- <li class="active">
+ <li >
<a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
</li>
<li >
@@ -149,7 +149,7 @@
<li>
- <a href="/zh/contributing/contribute-documentation.html">中文版</a>
+ <a href="/zh/contributing/code-style-and-quality-scala.html">中文版</a>
</li>
@@ -173,83 +173,147 @@
<div class="col-sm-9">
<div class="row-fluid">
<div class="col-sm-12">
- <h1>Contribute Documentation</h1>
+ <h1>Apache Flink Code Style and Quality Guide — Scala</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
- <p>Good documentation is crucial for any kind of software. This is especially true for sophisticated software systems such as distributed data processing engines like Apache Flink. The Apache Flink community aims to provide concise, precise, and complete documentation and welcomes any contribution to improve Apache Flink’s documentation.</p>
+<hr />
<div class="page-toc">
<ul id="markdown-toc">
- <li><a href="#obtain-the-documentation-sources" id="markdown-toc-obtain-the-documentation-sources">Obtain the documentation sources</a></li>
- <li><a href="#before-you-start-working-on-the-documentation-" id="markdown-toc-before-you-start-working-on-the-documentation-">Before you start working on the documentation …</a></li>
- <li><a href="#update-or-extend-the-documentation" id="markdown-toc-update-or-extend-the-documentation">Update or extend the documentation</a></li>
- <li><a href="#chinese-documentation-translation" id="markdown-toc-chinese-documentation-translation">Chinese documentation translation</a></li>
- <li><a href="#submit-your-contribution" id="markdown-toc-submit-your-contribution">Submit your contribution</a></li>
+ <li><a href="#scala-language-features" id="markdown-toc-scala-language-features">Scala Language Features</a> <ul>
+ <li><a href="#where-to-use-and-not-use-scala" id="markdown-toc-where-to-use-and-not-use-scala">Where to use (and not use) Scala</a></li>
+ <li><a href="#api-parity" id="markdown-toc-api-parity">API Parity</a></li>
+ <li><a href="#language-features" id="markdown-toc-language-features">Language Features</a></li>
+ <li><a href="#coding-formatting" id="markdown-toc-coding-formatting">Coding Formatting</a></li>
+ </ul>
+ </li>
</ul>
</div>
-<h2 id="obtain-the-documentation-sources">Obtain the documentation sources</h2>
-
-<p>Apache Flink’s documentation is maintained in the same <a href="http://git-scm.com/">git</a> repository as the code base. This is done to ensure that code and documentation can be easily kept in sync.</p>
-
-<p>The easiest way to contribute documentation is to fork <a href="https://github.com/apache/flink">Flink’s mirrored repository on GitHub</a> into your own GitHub account by clicking on the fork button at the top right. If you have no GitHub account, you can create one for free.</p>
-
-<p>Next, clone your fork to your local machine.</p>
-
-<div class="highlight"><pre><code>git clone https://github.com/<your-user-name>/flink.git
-</code></pre></div>
-
-<p>The documentation is located in the <code>docs/</code> subdirectory of the Flink code base.</p>
-
-<h2 id="before-you-start-working-on-the-documentation-">Before you start working on the documentation …</h2>
+<h2 id="scala-language-features">Scala Language Features</h2>
-<p>… please make sure there exists a <a href="https://issues.apache.org/jira/browse/FLINK">Jira</a> issue that corresponds to your contribution. We require all documentation changes to refer to a Jira issue, except for trivial fixes such as typos.</p>
+<h3 id="where-to-use-and-not-use-scala">Where to use (and not use) Scala</h3>
-<h2 id="update-or-extend-the-documentation">Update or extend the documentation</h2>
+<p><strong>We use Scala for Scala APIs or pure Scala Libraries.</strong></p>
-<p>The Flink documentation is written in <a href="http://daringfireball.net/projects/markdown/">Markdown</a>. Markdown is a lightweight markup language which can be translated to HTML.</p>
+<p><strong>We do not use Scala in the core APIs and runtime components. We aim to remove existing Scala use (code and dependencies) from those components.</strong></p>
-<p>In order to update or extend the documentation you have to modify the Markdown (<code>.md</code>) files. Please verify your changes by starting the build script in preview mode.</p>
+<p>⇒ This is not because we do not like Scala, it is a consequence of “the right tool for the right job” approach (see below).</p>
-<div class="highlight"><pre><code>cd docs
-./build_docs.sh -p
-</code></pre></div>
+<p>For APIs, we develop the foundation in Java, and layer Scala on top.</p>
-<p>The script compiles the Markdown files into static HTML pages and starts a local webserver. Open your browser at <code>http://localhost:4000</code> to view the compiled documentation including your changes. The served documentation is automatically re-compiled and updated when you modify and save Markdown files and refresh your browser.</p>
-
-<p>Please feel free to ask any questions you have on the developer mailing list.</p>
+<ul>
+ <li>This has traditionally given the best interoperability for both Java and Scala</li>
+ <li>It does mean dedicated effort to keep the Scala API up to date</li>
+</ul>
-<h2 id="chinese-documentation-translation">Chinese documentation translation</h2>
+<p>Why don’t we use Scala in the core APIs and runtime?</p>
-<p>The Flink community is maintaining both English and Chinese documentation. If you want to update or extend the documentation, both English and Chinese documentation should be updated. If you are not familiar with Chinese language, please open a JIRA tagged with the <code>chinese-translation</code> component for Chinese documentation translation and link it with the current JIRA issue. If you are familiar with Chinese language, you are encouraged to update both sides in one pull request.</p>
+<ul>
+ <li>The past has shown that Scala evolves too quickly with tricky changes in functionality. Each Scala version upgrade was a rather big effort process for the Flink community.</li>
+ <li>Scala does not always interact nicely with Java classes, e.g. Scala’s visibility scopes work differently and often expose more to Java consumers than desired</li>
+ <li>Scala adds an additional layer of complexity to artifact/dependency management.
+ <ul>
+ <li>We may want to keep Scala dependent libraries like Akka in the runtime, but abstract them via an interface and load them in a separate classloader, to keep them shielded and avoid version conflicts.</li>
+ </ul>
+ </li>
+ <li>Scala makes it very easy for knowledgeable Scala programmers to write code that is very hard to understand for programmers that are less knowledgeable in Scala. That is especially tricky for an open source project with a broad community of diverse experience levels. Working around this means restricting the Scala feature set by a lot, which defeats a good amount of the purpose of using Scala in the first place.</li>
+</ul>
-<p><em>NOTE: The Flink community is still in the process of translating Chinese documentations, some documents may not have been translated yet. If the document you are updating is not translated yet, you can just copy the English changes to the Chinese document.</em></p>
+<h3 id="api-parity">API Parity</h3>
-<p>The Chinese documents are located in the same folders as the corresponding English documents. The files for both languages have the same names except that the Chinese version have the <code>.zh.md</code> suffix. You can update or extend the <code>.zh.md</code> file according to the English documents changes. If the corresponding <code>.zh.md</code> file doesn’t exited, just copy the existed English file and rename to <code>.zh.md</code> suffix. It will generate under <code>/zh</code> [...]
+<p>Keep Java API and Scala API in sync in terms of functionality and code quality.</p>
-<h2 id="submit-your-contribution">Submit your contribution</h2>
+<p>The Scala API should cover all the features of the Java APIs as well.</p>
-<p>The Flink project accepts documentation contributions through the <a href="https://github.com/apache/flink">GitHub Mirror</a> as <a href="https://help.github.com/articles/using-pull-requests">Pull Requests</a>. Pull requests are a simple way of offering a patch by providing a pointer to a code branch that contains the changes.</p>
+<p>Scala APIs should have a “completeness test”, like the following example from the DataStream API: <a href="https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala">https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala</a></p>
-<p>To prepare and submit a pull request follow these steps.</p>
+<h3 id="language-features">Language Features</h3>
-<ol>
- <li>
- <p>Commit your changes to your local git repository. The commit message should point to the corresponding Jira issue by starting with <code>[FLINK-XXXX]</code>.</p>
+<ul>
+ <li><strong>Avoid Scala implicits.</strong>
+ <ul>
+ <li>Scala’s implicits should only be used for user-facing API improvements such as the Table API expressions or type information extraction.</li>
+ <li>Don’t use them for internal “magic”.</li>
+ </ul>
</li>
- <li>
- <p>Push your committed contribution to your fork of the Flink repository at GitHub.</p>
-
- <p><code>
- git push origin myBranch
-</code></p>
+ <li><strong>Add explicit types for class members.</strong>
+ <ul>
+ <li>
+ <p>Don’t rely on implicit type inference for class fields and methods return types:</p>
+
+ <p><strong>Don’t:</strong>
+ <code>
+ var expressions = new java.util.ArrayList[String]()
+ </code></p>
+
+ <p><strong>Do:</strong>
+ <code>
+ var expressions: java.util.List[String] = new java.util.ArrayList[]()
+ </code></p>
+ </li>
+ <li>
+ <p>Type inference for local variables on the stack is fine.</p>
+ </li>
+ </ul>
</li>
- <li>
- <p>Go to the website of your repository fork (<code>https://github.com/<your-user-name>/flink</code>) and use the “Create Pull Request” button to start creating a pull request. Make sure that the base fork is <code>apache/flink master</code> and the head fork selects the branch with your changes. Give the pull request a meaningful description and submit it.</p>
+ <li><strong>Use strict visibility.</strong>
+ <ul>
+ <li>Avoid Scala’s package private features (such as private[flink]) and use regular private/protected instead.</li>
+ <li>Keep in mind that <code>private[flink]</code> and <code>protected</code> members are public in Java.</li>
+ <li>Keep in mind that <code>private[flink]</code> still exposes all members in Flink provided examples.</li>
+ </ul>
</li>
-</ol>
+</ul>
+
+<h3 id="coding-formatting">Coding Formatting</h3>
-<p>It is also possible to attach a patch to a <a href="https://issues.apache.org/jira/browse/FLINK">Jira</a> issue.</p>
+<p><strong>Use line wrapping to structure your code.</strong></p>
+
+<ul>
+ <li>Scala’s functional nature allows for long transformation chains (<code>x.map().map().foreach()</code>).</li>
+ <li>In order to force implementers to structure their code, the line length is therefore limited to 100 characters.</li>
+ <li>Use one line per transformation for better maintainability.</li>
+</ul>
</div>
diff --git a/content/contributing/contribute-code.html b/content/contributing/contribute-code.html
index f1212db..2dd548f 100644
--- a/content/contributing/contribute-code.html
+++ b/content/contributing/contribute-code.html
@@ -127,7 +127,7 @@
<a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
</li>
<li >
- <a href="/contributing/code-style-and-quality.html">Code Style and Quality Guide</a>
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
</li>
<li >
<a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
@@ -278,7 +278,7 @@
<div class="panel panel-default">
<div class="panel-body">
<h2><span class="number">2</span><a href="#implement">Implement</a></h2>
- <p>Implement the change according to the <a href="/contributing/code-style-and-quality.html">Code Style and Quality Guide</a> and the approach agreed upon in the Jira ticket.</p> <br />
+ <p>Implement the change according to the <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a> and the approach agreed upon in the Jira ticket.</p> <br />
<p><b>Only start working on the implementation if there is consensus on the approach (e.g. you are assigned to the ticket)</b></p>
</div>
</div>
@@ -383,7 +383,7 @@ Only committers have the permission to assign somebody.</p>
<ul>
<li><a href="https://cwiki.apache.org/confluence/display/FLINK/Setting+up+a+Flink+development+environment">Set up a Flink development environment</a></li>
- <li>Follow the <a href="/contributing/code-style-and-quality.html">Code Style and Quality Guide</a> of Flink</li>
+ <li>Follow the <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a> of Flink</li>
<li>Take any discussions and requirements from the Jira issue or design document into account.</li>
<li>Do not mix unrelated issues into one contribution.</li>
</ul>
diff --git a/content/contributing/contribute-documentation.html b/content/contributing/contribute-documentation.html
index 9552e41..03ecdc1 100644
--- a/content/contributing/contribute-documentation.html
+++ b/content/contributing/contribute-documentation.html
@@ -127,7 +127,7 @@
<a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
</li>
<li >
- <a href="/contributing/code-style-and-quality.html">Code Style and Quality Guide</a>
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
</li>
<li class="active">
<a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
diff --git a/content/contributing/how-to-contribute.html b/content/contributing/how-to-contribute.html
index 1054ce0..4b9aa92 100644
--- a/content/contributing/how-to-contribute.html
+++ b/content/contributing/how-to-contribute.html
@@ -127,7 +127,7 @@
<a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
</li>
<li >
- <a href="/contributing/code-style-and-quality.html">Code Style and Quality Guide</a>
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
</li>
<li >
<a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
diff --git a/content/contributing/improve-website.html b/content/contributing/improve-website.html
index 43546ec..c940712 100644
--- a/content/contributing/improve-website.html
+++ b/content/contributing/improve-website.html
@@ -127,7 +127,7 @@
<a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
</li>
<li >
- <a href="/contributing/code-style-and-quality.html">Code Style and Quality Guide</a>
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
</li>
<li >
<a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
diff --git a/content/contributing/reviewing-prs.html b/content/contributing/reviewing-prs.html
index cf3a0e1..57b1c8b 100644
--- a/content/contributing/reviewing-prs.html
+++ b/content/contributing/reviewing-prs.html
@@ -127,7 +127,7 @@
<a href="/contributing/reviewing-prs.html">Review Pull Requests</a>
</li>
<li >
- <a href="/contributing/code-style-and-quality.html">Code Style and Quality Guide</a>
+ <a href="/contributing/code-style-and-quality-preamble.html">Code Style and Quality Guide</a>
</li>
<li >
<a href="/contributing/contribute-documentation.html">Contribute Documentation</a>
@@ -270,7 +270,7 @@ If the implementation is different from the agreed approach in the consensus dis
<li>If dependencies have been changed, were the NOTICE files updated?</li>
</ul>
-<p>Code guidelines can be found in the <a href="/contributing/code-style-and-quality.html">Flink Code Style and Quality Guide</a>.</p>
+<p>Code guidelines can be found in the <a href="/contributing/code-style-and-quality-preamble.html">Flink Code Style and Quality Guide</a>.</p>
<hr />
diff --git a/content/zh/contributing/code-style-and-quality-common.html b/content/zh/contributing/code-style-and-quality-common.html
new file mode 100644
index 0000000..fe010bb
--- /dev/null
+++ b/content/zh/contributing/code-style-and-quality-common.html
@@ -0,0 +1,618 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Common Rules</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/zh/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/zh/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/zh/flink-architecture.html">Apache Flink 是什么?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/zh/usecases.html">应用场景</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/zh/poweredby.html">Flink 用户</a></li>
+
+ <!-- FAQ -->
+ <li><a href="/zh/faq.html">常见问题</a></li>
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/zh/downloads.html">下载</a></li>
+
+ <!-- Quickstart -->
+ <li>
+ <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8/quickstart/setup_quickstart.html" target="_blank">教程 <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">文档<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8" target="_blank">1.8 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/zh/gettinghelp.html">获取帮助</a></li>
+
+ <!-- Blog -->
+ <li><a href="/blog/"><b>Flink 博客</b></a></li>
+
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/zh/community.html">社区 & 项目信息</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/zh/roadmap.html">开发计划</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/zh/contributing/how-to-contribute.html">如何参与贡献</a></li>
+
+ <ul class="nav navbar-nav navbar-subnav">
+ <li >
+ <a href="/zh/contributing/contribute-code.html">贡献代码</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/contribute-documentation.html">贡献文档</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/improve-website.html">贡献网站</a>
+ </li>
+ </ul>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+ <a href="/contributing/code-style-and-quality-common.html">English</a>
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <h1>Apache Flink Code Style and Quality Guide — Common Rules</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
+
+<hr />
+
+<div class="page-toc">
+<ul id="markdown-toc">
+ <li><a href="#tools" id="markdown-toc-tools">1. Tools</a> <ul>
+ <li><a href="#warnings" id="markdown-toc-warnings">Warnings</a></li>
+ </ul>
+ </li>
+ <li><a href="#comments-and-code-readability" id="markdown-toc-comments-and-code-readability">2. Comments And Code Readability</a> <ul>
+ <li><a href="#comments" id="markdown-toc-comments">Comments</a></li>
+ <li><a href="#branches-and-nesting" id="markdown-toc-branches-and-nesting">Branches and Nesting</a></li>
+ </ul>
+ </li>
+ <li><a href="#design-and-structure" id="markdown-toc-design-and-structure">3. Design and Structure</a> <ul>
+ <li><a href="#immutability-and-eager-initialization" id="markdown-toc-immutability-and-eager-initialization">Immutability and Eager Initialization</a></li>
+ <li><a href="#nullability-of-the-mutable-parts" id="markdown-toc-nullability-of-the-mutable-parts">Nullability of the Mutable Parts</a></li>
+ <li><a href="#avoid-code-duplication" id="markdown-toc-avoid-code-duplication">Avoid Code Duplication</a></li>
+ <li><a href="#design-for-testability" id="markdown-toc-design-for-testability">Design for Testability</a></li>
+ <li><a href="#performance-awareness" id="markdown-toc-performance-awareness">Performance Awareness</a></li>
+ </ul>
+ </li>
+ <li><a href="#concurrency-and-threading" id="markdown-toc-concurrency-and-threading">4. Concurrency and Threading</a></li>
+ <li><a href="#dependencies-and-modules" id="markdown-toc-dependencies-and-modules">5. Dependencies and Modules</a></li>
+</ul>
+
+</div>
+
+<h2 id="tools">1. Tools</h2>
+
+<p>We recommend to follow the <a href="https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java">IDE Setup Guide</a> to get IDE tooling configured.</p>
+
+<!---
+### Use inspections in IntelliJ
+
+* Import the inspections settings into the IDE (see IDE setup guide)
+ * TODO: Need to agree on a profile and export it (like checkstyle)
+* Write the code such that inspection warnings are addressed
+ * There are few exceptions where an inspection warning is not meaningful. In that case, suppress the inspection warning.
+-->
+
+<h3 id="warnings">Warnings</h3>
+
+<ul>
+ <li>We strive for zero warnings</li>
+ <li>Even though there are many warnings in existing code, new changes should not add any additional compiler warnings</li>
+ <li>If it is not possible to address the warning in a sane way (in some cases when working with generics) add an annotation to suppress the warning</li>
+ <li>When deprecating methods, check that this does not introduce additional warnings</li>
+</ul>
+
+<h2 id="comments-and-code-readability">2. Comments And Code Readability</h2>
+
+<h3 id="comments">Comments</h3>
+
+<p><strong>Golden rule: Comment as much as necessary to support code understanding, but don’t add redundant information.</strong></p>
+
+<p>Think about</p>
+
+<ul>
+ <li><span style="text-decoration:underline;">What</span> is the code doing?</li>
+ <li><span style="text-decoration:underline;">How</span> does the code do this?</li>
+ <li><span style="text-decoration:underline;">Why</span> is the code like that?</li>
+</ul>
+
+<p>The code alone should explain as much as possible the “<span style="text-decoration:underline;">what</span>” and the “<span style="text-decoration:underline;">how</span>”</p>
+
+<ul>
+ <li>Use JavaDocs to describe the roles of classes and the contracts of methods, in cases where the contract is not obvious or intuitive from the method name (the “what”).</li>
+ <li>The flow of the code should give a good description of the “how”.
+Think of variable and method names as part of the code documenting itself.</li>
+ <li>It often makes reading the code easier if larger blocks that form a unit are moved into a private method with a descriptive name of what that block is doing</li>
+</ul>
+
+<p>In-code comments help explain the <span style="text-decoration:underline;">“why”</span></p>
+
+<ul>
+ <li>For example <code>// this specific code layout helps the JIT to better do this or that</code></li>
+ <li>Or <code>// nulling out this field here means future write attempts are fail-fast</code></li>
+ <li>Or <code>// for arguments with which this method is actually called, this seemingly naive approach works actually better than any optimized/smart version</code></li>
+</ul>
+
+<p>In-code comments should not state redundant information about the “what” and “how” that is already obvious in the code itself.</p>
+
+<p>JavaDocs should not state meaningless information (just to satisfy the Checkstyle checker).</p>
+
+<p><strong>Don’t:</strong></p>
+
+<div class="highlight"><pre><code>/**
+ * The symbol expression.
+ */
+public class CommonSymbolExpression {}
+</code></pre></div>
+<p><strong>Do:</strong></p>
+
+<div class="highlight"><pre><code>/**
+ * An expression that wraps a single specific symbol.
+ * A symbol could be a unit, an alias, a variable, etc.
+ */
+public class CommonSymbolExpression {}
+</code></pre></div>
+
+<h3 id="branches-and-nesting">Branches and Nesting</h3>
+
+<p>Avoid deep nesting of scopes, by flipping the if condition and exiting early.</p>
+
+<p><strong>Don’t:</strong></p>
+
+<div class="highlight"><pre><code>if (a) {
+ if (b) {
+ if (c) {
+ the main path
+ }
+ }
+}
+</code></pre></div>
+
+<p><strong>Do</strong></p>
+
+<div class="highlight"><pre><code>if (!a) {
+ return ..
+}
+
+if (!b) {
+ return ...
+}
+
+if (!c) {
+ return ...
+}
+
+the main path
+</code></pre></div>
+
+<h2 id="design-and-structure">3. Design and Structure</h2>
+
+<p>While it is hard to exactly specify what constitutes a good design, there are some properties that can serve as a <em>litmus test</em> for a good design. If these properties are given, the chances are good that the design is going into a good direction. If these properties cannot be achieved, there is a high probability that the design is flawed.</p>
+
+<h3 id="immutability-and-eager-initialization">Immutability and Eager Initialization</h3>
+
+<ol>
+ <li>Try to use immutable types where possible, especially for APIs, messages, identifiers, properties, configuration, etc.</li>
+ <li>A good general approach is to try and make as many fields of a class <code>final</code> as possible.</li>
+ <li>Classes that are used as keys in maps should be strictly immutable and only have <code>final</code> fields (except maybe auxiliary fields, like lazy cached hash codes).</li>
+ <li>Eagerly initialize classes. There should be no <code>init()</code> or <code>setup()</code> methods. Once the constructor completes, the object should be usable.</li>
+</ol>
+
+<h3 id="nullability-of-the-mutable-parts">Nullability of the Mutable Parts</h3>
+
+<p>For nullability, the Flink codebase aims to follow these conventions:</p>
+
+<ul>
+ <li>Fields, parameters, and return types are always non-null, unless indicated otherwise</li>
+ <li>All fields, parameters and method types that can be null should be annotated with <code>@javax.annotation.Nullable</code>.
+That way, you get warnings from IntelliJ about all sections where you have to reason about potential null values.</li>
+ <li>For all mutable (non-final) fields that are not annotated, the assumption is that while the field value changes, there always is a value.
+ <ul>
+ <li>This should be double check whether these can in fact not be null throughout the lifetime of the object.</li>
+ </ul>
+ </li>
+</ul>
+
+<p><em>Note: This means that <code>@Nonnull</code> annotations are usually not necessary, but can be used in certain cases to override a previous annotation, or to point non-nullability out in a context where one would expect a nullable value.</em></p>
+
+<p><code>Optional</code> is a good solution as a return type for method that may or may not have a result, so nullable return types are good candidates to be replaced with <code>Optional</code>.
+For fields and parameters, <code>Optional</code> is disputed in Java and most parts of the Flink code case don’t use optional for fields.</p>
+
+<h3 id="avoid-code-duplication">Avoid Code Duplication</h3>
+
+<ol>
+ <li>Whenever you are about to copy/paste some code, or reproduce a similar type of functionality in a different place, think about the ways how to refactor/reuse/abstract the changes to avoid the duplication.</li>
+ <li>Common behavior between different specializations should be shared in a common component (or a shared superclass).</li>
+ <li>Always use “private static final” constants instead of duplicating strings or other special values at different locations. Constants should be declared in the top member area of a class.</li>
+</ol>
+
+<h3 id="design-for-testability">Design for Testability</h3>
+
+<p>Code that is easily testable typically has good separation of concerns and is structured to be reusable outside the original context (by being easily reusable in tests).</p>
+
+<p>A good summary or problems / symptoms and recommended refactoring is in the PDF linked below.
+Please note that while the examples in the PDF often use a dependency injection framework (Guice), it works in the same way without such a framework.<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup></p>
+
+<p><a href="http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf">http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf</a></p>
+
+<p>Here is a compact summary of the most important aspects.</p>
+
+<p><strong>Inject dependencies</strong></p>
+
+<p>Reusability becomes easier if constructors don’t create their dependencies (the objects assigned to the fields), but accept them as parameters.</p>
+
+<ul>
+ <li>Effectively, constructors should have no <code>new</code> keyword.</li>
+ <li>Exceptions are creating a new empty collection (<code>new ArrayList<>()</code>) or similar auxiliary fields (objects that have only primitive dependencies).</li>
+</ul>
+
+<p>To make instantiation easy / readable, add factory methods or additional convenience constructors to construct whole object with dependencies.</p>
+
+<p>In no case should it ever be required to use a reflection or a “Whitebox” util to change the fields of an object in a test, or to use PowerMock to intercept a “new” call and supply a mock.</p>
+
+<p><strong>Avoid “too many collaborators”</strong></p>
+
+<p>If you have to take a big set of other components into account during testing (“too many collaborators”), consider refactoring.</p>
+
+<p>The component/class you want to test probably depends on another broad component (and its implementation), rather than on the minimal interface (abstraction) required for its work.</p>
+
+<p>In that case, segregate the interfaces (factor out the minimal required interface) and supply a test stub in that case.</p>
+
+<ul>
+ <li>For example, if testing a S3RecoverableMultiPartUploader requires actual S3 access
+then the S3 access should be factored out into an interface and test should replace it by a test stub</li>
+ <li>This naturally requires to be able to inject dependencies (see above)</li>
+</ul>
+
+<p>⇒ Please note that these steps often require more effort in implementing tests (factoring out interfaces, creating dedicated test stubs), but make the tests more resilient to changes in other components, i.e., you do not need to touch the tests when making unrelated changes.</p>
+
+<p><strong>Write targeted tests</strong></p>
+
+<ul>
+ <li><span style="text-decoration:underline;">Test contracts not implementations</span>: Test that after a sequence of actions, the components are in a certain state, rather than testing that the components followed a sequence of internal state modifications.
+ <ul>
+ <li>For example, a typical antipattern is to check whether one specific method was called as part of the test</li>
+ </ul>
+ </li>
+ <li>
+ <p>A way to enforce this is to try to follow the <em>Arrange</em>, <em>Act</em>, <em>Assert</em> test structure when writing a unit test (<a href="https://xp123.com/articles/3a-arrange-act-assert/">https://xp123.com/articles/3a-arrange-act-assert/</a>)</p>
+
+ <p>This helps to communicate the intention of the test (what is the scenario under test) rather than the mechanics of the tests. The technical bits go to a static methods at the bottom of the test class.</p>
+
+ <p>Example of tests in Flink that follow this pattern are:</p>
+
+ <ul>
+ <li><a href="https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java">https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java</a></li>
+ <li><a href="https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java">https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java</a></li>
+ </ul>
+ </li>
+</ul>
+
+<p><strong>Avoid Mockito - Use reusable test implementations</strong></p>
+
+<ul>
+ <li>Mockito-based tests tend to be costly to maintain in the long run by encouraging duplication of functionality and testing for implementation rather than effect
+ <ul>
+ <li>More details: <a href="https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8">https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8</a></li>
+ </ul>
+ </li>
+ <li>Instead, create reusable test implementations and utilities
+ <ul>
+ <li>That way, when some class changes, we only have to update a few test utils or mocks</li>
+ </ul>
+ </li>
+</ul>
+
+<h3 id="performance-awareness">Performance Awareness</h3>
+
+<p>We can conceptually distinguish between code that “coordinates” and code that “processes data”. Code that coordinates should always favor simplicity and cleanness. Data processing code is highly performance critical and should optimize for performance.</p>
+
+<p>That means still applying the general idea of the sections above, but possibly forgoing some aspects in some place, in order to achieve higher performance.</p>
+
+<p><strong>Which code paths are Data Processing paths?</strong></p>
+
+<ul>
+ <li><span style="text-decoration:underline;">Per-record code paths:</span> Methods and code paths that are called for each record. Found for example in Connectors, Serializers, State Backends, Formats, Tasks, Operators, Metrics, runtime data structures, etc.</li>
+ <li><span style="text-decoration:underline;">I/O methods:</span> Transferring messages or chunks of data in buffers. Examples are in the RPC system, Network Stack, FileSystems, Encoders / Decoders, etc.</li>
+</ul>
+
+<p><strong>Things that performance critical code may do that we would otherwise avoid</strong></p>
+
+<ul>
+ <li>Using (and reusing) mutable objects to take pressure off the GC (and sometimes help with cache locality), thus forgoing the strive for immutability.</li>
+ <li>Using primitive types, arrays of primitive types, or MemorySegment/ByteBuffer and encoding meaning into the primitive types and byte sequences, rather than encapsulating the behavior in dedicated classes and using objects.</li>
+ <li>Structuring the code to amortize expensive work (allocations, lookups, virtual method calls, …) across multiple records, for example by doing the work once per buffer/bundle/batch.</li>
+ <li>Code layout optimized for the JIT rather than for readability. Examples are inlining fields from other classes (in cases where it is doubtful whether the JIT would do that optimization at runtime), or structuring code to help the JIT compiler with inlining, loop unrolling, vectorization, etc.</li>
+</ul>
+
+<h2 id="concurrency-and-threading">4. Concurrency and Threading</h2>
+
+<p><strong>Most code paths should not require any concurrency.</strong> The right internal abstractions should obviate the need for concurrency in almost all cases.</p>
+
+<ul>
+ <li>The Flink core and runtime use concurrency to provide these building blocks.
+Examples are in the RPC system, Network Stack, in the Task’s mailbox model, or some predefined Source / Sink utilities.</li>
+ <li>We are not fully there, but any new addition that introduces implements its own concurrency should be under scrutiny, unless it falls into the above category of core system building blocks.</li>
+ <li>Contributors should reach out to committers if they feel they need to implement concurrent code to see if there is an existing abstraction/building-block, or if one should be added.</li>
+</ul>
+
+<p><strong>When developing a component think about threading model and synchronization points ahead.</strong></p>
+
+<ul>
+ <li>For example: single threaded, blocking, non-blocking, synchronous, asynchronous, multi threaded, thread pool, message queues, volatile, synchronized block/methods, mutexes, atomics, callbacks, …</li>
+ <li>Getting those things right and thinking about them ahead is even more important than designing classes interfaces/responsibilities, since it’s much harder to change later on.</li>
+</ul>
+
+<p><strong>Try to avoid using threads all together if possible in any way.</strong></p>
+
+<ul>
+ <li>If you feel you have a case for spawning a thread, point this out in the pull request as something to be explicitly reviewed.</li>
+</ul>
+
+<p><strong>Be aware that using threads is in fact much harder than it initially looks</strong></p>
+
+<ul>
+ <li>Clean shutdown of threads is very tricky.</li>
+ <li>Handling interruptions in a rock solid fashion (avoid both slow shutdown and live locks) requires almost a Java Wizard</li>
+ <li>Ensuring clean error propagation out of threads in all cases needs thorough design.</li>
+ <li>Complexity of multi-threaded application/component/class grows exponentially, with each additional synchronisation point/block/critical section. Your code initially might be easy enough to understand, but can quickly grow beyond that point.</li>
+ <li>Proper testing of multithreaded code is basically impossible, while alternative approaches (like asynchronous code, non-blocking code, actor model with message queues) are quite easy to test.</li>
+ <li>Usually multi-threaded code is often even less efficient compared to alternative approaches on modern hardware.</li>
+</ul>
+
+<p><strong>Be aware of the java.util.concurrent.CompletableFuture</strong></p>
+
+<ul>
+ <li>Like with other concurrent code, there should rarely be the need to use a CompletableFuture</li>
+ <li>Completing a future would also complete on the calling thread any chained futures that are waiting for the result to be completed, unless a completion executor specified explicitly.</li>
+ <li>This can be intentional, if the entire execution should be synchronous / single-threaded, as for example in parts of the Scheduler / ExecutionGraph.
+ <ul>
+ <li>Flink even makes use of a “main-thread executor” to allow calling chained handlers in the same thread as a single-threaded RPC endpoint runs</li>
+ </ul>
+ </li>
+ <li>This can be unexpected, if the thread that completes the future is a sensitive thread.
+ <ul>
+ <li>It may be better to use <code>CompletableFuture.supplyAsync(value, executor)</code> in that case, instead of <code>future.complete(value)</code> when an executor is available</li>
+ </ul>
+ </li>
+ <li>When blocking on a future awaiting completion, always supply a timeout for a result instead of waiting indefinitely, and handle timeouts explicitly.</li>
+ <li>Use <code>CompletableFuture.allOf()</code>/<code>anyOf()</code>, <code>ExecutorCompletionService</code>, or <code>org.apache.flink.runtime.concurrent.FutureUtils#waitForAll</code> if you need to wait for: all the results/any of the results/all the results but handled by (approximate) completion order.</li>
+</ul>
+
+<h2 id="dependencies-and-modules">5. Dependencies and Modules</h2>
+
+<ul>
+ <li><strong>Keep the dependency footprint small</strong>
+ <ul>
+ <li>The more dependencies the harder it gets for the community to manage them as a whole.</li>
+ <li>Dependency management includes dependency conflicts, maintaining licenses and related notices, and handling security vulnerabilities.</li>
+ <li>Discuss whether the dependency should be shaded/relocated to avoid future conflicts.</li>
+ </ul>
+ </li>
+ <li><strong>Don’t add a dependency for just one method</strong>
+ <ul>
+ <li>Use Java built-in means if possible.</li>
+ <li>If the method is Apache-licensed, you can copy the method into a Flink utility class with proper attribution.</li>
+ </ul>
+ </li>
+ <li><strong>Declaration of dependencies</strong>
+ <ul>
+ <li>Declare dependencies that you explicitly rely on, whether it provides classes you directly import and use or it’s something that provides a service you directly use, like Log4J.</li>
+ <li>Transitive dependencies should only supply dependencies that are needed at runtime but that you don’t use yourself.</li>
+ <li>[<a href="https://stackoverflow.com/questions/15177661/maven-transitive-dependencies">source</a>]</li>
+ </ul>
+ </li>
+ <li><strong>Location of classes in the Maven modules</strong>
+ <ul>
+ <li>Whenever you create a new class, think about where to put it.</li>
+ <li>A class might be used by multiple modules in the future and might belong into a <code>common</code> module in this case.</li>
+ </ul>
+ </li>
+</ul>
+
+<hr />
+
+<div class="footnotes">
+ <ol>
+ <li id="fn:1">
+
+ <p>We are keeping such frameworks out of Flink, to make debugging easier and avoid dependency clashes. <a href="#fnref:1" class="reversefootnote">↩</a></p>
+ </li>
+ </ol>
+</div>
+
+
+ </div>
+</div>
+
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/zh/contributing/code-style-and-quality-components.html b/content/zh/contributing/code-style-and-quality-components.html
new file mode 100644
index 0000000..5305e79
--- /dev/null
+++ b/content/zh/contributing/code-style-and-quality-components.html
@@ -0,0 +1,413 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Components</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/zh/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/zh/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/zh/flink-architecture.html">Apache Flink 是什么?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/zh/usecases.html">应用场景</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/zh/poweredby.html">Flink 用户</a></li>
+
+ <!-- FAQ -->
+ <li><a href="/zh/faq.html">常见问题</a></li>
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/zh/downloads.html">下载</a></li>
+
+ <!-- Quickstart -->
+ <li>
+ <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8/quickstart/setup_quickstart.html" target="_blank">教程 <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">文档<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8" target="_blank">1.8 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/zh/gettinghelp.html">获取帮助</a></li>
+
+ <!-- Blog -->
+ <li><a href="/blog/"><b>Flink 博客</b></a></li>
+
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/zh/community.html">社区 & 项目信息</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/zh/roadmap.html">开发计划</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/zh/contributing/how-to-contribute.html">如何参与贡献</a></li>
+
+ <ul class="nav navbar-nav navbar-subnav">
+ <li >
+ <a href="/zh/contributing/contribute-code.html">贡献代码</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/contribute-documentation.html">贡献文档</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/improve-website.html">贡献网站</a>
+ </li>
+ </ul>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+ <a href="/contributing/code-style-and-quality-components.html">English</a>
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <h1>Apache Flink Code Style and Quality Guide — Components</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
+
+<hr />
+
+<div class="page-toc">
+<ul id="markdown-toc">
+ <li><a href="#component-specific-guidelines" id="markdown-toc-component-specific-guidelines">Component Specific Guidelines</a> <ul>
+ <li><a href="#configuration-changes" id="markdown-toc-configuration-changes">Configuration Changes</a></li>
+ <li><a href="#connectors" id="markdown-toc-connectors">Connectors</a></li>
+ <li><a href="#examples" id="markdown-toc-examples">Examples</a></li>
+ <li><a href="#table--sql-api" id="markdown-toc-table--sql-api">Table & SQL API</a></li>
+ </ul>
+ </li>
+</ul>
+
+</div>
+
+<h2 id="component-specific-guidelines">Component Specific Guidelines</h2>
+
+<p><em>Additional guidelines about changes in specific components.</em></p>
+
+<h3 id="configuration-changes">Configuration Changes</h3>
+
+<p>Where should the config option go?</p>
+
+<ul>
+ <li>
+ <p><span style="text-decoration:underline;">‘flink-conf.yaml’:</span> All configuration that pertains to execution behavior that one may want to standardize across jobs. Think of it as parameters someone would set wearing an “ops” hat, or someone that provides a stream processing platform to other teams.</p>
+ </li>
+ <li><span style="text-decoration:underline;">‘ExecutionConfig’</span>: Parameters specific to an individual Flink application, needed by the operators during execution. Typical examples are watermark interval, serializer parameters, object reuse.</li>
+ <li><span style="text-decoration:underline;">ExecutionEnvironment (in code)</span>: Everything that is specific to an individual Flink application and is only needed to build program / dataflow, not needed inside the operators during execution.</li>
+</ul>
+
+<p>How to name config keys:</p>
+
+<ul>
+ <li>
+ <p>Config key names should be hierarchical.
+Think of the configuration as nested objects (JSON style)</p>
+
+ <div class="highlight"><pre><code>taskmanager: {
+ jvm-exit-on-oom: true,
+ network: {
+ detailed-metrics: false,
+ request-backoff: {
+ initial: 100,
+ max: 10000
+ },
+ memory: {
+ fraction: 0.1,
+ min: 64MB,
+ max: 1GB,
+ buffers-per-channel: 2,
+ floating-buffers-per-gate: 16
+ }
+ }
+}
+</code></pre></div>
+ </li>
+ <li>
+ <p>The resulting config keys should hence be:</p>
+
+ <p><strong>NOT</strong> <code>"taskmanager.detailed.network.metrics"</code></p>
+
+ <p><strong>But rather</strong> <code>"taskmanager.network.detailed-metrics"</code></p>
+ </li>
+</ul>
+
+<h3 id="connectors">Connectors</h3>
+
+<p>Connectors are historically hard to implement and need to deal with many aspects of threading, concurrency, and checkpointing.</p>
+
+<p>As part of <a href="https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface">FLIP-27</a> we are working on making this much simpler for sources. New sources should not have to deal with any aspect of concurrency/threading and checkpointing any more.</p>
+
+<p>A similar FLIP can be expected for sinks in the near future.</p>
+
+<h3 id="examples">Examples</h3>
+
+<p>Examples should be self-contained and not require systems other than Flink to run. Except for examples that show how to use specific connectors, like the Kafka connector. Sources/sinks that are ok to use are <code>StreamExecutionEnvironment.socketTextStream</code>, which should not be used in production but is quite handy for exploring how things work, and file-based sources/sinks. (For streaming, there is the continuous file source)</p>
+
+<p>Examples should also not be pure toy-examples but strike a balance between real-world code and purely abstract examples. The WordCount example is quite long in the tooth by now but it’s a good showcase of simple code that highlights functionality and can do useful things.</p>
+
+<p>Examples should also be heavy in comments. They should describe the general idea of the example in the class-level Javadoc and describe what is happening and what functionality is used throughout the code. The expected input data and output data should also be described.</p>
+
+<p>Examples should include parameter parsing, so that you can run an example (from the Jar that is created for each example using <code>bin/flink run path/to/myExample.jar --param1 … --param2</code>.</p>
+
+<h3 id="table--sql-api">Table & SQL API</h3>
+
+<h4 id="semantics">Semantics</h4>
+
+<p><strong>The SQL standard should be the main source of truth.</strong></p>
+
+<ul>
+ <li>Syntax, semantics, and features should be aligned with SQL!</li>
+ <li>We don’t need to reinvent the wheel. Most problems have already been discussed industry-wide and written down in the SQL standard.</li>
+ <li>We rely on the newest standard (SQL:2016 or ISO/IEC 9075:2016 when writing this document <a href="https://standards.iso.org/ittf/PubliclyAvailableStandards/c065143_ISO_IEC_TR_19075-5_2016.zip">[download]</a> ). Not every part is available online but a quick web search might help here.</li>
+</ul>
+
+<p>Discuss divergence from the standard or vendor-specific interpretations.</p>
+
+<ul>
+ <li>Once a syntax or behavior is defined it cannot be undone easily.</li>
+ <li>Contributions that need to extent or interpret the standard need a thorough discussion with the community.</li>
+ <li>Please help committers by performing some initial research about how other vendors such as Postgres, Microsoft SQL Server, Oracle, Hive, Calcite, Beam are handling such cases.</li>
+</ul>
+
+<p>Consider the Table API as a bridge between the SQL and Java/Scala programming world.</p>
+
+<ul>
+ <li>The Table API is an Embedded Domain Specific Language for analytical programs following the relational model.
+It is not required to strictly follow the SQL standard in regards of syntax and names, but can be closer to the way a programming language would do/name functions and features, if that helps make it feel more intuitive.</li>
+ <li>The Table API might have some non-SQL features (e.g. map(), flatMap(), etc.) but should nevertheless “feel like SQL”. Functions and operations should have equal semantics and naming if possible.</li>
+</ul>
+
+<h4 id="common-mistakes">Common mistakes</h4>
+
+<ul>
+ <li>Support SQL’s type system when adding a feature.
+ <ul>
+ <li>A SQL function, connector, or format should natively support most SQL types from the very beginning.</li>
+ <li>Unsupported types lead to confusion, limit the usability, and create overhead by touching the same code paths multiple times.</li>
+ <li>For example, when adding a <code>SHIFT_LEFT</code> function, make sure that the contribution is general enough not only for <code>INT</code> but also <code>BIGINT</code> or <code>TINYINT</code>.</li>
+ </ul>
+ </li>
+</ul>
+
+<h4 id="testing">Testing</h4>
+
+<p>Test for nullability.</p>
+
+<ul>
+ <li>SQL natively supports <code>NULL</code> for almost every operation and has a 3-valued boolean logic.</li>
+ <li>Make sure to test every feature for nullability as well.</li>
+</ul>
+
+<p>Avoid full integration tests</p>
+
+<ul>
+ <li>Spawning a Flink mini-cluster and performing compilation of generated code for a SQL query is expensive.</li>
+ <li>Avoid integration tests for planner tests or variations of API calls.</li>
+ <li>Instead, use unit tests that validate the optimized plan which comes out of a planner. Or test the behavior of a runtime operator directly.</li>
+</ul>
+
+<h4 id="compatibility">Compatibility</h4>
+
+<p>Don’t introduce physical plan changes in minor releases!</p>
+
+<ul>
+ <li>Backwards compatibility for state in streaming SQL relies on the fact that the physical execution plan remains stable. Otherwise the generated Operator Names/IDs change and state cannot be matched and restored.</li>
+ <li>Every bug fix that leads to changes in the optimized physical plan of a streaming pipeline hences breaks compatibility.</li>
+ <li>As a consequence, changes of the kind that lead to different optimizer plans can only be merged in major releases for now.</li>
+</ul>
+
+<h4 id="scala--java-interoperability-legacy-code-parts">Scala / Java interoperability (legacy code parts)</h4>
+
+<p>Keep Java in mind when designing interfaces.</p>
+
+<ul>
+ <li>Consider whether a class will need to interact with a Java class in the future.</li>
+ <li>Use Java collections and Java Optional in interfaces for a smooth integration with Java code.</li>
+ <li>Don’t use features of case classes such as .copy() or apply() for construction if a class is subjected to be converted to Java.</li>
+ <li>Pure Scala user-facing APIs should use pure Scala collections/iterables/etc. for natural and idiomatic (“scalaesk”) integration with Scala.</li>
+</ul>
+
+
+
+ </div>
+</div>
+
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/zh/contributing/contribute-documentation.html b/content/zh/contributing/code-style-and-quality-formatting.html
similarity index 55%
copy from content/zh/contributing/contribute-documentation.html
copy to content/zh/contributing/code-style-and-quality-formatting.html
index 3b28504..52d40e3 100644
--- a/content/zh/contributing/contribute-documentation.html
+++ b/content/zh/contributing/code-style-and-quality-formatting.html
@@ -5,7 +5,7 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
- <title>Apache Flink: 贡献文档</title>
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Formatting</title>
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
<link rel="icon" href="/favicon.ico" type="image/x-icon">
@@ -127,9 +127,9 @@
<a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
</li>
<li >
- <a href="/zh/contributing/code-style-and-quality.html">代码样式与质量指南</a>
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
</li>
- <li class="active">
+ <li >
<a href="/zh/contributing/contribute-documentation.html">贡献文档</a>
</li>
<li >
@@ -148,7 +148,7 @@
<!-- Language Switcher -->
<li>
- <a href="/contributing/contribute-documentation.html">English</a>
+ <a href="/contributing/code-style-and-quality-formatting.html">English</a>
</li>
@@ -171,83 +171,153 @@
<div class="col-sm-9">
<div class="row-fluid">
<div class="col-sm-12">
- <h1>贡献文档</h1>
+ <h1>Apache Flink Code Style and Quality Guide — Formatting</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
- <p>良好的文档对任何类型的软件都至关重要。这对于复杂的软件系统尤其如此，例如 Apache Flink 这样的分布式数据处理引擎。Apache Flink 社区旨在提供简明、精确和完整的文档，并欢迎任何改进 Apache Flink 文档的贡献。</p>
+<hr />
<div class="page-toc">
<ul id="markdown-toc">
- <li><a href="#section" id="markdown-toc-section">获取文档资源</a></li>
- <li><a href="#section-1" id="markdown-toc-section-1">在开始贡献文档之前…</a></li>
- <li><a href="#section-2" id="markdown-toc-section-2">更新或扩展文档</a></li>
- <li><a href="#section-3" id="markdown-toc-section-3">中文文档翻译</a></li>
- <li><a href="#section-4" id="markdown-toc-section-4">提交你的贡献</a></li>
+ <li><a href="#java-code-formatting-style" id="markdown-toc-java-code-formatting-style">Java Code Formatting Style</a> <ul>
+ <li><a href="#license" id="markdown-toc-license">License</a></li>
+ <li><a href="#imports" id="markdown-toc-imports">Imports</a></li>
+ <li><a href="#naming" id="markdown-toc-naming">Naming</a></li>
+ <li><a href="#whitespaces" id="markdown-toc-whitespaces">Whitespaces</a></li>
+ <li><a href="#braces" id="markdown-toc-braces">Braces</a></li>
+ <li><a href="#javadocs" id="markdown-toc-javadocs">Javadocs</a></li>
+ <li><a href="#modifiers" id="markdown-toc-modifiers">Modifiers</a></li>
+ <li><a href="#files" id="markdown-toc-files">Files</a></li>
+ <li><a href="#misc" id="markdown-toc-misc">Misc</a></li>
+ </ul>
+ </li>
</ul>
</div>
-<h2 id="section">获取文档资源</h2>
-
-<p>Apache Flink 的文档和代码保存在相同的 <a href="http://git-scm.com/">git</a> 仓库中。这样做是为了确保代码和文档可以轻松保持同步。</p>
-
-<p>贡献文档的最简单方法是在 <a href="https://github.com/apache/flink">GitHub 上 Flink 的镜像仓库</a> 页面，通过单击右上角的 fork 按钮讲 Flink 克隆到你自己的 GitHub 帐户中。如果你没有 GitHub 帐户，可以免费创建一个帐户。</p>
+<h2 id="java-code-formatting-style">Java Code Formatting Style</h2>
-<p>接下来，将 fork 的代码克隆到本地计算机。</p>
+<p>We recommend to set up the IDE to automatically check the code style. Please follow the <a href="https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/ide_setup.html#checkstyle-for-java">IDE setup guide</a> for that.</p>
-<div class="highlight"><pre><code>git clone https://github.com/<your-user-name>/flink.git
-</code></pre></div>
+<h3 id="license">License</h3>
-<p>文档位于 Flink 代码库的 <code>docs/</code> 子目录中。</p>
-
-<h2 id="section-1">在开始贡献文档之前…</h2>
+<ul>
+ <li><strong>Apache license headers.</strong> Make sure you have Apache License headers in your files. The RAT plugin is checking for that when you build the code.</li>
+</ul>
-<p>…请确保已经有一个相对应的 <a href="https://issues.apache.org/jira/browse/FLINK">Jira</a> issue 存在了。我们要求所有文档更改都需要关联一个 Jira issue，除了一些微不足道的修复，如拼写错误。</p>
+<h3 id="imports">Imports</h3>
+
+<ul>
+ <li><strong>Empty line before and after package declaration.</strong></li>
+ <li><strong>No unused imports.</strong></li>
+ <li><strong>No redundant imports.</strong></li>
+ <li><strong>No wildcard imports.</strong> They can cause problems when adding to the code and in some cases even during refactoring.</li>
+ <li><strong>Import order.</strong> Imports must be ordered alphabetically, grouped into the following blocks, with each block separated by an empty line:
+ <ul>
+ <li><imports from org.apache.flink.*></li>
+ <li><imports from org.apache.flink.shaded.*></li>
+ <li><imports from other libraries></li>
+ <li><imports from javax.*></li>
+ <li><imports from java.*></li>
+ <li><imports from scala.*></li>
+ <li><static imports></li>
+ </ul>
+ </li>
+</ul>
-<h2 id="section-2">更新或扩展文档</h2>
+<h3 id="naming">Naming</h3>
-<p>Flink 文档是用 <a href="http://daringfireball.net/projects/markdown/">Markdown</a> 编写的。Markdown 是一种轻量级标记语言，可以通过工具转化成 HTML。</p>
+<ul>
+ <li><strong>Package names must start with a letter, and must not contain upper-case letters or special characters.</strong>
+ <strong>Non-private static final fields must be upper-case, with words being separated by underscores.</strong>(<code>MY_STATIC_VARIABLE</code>)</li>
+ <li><strong>Non-static fields/methods must be in lower camel case.</strong> (<code>myNonStaticField</code>)</li>
+</ul>
-<p>为了更新或扩展文档，你必须修改 Markdown (<code>.md</code>) 文件。请通过在预览模式下启动构建脚本来验证你的更改。</p>
+<h3 id="whitespaces">Whitespaces</h3>
-<div class="highlight"><pre><code>cd docs
-./build_docs.sh -p
-</code></pre></div>
+<ul>
+ <li><strong>Tabs vs. spaces.</strong> We are using tabs for indentation, not spaces.
+We are aware that spaces are a bit nicer; it just happened to be that we started with tabs a long time ago (because Eclipse’s default style used tabs then), and we tried to keep the code base homogeneous (not mix tabs and spaces).</li>
+ <li><strong>No trailing whitespace.</strong></li>
+ <li><strong>Spaces around operators/keywords.</strong> Operators (<code>+</code>, <code>=</code>, <code>></code>, …) and keywords (<code>if</code>, <code>for</code>, <code>catch</code>, …) must have a space before and after them, provided they are not at the start or end of the line.</li>
+</ul>
-<p>该脚本会将 Markdown 文件编译成静态 HTML 页面并在本地启动一个 Web 服务器。在浏览器中打开 <code>http://localhost:4000</code>，查看包含更改文档页面。当你修改并保存 Markdown 文件，然后刷新浏览器，修改过的文档将自动被重新编译和更新。</p>
+<h3 id="braces">Braces</h3>
-<p>如果有任何疑问，请在开发者邮件列表随时提问。</p>
+<ul>
+ <li><strong>Left curly braces (<code>{</code>) must not be placed on a new line.</strong></li>
+ <li><strong>Right curly braces (<code>}</code>) must always be placed at the beginning of the line.</strong></li>
+ <li><strong>Blocks.</strong> All statements after <code>if</code>, <code>for</code>, <code>while</code>, <code>do</code>, … must always be encapsulated in a block with curly braces (even if the block contains one statement).</li>
+</ul>
-<h2 id="section-3">中文文档翻译</h2>
+<h3 id="javadocs">Javadocs</h3>
-<p>Flink 社区正在同时维护英文和中文文档。所以如果你想要更新或扩展文档，英文和中文文档都需要更新。如果你不熟悉中文，请创建一个用于中文文档翻译的 JIRA 并附上 <code>chinese-translation</code> 的组件名，并与当前JIRA关联起来。如果你熟悉中文，我们鼓励在一个 pull request 中同时更新两边的文档。</p>
+<ul>
+ <li><strong>All public/protected methods and classes must have a Javadoc.</strong></li>
+ <li><strong>The first sentence of the Javadoc must end with a period.</strong></li>
+ <li><strong>Paragraphs must be separated with a new line, and started with <p>.</strong></li>
+</ul>
-<p><em>注意：Flink 社区目前正在翻译中文文档，有部分文档可能还未翻译。如果你正在更新的文档还未翻译，可以简单地将英文改动复制到中文文档中。</em></p>
+<h3 id="modifiers">Modifiers</h3>
-<p>中文文档与对应的英文文档位于相同的目录下。两种语言的文件名是一样的，只不过中文文档会以 <code>.zh.md</code> 作为后缀。你可以根据英文文档的改动来更新或扩展 <code>.zh.md</code> 文件。如果对应的 <code>.zh.md</code> 文件还未创建，你可以拷贝现有的英文文件，名称以 <code>.zh.md</code> 作为后缀。它将在 <code>/zh</code> 文件夹下生成与英文文档同名的 html 文件。</p>
+<ul>
+ <li><strong>No redundant modifiers.</strong> For example, public modifiers in interface methods.</li>
+ <li><strong>Follow JLS3 modifier order.</strong> Modifiers must be ordered in the following order: public, protected, private, abstract, static, final, transient, volatile, synchronized, native, strictfp.</li>
+</ul>
-<h2 id="section-4">提交你的贡献</h2>
+<h3 id="files">Files</h3>
-<p>Flink 项目通过 <a href="https://github.com/apache/flink">GitHub Mirror</a> 的 <a href="https://help.github.com/articles/using-pull-requests">Pull Request</a> 方式接受文档的贡献。Pull request 是一种提供补丁的简单方法，它提供了一个指向包含更改的代码分支的链接。</p>
+<ul>
+ <li><strong>All files must end with <code>\n</code>.</strong></li>
+ <li><strong>File length must not exceed 3000 lines.</strong></li>
+</ul>
-<p>请按照以下步骤准备和提交 pull request。</p>
+<h3 id="misc">Misc</h3>
-<ol>
- <li>
- <p>将更改提交到本地 git 仓库。提交消息应该以 <code>[FLINK-XXXX]</code> 开头，对应了相关联的 Jira issue。</p>
- </li>
- <li>
- <p>将你提交的贡献推送到 GitHub 上你 fork 的 Flink 仓库中。</p>
+<ul>
+ <li><strong>Arrays must be defined Java-style.</strong> For example, <code>public String[] array</code>.</li>
+ <li><strong>Use Flink Preconditions.</strong> To increase homogeneity, consistently use the <code>org.apache.flink.Preconditions</code> methods <code>checkNotNull</code> and <code>checkArgument</code> rather than Apache Commons Validate or Google Guava.</li>
+</ul>
- <p><code>
- git push origin myBranch
-</code></p>
- </li>
- <li>
- <p>打开你的 fork 仓库网页（<code>https://github.com/<your-user-name>/flink</code>）并使用 “Create Pull Request” 按钮开始创建 pull request。确保 base fork 是<code>apache/flink master</code>，并且 head fork 是包含更改的分支。再为 pull request 添加一个有意义的描述并创建它。</p>
- </li>
-</ol>
+<hr />
-<p>也可以将补丁（patch）附加到 <a href="https://issues.apache.org/jira/browse/FLINK">Jira</a> issue 上。</p>
</div>
diff --git a/content/zh/contributing/code-style-and-quality-java.html b/content/zh/contributing/code-style-and-quality-java.html
new file mode 100644
index 0000000..bc233ee
--- /dev/null
+++ b/content/zh/contributing/code-style-and-quality-java.html
@@ -0,0 +1,383 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Java</title>
+ <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Bootstrap -->
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css">
+ <link rel="stylesheet" href="/css/flink.css">
+ <link rel="stylesheet" href="/css/syntax.css">
+
+ <!-- Blog RSS feed -->
+ <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" />
+
+ <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+ <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+ <script src="/js/jquery.min.js"></script>
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+
+
+ <!-- Main content. -->
+ <div class="container">
+ <div class="row">
+
+
+ <div id="sidebar" class="col-sm-3">
+
+
+<!-- Top navbar. -->
+ <nav class="navbar navbar-default">
+ <!-- The logo. -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <div class="navbar-logo">
+ <a href="/zh/">
+ <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px">
+ </a>
+ </div>
+ </div><!-- /.navbar-header -->
+
+ <!-- The navigation links. -->
+ <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+ <ul class="nav navbar-nav navbar-main">
+
+ <!-- First menu section explains visitors what Flink is -->
+
+ <!-- What is Stream Processing? -->
+ <!--
+ <li><a href="/zh/streamprocessing1.html">What is Stream Processing?</a></li>
+ -->
+
+ <!-- What is Flink? -->
+ <li><a href="/zh/flink-architecture.html">Apache Flink 是什么?</a></li>
+
+
+
+ <!-- Use cases -->
+ <li><a href="/zh/usecases.html">应用场景</a></li>
+
+ <!-- Powered by -->
+ <li><a href="/zh/poweredby.html">Flink 用户</a></li>
+
+ <!-- FAQ -->
+ <li><a href="/zh/faq.html">常见问题</a></li>
+
+
+ <!-- Second menu section aims to support Flink users -->
+
+ <!-- Downloads -->
+ <li><a href="/zh/downloads.html">下载</a></li>
+
+ <!-- Quickstart -->
+ <li>
+ <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8/quickstart/setup_quickstart.html" target="_blank">教程 <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+ <!-- Documentation -->
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">文档<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.8" target="_blank">1.8 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+ </ul>
+ </li>
+
+ <!-- getting help -->
+ <li><a href="/zh/gettinghelp.html">获取帮助</a></li>
+
+ <!-- Blog -->
+ <li><a href="/blog/"><b>Flink 博客</b></a></li>
+
+
+
+ <!-- Third menu section aim to support community and contributors -->
+
+ <!-- Community -->
+ <li><a href="/zh/community.html">社区 & 项目信息</a></li>
+
+ <!-- Roadmap -->
+ <li><a href="/zh/roadmap.html">开发计划</a></li>
+
+ <!-- Contribute -->
+ <li><a href="/zh/contributing/how-to-contribute.html">如何参与贡献</a></li>
+
+ <ul class="nav navbar-nav navbar-subnav">
+ <li >
+ <a href="/zh/contributing/contribute-code.html">贡献代码</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/contribute-documentation.html">贡献文档</a>
+ </li>
+ <li >
+ <a href="/zh/contributing/improve-website.html">贡献网站</a>
+ </li>
+ </ul>
+
+
+ <!-- GitHub -->
+ <li>
+ <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a>
+ </li>
+
+
+
+ <!-- Language Switcher -->
+ <li>
+
+ <a href="/contributing/code-style-and-quality-java.html">English</a>
+
+ </li>
+
+ </ul>
+
+ <ul class="nav navbar-nav navbar-bottom">
+ <hr />
+
+ <!-- Twitter -->
+ <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ <!-- Visualizer -->
+ <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li>
+
+ </ul>
+ </div><!-- /.navbar-collapse -->
+ </nav>
+
+ </div>
+ <div class="col-sm-9">
+ <div class="row-fluid">
+ <div class="col-sm-12">
+ <h1>Apache Flink Code Style and Quality Guide — Java</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
+
+<hr />
+
+<div class="page-toc">
+<ul id="markdown-toc">
+ <li><a href="#java-language-features-and-libraries" id="markdown-toc-java-language-features-and-libraries">Java Language Features and Libraries</a> <ul>
+ <li><a href="#preconditions-and-log-statements" id="markdown-toc-preconditions-and-log-statements">Preconditions and Log Statements</a></li>
+ <li><a href="#generics" id="markdown-toc-generics">Generics</a></li>
+ <li><a href="#equals--hashcode" id="markdown-toc-equals--hashcode">equals() / hashCode()</a></li>
+ <li><a href="#java-serialization" id="markdown-toc-java-serialization">Java Serialization</a></li>
+ <li><a href="#java-reflection" id="markdown-toc-java-reflection">Java Reflection</a></li>
+ <li><a href="#collections" id="markdown-toc-collections">Collections</a></li>
+ <li><a href="#lambdas" id="markdown-toc-lambdas">Lambdas</a></li>
+ <li><a href="#java-streams" id="markdown-toc-java-streams">Java Streams</a></li>
+ </ul>
+ </li>
+</ul>
+
+</div>
+
+<h2 id="java-language-features-and-libraries">Java Language Features and Libraries</h2>
+
+<h3 id="preconditions-and-log-statements">Preconditions and Log Statements</h3>
+
+<ul>
+ <li>Never concatenate strings in the parameters
+ <ul>
+ <li><span style="text-decoration:underline;">Don’t:</span> <code>Preconditions.checkState(value <= threshold, "value must be below " + threshold)</code></li>
+ <li><span style="text-decoration:underline;">Don’t:</span> <code>LOG.debug("value is " + value)</code></li>
+ <li><span style="text-decoration:underline;">Do:</span> <code>Preconditions.checkState(value <= threshold, "value must be below %s", threshold)</code></li>
+ <li><span style="text-decoration:underline;">Do:</span> <code>LOG.debug("value is {}", value)</code></li>
+ </ul>
+ </li>
+</ul>
+
+<h3 id="generics">Generics</h3>
+
+<ul>
+ <li><strong>No raw types:</strong> Do not use raw types, unless strictly necessary (sometimes necessary for signature matches, arrays).</li>
+ <li><strong>Suppress warnings for unchecked conversions:</strong> Add annotations to suppress warnings, if they cannot be avoided (such as “unchecked”, or “serial”). Otherwise warnings about generics flood the build and drown relevant warnings.</li>
+</ul>
+
+<h3 id="equals--hashcode">equals() / hashCode()</h3>
+
+<ul>
+ <li><strong>equals() / hashCode() should be added when they are well defined only.</strong></li>
+ <li>They should <strong>not be added to enable a simpler assertion in tests</strong> when they are not well defined. Use hamcrest matchers in that case: <a href="https://github.com/junit-team/junit4/wiki/matchers-and-assertthat">https://github.com/junit-team/junit4/wiki/matchers-and-assertthat</a></li>
+ <li>A common indicator that the methods are not well defined is when they take a subset of the fields into account (other than fields that are purely auxiliary).</li>
+ <li>When the methods take mutable fields into account, you often have a design issue. The <code>equals()</code>/<code>hashCode()</code> methods suggest to use the type as a key, but the signatures suggest it is safe to keep mutating the type.</li>
+</ul>
+
+<h3 id="java-serialization">Java Serialization</h3>
+
+<ul>
+ <li><strong>Do not use Java Serialization for anything !!!</strong></li>
+ <li><strong>Do not use Java Serialization for anything !!! !!!</strong></li>
+ <li><strong>Do not use Java Serialization for anything !!! !!! !!!</strong></li>
+ <li>Internal to Flink, Java serialization is used to transport messages and programs through RPC. This is the only case where we use Java serialization. Because of that, some classes need to be serializable (if they are transported via RPC).</li>
+ <li>
+ <p><strong>Serializable classes must define a Serial Version UID:</strong></p>
+
+ <p><code>private static final long serialVersionUID = 1L;</code></p>
+ </li>
+ <li><strong>The Serial Version UID for new classes should start at 1</strong> and should generally be bumped on every incompatible change to the class according to the Java serialization compatibility definition (i.e: changing the type of a field, or moving the position of a class in the class hierarchy).</li>
+</ul>
+
+<h3 id="java-reflection">Java Reflection</h3>
+
+<p><strong>Avoid using Java’s Reflection API</strong></p>
+
+<ul>
+ <li>Java’s Reflection API can be a very useful tool in certain cases but in all cases it is a hack and one should research for alternatives. The only cases where Flink should use reflection are
+ <ul>
+ <li>Dynamically loading implementations from another module (like webUI, additional serializers, pluggable query processors).</li>
+ <li>Extracting types inside the TypeExtractor class. This is fragile enough and should not be done outside the TypeExtractor class.</li>
+ <li>Some cases of cross-JDK version features, where we need to use reflection because we cannot assume a class/method to be present in all versions.</li>
+ </ul>
+ </li>
+ <li>If you need reflection for accessing methods or fields in tests, it usually indicates some deeper architectural issues, like wrong scoping, bad separation of concerns, or that there is no clean way to provide components / dependencies to the class that is tested</li>
+</ul>
+
+<h3 id="collections">Collections</h3>
+
+<ul>
+ <li><strong>ArrayList and ArrayDeque are almost always superior to LinkedList</strong>, except when frequently insert and deleting in the middle of the list</li>
+ <li><strong>For Maps, avoid patterns that require multiple lookups</strong>
+ <ul>
+ <li><code>contains()</code> before <code>get()</code> → <code>get()</code> and check null</li>
+ <li><code>contains()</code> before <code>put()</code> → <code>putIfAbsent()</code> or <code>computeIfAbsent()</code></li>
+ <li>Iterating over keys, getting values → iterate over <code>entrySet()</code></li>
+ </ul>
+ </li>
+</ul>
+
+<h3 id="lambdas">Lambdas</h3>
+
+<ul>
+ <li>
+ <p>Prefer non-capturing lambdas (lambdas that do not contain references to the outer scope). Capturing lambdas need to create a new object instance for every call. Non-capturing lambdas can use the same instance for each invocation.</p>
+
+ <p><strong>don’t:</strong>
+<code>
+map.computeIfAbsent(key, x -> key.toLowerCase())
+</code></p>
+
+ <p><strong>do:</strong>
+<code>
+map.computeIfAbsent(key, k -> k.toLowerCase());
+</code></p>
+ </li>
+ <li>
+ <p>Consider method references instead of inline lambdas</p>
+
+ <p><strong>don’t</strong>:
+<code>
+map.computeIfAbsent(key, k-> Loader.load(k));
+</code></p>
+
+ <p><strong>do:</strong>
+<code>
+map.computeIfAbsent(key, Loader::load);
+</code></p>
+ </li>
+</ul>
+
+<h3 id="java-streams">Java Streams</h3>
+
+<ul>
+ <li>Avoid Java Streams in any performance critical code.</li>
+ <li>The main motivation to use Java Streams would be to improve code readability. As such, they can be a good match in parts of the code that are not data-intensive, but deal with coordination..</li>
+ <li>Even in the latter case, try to limit the scope to a method, or a few private methods within an internal class.</li>
+</ul>
+
+
+
+ </div>
+</div>
+
+ </div>
+ </div>
+
+ <hr />
+
+ <div class="row">
+ <div class="footer text-center col-sm-12">
+ <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p>
+ <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
+ <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p>
+ </div>
+ </div>
+ </div><!-- /.container -->
+
+ <!-- Include all compiled plugins (below), or include individual files as needed -->
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script>
+ <script src="/js/codetabs.js"></script>
+ <script src="/js/stickysidebar.js"></script>
+
+ <!-- Google Analytics -->
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-52545728-1', 'auto');
+ ga('send', 'pageview');
+ </script>
+ </body>
+</html>
diff --git a/content/zh/contributing/code-style-and-quality.html b/content/zh/contributing/code-style-and-quality-preamble.html
similarity index 69%
rename from content/zh/contributing/code-style-and-quality.html
rename to content/zh/contributing/code-style-and-quality-preamble.html
index 12d48cb..b7617d4 100644
--- a/content/zh/contributing/code-style-and-quality.html
+++ b/content/zh/contributing/code-style-and-quality-preamble.html
@@ -5,7 +5,7 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
- <title>Apache Flink: Code Style and Quality Guide (PLACEHOLDER)</title>
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Preamble</title>
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
<link rel="icon" href="/favicon.ico" type="image/x-icon">
@@ -127,7 +127,7 @@
<a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
</li>
<li class="active">
- <a href="/zh/contributing/code-style-and-quality.html">代码样式与质量指南</a>
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
</li>
<li >
<a href="/zh/contributing/contribute-documentation.html">贡献文档</a>
@@ -148,7 +148,7 @@
<!-- Language Switcher -->
<li>
- <a href="/contributing/code-style-and-quality.html">English</a>
+ <a href="/contributing/code-style-and-quality-preamble.html">English</a>
</li>
@@ -171,13 +171,71 @@
<div class="col-sm-9">
<div class="row-fluid">
<div class="col-sm-12">
- <h1>Code Style and Quality Guide (PLACEHOLDER)</h1>
-
- <p><br />
-<br /></p>
-
-<p>This page is a placeholder for an updated Code Style and Quality Guide to be posted here, once the <a href="https://lists.apache.org/thread.html/461d0922787c62447c3baff3177a0c8f34fee7c95e2896e5ddf59691@%3Cdev.flink.apache.org%3E">discussion on the mailing list</a> has concluded.</p>
-
+ <h1>Apache Flink Code Style and Quality Guide — Preamble</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
+
+<hr />
+
+<p>This is an attempt to capture the code and quality standard that we want to maintain.</p>
+
+<p>A code contribution (or any piece of code) can be evaluated in various ways: One set of properties is whether the code is correct and efficient. This requires solving the <em>logical or algorithmic problem</em> correctly and well.</p>
+
+<p>Another set of properties is whether the code follows an intuitive design and architecture, whether it is well structured with right separation of concerns, and whether the code is easily understandable and makes its assumptions explicit. That set of properties requires solving the <em>software engineering problem</em> well. A good solution implies that the code is easily testable, maintainable also by other people than the original authors (because it is harder to accidentally break) [...]
+
+<p>While the first set of properties has rather objective approval criteria, the second set of properties is much harder to assess, but is of high importance for an open source project like Apache Flink. To make the code base inviting to many contributors, to make contributions easy to understand for developers that did not write the original code, and to make the code robust in the face of many contributions, well engineered code is crucial.<sup id="fnref:1"><a href="#fn:1" class="footn [...]
+
+<p>This is of course not a full guide on how to write well engineered code. There is a world of big books that try to capture that. This guide is meant as a checklist of best practices, patterns, anti-patterns, and common mistakes that we observed in the context of developing Flink.</p>
+
+<p>A big part of high-quality open source contributions is about helping the reviewer to understand the contribution and double-check the implications, so an important part of this guide is about how to structure a pull request for review.</p>
+
+<hr />
+
+<div class="footnotes">
+ <ol>
+ <li id="fn:1">
+
+ <p>In earlier days, we (the Flink community) did not always pay sufficient attention to this, making some components of Flink harder to evolve and to contribute to. <a href="#fnref:1" class="reversefootnote">↩</a></p>
+ </li>
+ </ol>
+</div>
</div>
diff --git a/content/zh/contributing/contribute-documentation.html b/content/zh/contributing/code-style-and-quality-scala.html
similarity index 56%
copy from content/zh/contributing/contribute-documentation.html
copy to content/zh/contributing/code-style-and-quality-scala.html
index 3b28504..3111e48 100644
--- a/content/zh/contributing/contribute-documentation.html
+++ b/content/zh/contributing/code-style-and-quality-scala.html
@@ -5,7 +5,7 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
- <title>Apache Flink: 贡献文档</title>
+ <title>Apache Flink: Apache Flink Code Style and Quality Guide — Scala</title>
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
<link rel="icon" href="/favicon.ico" type="image/x-icon">
@@ -127,9 +127,9 @@
<a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
</li>
<li >
- <a href="/zh/contributing/code-style-and-quality.html">代码样式与质量指南</a>
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
</li>
- <li class="active">
+ <li >
<a href="/zh/contributing/contribute-documentation.html">贡献文档</a>
</li>
<li >
@@ -148,7 +148,7 @@
<!-- Language Switcher -->
<li>
- <a href="/contributing/contribute-documentation.html">English</a>
+ <a href="/contributing/code-style-and-quality-scala.html">English</a>
</li>
@@ -171,83 +171,147 @@
<div class="col-sm-9">
<div class="row-fluid">
<div class="col-sm-12">
- <h1>贡献文档</h1>
+ <h1>Apache Flink Code Style and Quality Guide — Scala</h1>
+
+
+<ul class="list-group" style="padding-top: 30px; font-weight: bold;">
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-preamble.html">
+ Preamble
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-pull-requests.html">
+ Pull Requests & Changes
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-common.html">
+ Common Coding Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-java.html">
+ Java Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-scala.html">
+ Scala Language Guide
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-components.html">
+ Component Guides
+ </a>
+ </li>
+ <li class="list-group-item">
+ <a href="/contributing/code-style-and-quality-formatting.html">
+ Formatting Guide
+ </a>
+ </li>
+</ul>
- <p>良好的文档对任何类型的软件都至关重要。这对于复杂的软件系统尤其如此，例如 Apache Flink 这样的分布式数据处理引擎。Apache Flink 社区旨在提供简明、精确和完整的文档，并欢迎任何改进 Apache Flink 文档的贡献。</p>
+<hr />
<div class="page-toc">
<ul id="markdown-toc">
- <li><a href="#section" id="markdown-toc-section">获取文档资源</a></li>
- <li><a href="#section-1" id="markdown-toc-section-1">在开始贡献文档之前…</a></li>
- <li><a href="#section-2" id="markdown-toc-section-2">更新或扩展文档</a></li>
- <li><a href="#section-3" id="markdown-toc-section-3">中文文档翻译</a></li>
- <li><a href="#section-4" id="markdown-toc-section-4">提交你的贡献</a></li>
+ <li><a href="#scala-language-features" id="markdown-toc-scala-language-features">Scala Language Features</a> <ul>
+ <li><a href="#where-to-use-and-not-use-scala" id="markdown-toc-where-to-use-and-not-use-scala">Where to use (and not use) Scala</a></li>
+ <li><a href="#api-parity" id="markdown-toc-api-parity">API Parity</a></li>
+ <li><a href="#language-features" id="markdown-toc-language-features">Language Features</a></li>
+ <li><a href="#coding-formatting" id="markdown-toc-coding-formatting">Coding Formatting</a></li>
+ </ul>
+ </li>
</ul>
</div>
-<h2 id="section">获取文档资源</h2>
-
-<p>Apache Flink 的文档和代码保存在相同的 <a href="http://git-scm.com/">git</a> 仓库中。这样做是为了确保代码和文档可以轻松保持同步。</p>
-
-<p>贡献文档的最简单方法是在 <a href="https://github.com/apache/flink">GitHub 上 Flink 的镜像仓库</a> 页面，通过单击右上角的 fork 按钮讲 Flink 克隆到你自己的 GitHub 帐户中。如果你没有 GitHub 帐户，可以免费创建一个帐户。</p>
-
-<p>接下来，将 fork 的代码克隆到本地计算机。</p>
-
-<div class="highlight"><pre><code>git clone https://github.com/<your-user-name>/flink.git
-</code></pre></div>
-
-<p>文档位于 Flink 代码库的 <code>docs/</code> 子目录中。</p>
-
-<h2 id="section-1">在开始贡献文档之前…</h2>
+<h2 id="scala-language-features">Scala Language Features</h2>
-<p>…请确保已经有一个相对应的 <a href="https://issues.apache.org/jira/browse/FLINK">Jira</a> issue 存在了。我们要求所有文档更改都需要关联一个 Jira issue，除了一些微不足道的修复，如拼写错误。</p>
+<h3 id="where-to-use-and-not-use-scala">Where to use (and not use) Scala</h3>
-<h2 id="section-2">更新或扩展文档</h2>
+<p><strong>We use Scala for Scala APIs or pure Scala Libraries.</strong></p>
-<p>Flink 文档是用 <a href="http://daringfireball.net/projects/markdown/">Markdown</a> 编写的。Markdown 是一种轻量级标记语言，可以通过工具转化成 HTML。</p>
+<p><strong>We do not use Scala in the core APIs and runtime components. We aim to remove existing Scala use (code and dependencies) from those components.</strong></p>
-<p>为了更新或扩展文档，你必须修改 Markdown (<code>.md</code>) 文件。请通过在预览模式下启动构建脚本来验证你的更改。</p>
+<p>⇒ This is not because we do not like Scala, it is a consequence of “the right tool for the right job” approach (see below).</p>
-<div class="highlight"><pre><code>cd docs
-./build_docs.sh -p
-</code></pre></div>
+<p>For APIs, we develop the foundation in Java, and layer Scala on top.</p>
-<p>该脚本会将 Markdown 文件编译成静态 HTML 页面并在本地启动一个 Web 服务器。在浏览器中打开 <code>http://localhost:4000</code>，查看包含更改文档页面。当你修改并保存 Markdown 文件，然后刷新浏览器，修改过的文档将自动被重新编译和更新。</p>
-
-<p>如果有任何疑问，请在开发者邮件列表随时提问。</p>
+<ul>
+ <li>This has traditionally given the best interoperability for both Java and Scala</li>
+ <li>It does mean dedicated effort to keep the Scala API up to date</li>
+</ul>
-<h2 id="section-3">中文文档翻译</h2>
+<p>Why don’t we use Scala in the core APIs and runtime?</p>
-<p>Flink 社区正在同时维护英文和中文文档。所以如果你想要更新或扩展文档，英文和中文文档都需要更新。如果你不熟悉中文，请创建一个用于中文文档翻译的 JIRA 并附上 <code>chinese-translation</code> 的组件名，并与当前JIRA关联起来。如果你熟悉中文，我们鼓励在一个 pull request 中同时更新两边的文档。</p>
+<ul>
+ <li>The past has shown that Scala evolves too quickly with tricky changes in functionality. Each Scala version upgrade was a rather big effort process for the Flink community.</li>
+ <li>Scala does not always interact nicely with Java classes, e.g. Scala’s visibility scopes work differently and often expose more to Java consumers than desired</li>
+ <li>Scala adds an additional layer of complexity to artifact/dependency management.
+ <ul>
+ <li>We may want to keep Scala dependent libraries like Akka in the runtime, but abstract them via an interface and load them in a separate classloader, to keep them shielded and avoid version conflicts.</li>
+ </ul>
+ </li>
+ <li>Scala makes it very easy for knowledgeable Scala programmers to write code that is very hard to understand for programmers that are less knowledgeable in Scala. That is especially tricky for an open source project with a broad community of diverse experience levels. Working around this means restricting the Scala feature set by a lot, which defeats a good amount of the purpose of using Scala in the first place.</li>
+</ul>
-<p><em>注意：Flink 社区目前正在翻译中文文档，有部分文档可能还未翻译。如果你正在更新的文档还未翻译，可以简单地将英文改动复制到中文文档中。</em></p>
+<h3 id="api-parity">API Parity</h3>
-<p>中文文档与对应的英文文档位于相同的目录下。两种语言的文件名是一样的，只不过中文文档会以 <code>.zh.md</code> 作为后缀。你可以根据英文文档的改动来更新或扩展 <code>.zh.md</code> 文件。如果对应的 <code>.zh.md</code> 文件还未创建，你可以拷贝现有的英文文件，名称以 <code>.zh.md</code> 作为后缀。它将在 <code>/zh</code> 文件夹下生成与英文文档同名的 html 文件。</p>
+<p>Keep Java API and Scala API in sync in terms of functionality and code quality.</p>
-<h2 id="section-4">提交你的贡献</h2>
+<p>The Scala API should cover all the features of the Java APIs as well.</p>
-<p>Flink 项目通过 <a href="https://github.com/apache/flink">GitHub Mirror</a> 的 <a href="https://help.github.com/articles/using-pull-requests">Pull Request</a> 方式接受文档的贡献。Pull request 是一种提供补丁的简单方法，它提供了一个指向包含更改的代码分支的链接。</p>
+<p>Scala APIs should have a “completeness test”, like the following example from the DataStream API: <a href="https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala">https://github.com/apache/flink/blob/master/flink-streaming-scala/src/test/scala/org/apache/flink/streaming/api/scala/StreamingScalaAPICompletenessTest.scala</a></p>
-<p>请按照以下步骤准备和提交 pull request。</p>
+<h3 id="language-features">Language Features</h3>
-<ol>
- <li>
- <p>将更改提交到本地 git 仓库。提交消息应该以 <code>[FLINK-XXXX]</code> 开头，对应了相关联的 Jira issue。</p>
+<ul>
+ <li><strong>Avoid Scala implicits.</strong>
+ <ul>
+ <li>Scala’s implicits should only be used for user-facing API improvements such as the Table API expressions or type information extraction.</li>
+ <li>Don’t use them for internal “magic”.</li>
+ </ul>
</li>
- <li>
- <p>将你提交的贡献推送到 GitHub 上你 fork 的 Flink 仓库中。</p>
-
- <p><code>
- git push origin myBranch
-</code></p>
+ <li><strong>Add explicit types for class members.</strong>
+ <ul>
+ <li>
+ <p>Don’t rely on implicit type inference for class fields and methods return types:</p>
+
+ <p><strong>Don’t:</strong>
+ <code>
+ var expressions = new java.util.ArrayList[String]()
+ </code></p>
+
+ <p><strong>Do:</strong>
+ <code>
+ var expressions: java.util.List[String] = new java.util.ArrayList[]()
+ </code></p>
+ </li>
+ <li>
+ <p>Type inference for local variables on the stack is fine.</p>
+ </li>
+ </ul>
</li>
- <li>
- <p>打开你的 fork 仓库网页（<code>https://github.com/<your-user-name>/flink</code>）并使用 “Create Pull Request” 按钮开始创建 pull request。确保 base fork 是<code>apache/flink master</code>，并且 head fork 是包含更改的分支。再为 pull request 添加一个有意义的描述并创建它。</p>
+ <li><strong>Use strict visibility.</strong>
+ <ul>
+ <li>Avoid Scala’s package private features (such as private[flink]) and use regular private/protected instead.</li>
+ <li>Keep in mind that <code>private[flink]</code> and <code>protected</code> members are public in Java.</li>
+ <li>Keep in mind that <code>private[flink]</code> still exposes all members in Flink provided examples.</li>
+ </ul>
</li>
-</ol>
+</ul>
+
+<h3 id="coding-formatting">Coding Formatting</h3>
-<p>也可以将补丁（patch）附加到 <a href="https://issues.apache.org/jira/browse/FLINK">Jira</a> issue 上。</p>
+<p><strong>Use line wrapping to structure your code.</strong></p>
+
+<ul>
+ <li>Scala’s functional nature allows for long transformation chains (<code>x.map().map().foreach()</code>).</li>
+ <li>In order to force implementers to structure their code, the line length is therefore limited to 100 characters.</li>
+ <li>Use one line per transformation for better maintainability.</li>
+</ul>
</div>
diff --git a/content/zh/contributing/contribute-code.html b/content/zh/contributing/contribute-code.html
index eac4d02..00caf31 100644
--- a/content/zh/contributing/contribute-code.html
+++ b/content/zh/contributing/contribute-code.html
@@ -127,7 +127,7 @@
<a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
</li>
<li >
- <a href="/zh/contributing/code-style-and-quality.html">代码样式与质量指南</a>
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
</li>
<li >
<a href="/zh/contributing/contribute-documentation.html">贡献文档</a>
diff --git a/content/zh/contributing/contribute-documentation.html b/content/zh/contributing/contribute-documentation.html
index 3b28504..32db782 100644
--- a/content/zh/contributing/contribute-documentation.html
+++ b/content/zh/contributing/contribute-documentation.html
@@ -127,7 +127,7 @@
<a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
</li>
<li >
- <a href="/zh/contributing/code-style-and-quality.html">代码样式与质量指南</a>
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
</li>
<li class="active">
<a href="/zh/contributing/contribute-documentation.html">贡献文档</a>
diff --git a/content/zh/contributing/how-to-contribute.html b/content/zh/contributing/how-to-contribute.html
index b0d7efc..e82bd51 100644
--- a/content/zh/contributing/how-to-contribute.html
+++ b/content/zh/contributing/how-to-contribute.html
@@ -127,7 +127,7 @@
<a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
</li>
<li >
- <a href="/zh/contributing/code-style-and-quality.html">代码样式与质量指南</a>
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
</li>
<li >
<a href="/zh/contributing/contribute-documentation.html">贡献文档</a>
diff --git a/content/zh/contributing/improve-website.html b/content/zh/contributing/improve-website.html
index f79d66e..9732131 100644
--- a/content/zh/contributing/improve-website.html
+++ b/content/zh/contributing/improve-website.html
@@ -127,7 +127,7 @@
<a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
</li>
<li >
- <a href="/zh/contributing/code-style-and-quality.html">代码样式与质量指南</a>
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
</li>
<li >
<a href="/zh/contributing/contribute-documentation.html">贡献文档</a>
diff --git a/content/zh/contributing/reviewing-prs.html b/content/zh/contributing/reviewing-prs.html
index f46feb2..7eb72a1 100644
--- a/content/zh/contributing/reviewing-prs.html
+++ b/content/zh/contributing/reviewing-prs.html
@@ -127,7 +127,7 @@
<a href="/zh/contributing/reviewing-prs.html">审核 Pull Request</a>
</li>
<li >
- <a href="/zh/contributing/code-style-and-quality.html">代码样式与质量指南</a>
+ <a href="/zh/contributing/code-style-and-quality-preamble.html">代码样式与质量指南</a>
</li>
<li >
<a href="/zh/contributing/contribute-documentation.html">贡献文档</a>