You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@incubator.apache.org by yu...@apache.org on 2016/12/23 06:33:26 UTC
[1/3] incubator-rocketmq-site git commit: Post a blog-Mastering
Component Compatible Dependency
Repository: incubator-rocketmq-site
Updated Branches:
refs/heads/master e1829bac2 -> 665dc7d5e
Post a blog-Mastering Component Compatible Dependency
Project: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/commit/a18e4888
Tree: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/tree/a18e4888
Diff: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/diff/a18e4888
Branch: refs/heads/master
Commit: a18e488888ff5b65e6b55f12c1bd602c7be6b592
Parents: e1829ba
Author: yukon <yu...@apache.org>
Authored: Fri Dec 23 11:37:27 2016 +0800
Committer: yukon <yu...@apache.org>
Committed: Fri Dec 23 11:37:27 2016 +0800
----------------------------------------------------------------------
README.md | 2 +-
...mastering-component-compatible-dependency.md | 270 +++++++++++++++++++
2 files changed, 271 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/blob/a18e4888/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 682cb39..408f001 100644
--- a/README.md
+++ b/README.md
@@ -21,7 +21,7 @@ Please refer to **01-quick-start-guide.md** for more details.
### How to post articles to **Blog**?
New a .md file in rocketmq-sites/_posts/, Jekyll will finish the rest of the work.
-Please refer to **2013-01-05-markup-title-with-markup.md** for more details.
+Please refer to **2016-12-23-mastering-component-compatible-dependency.md** for more details.
### How to modify the navigation?
Please refer to **_data/navigation.yml** for more details.
http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/blob/a18e4888/_posts/2016-12-23-mastering-component-compatible-dependency.md
----------------------------------------------------------------------
diff --git a/_posts/2016-12-23-mastering-component-compatible-dependency.md b/_posts/2016-12-23-mastering-component-compatible-dependency.md
new file mode 100644
index 0000000..c39662a
--- /dev/null
+++ b/_posts/2016-12-23-mastering-component-compatible-dependency.md
@@ -0,0 +1,270 @@
+---
+title: "Mastering Component Compatible Dependency"
+categories:
+ - Maven
+tags:
+ - Compatibility
+ - Dependency
+ - Java
+ - Maven
+---
+
+# Preface
+
+This article mainly includes three parts.at first,I will introduce compatibility principle(more details see [here](http://blog.csdn.net/fengjia10/article/details/7799227)) briefly.followed by a detailed elaborating about Java component compatible dependency,including the interface-oriented programming,single component signature protection,single component compatibility protection and multi-component compatibility compile time checking.Finally is the review and prospect,especially about **Dependency Mediator** project.
+
+{% include toc %}
+
+# Compatibility
+
+what is compatibility\uff1fit is also the key to understanding the compatible dependency.Compatibility, often catch our eyes through two forms,**binary compatibility**,that means they can run the same executable code, typically machine code for a general purpose computer CPU.within the framework of Release-to-Release Binary Compatibility in SOM (Forman, Conner, Danforth, and Raper, Proceedings of OOPSLA '95), Java programming language binaries are binary compatible under all relevant transformations that the authors identify (with some caveats with respect to the addition of instance variables). here is a list of some important binary compatible changes that the Java programming language supports:
+
+* Reimplementing existing methods, constructors, and initializers to improve performance.
+* Changing methods or constructors to return values on inputs for which they previously either threw exceptions that normally should not occur or failed by going into an infinite loop or causing a deadlock.
+* Adding new fields, methods, or constructors to an existing class or interface.
+* Deleting private fields, methods, or constructors of a class.
+* When an entire package is updated, deleting default (package-only) access fields, methods, or constructors of classes and interfaces in the package.
+* Reordering the fields, methods, or constructors in an existing type declaration.
+* Moving a method upward in the class hierarchy.
+* Reordering the list of direct superinterfaces of a class or interface.
+* Inserting new class or interface types in the type hierarchy.
+
+Generally,java language is **upwards binary-compatible** with previous version,such as Java SE 6 is upwards binary-compatible with J2SE 5.0 except for some minor incompatibilities.You can get details from [here](http://www.oracle.com/technetwork/java/javase/compatibility-137541.html#binary).
+
+Another compatibility form,we call it **source compatibility**,meaning that recompilation is necessary.Java language does not support downward source compatibility. Such as if source files use new language features or Java SE platform APIs, they will not be usable with an earlier version of the Java platform. Usually it will throw similar error like this:
+
+```java
+java.lang.UnsupportedClassVersionError: com.alibaba.mq.core.MessageConsumer :
+ Unsupported major.minor version 51.0
+ at java.lang.ClassLoader.defineClass1(Native Method)
+ at java.lang.ClassLoader.defineClassCond(Unknown Source)
+```
+
+Java language source compatibility policy is as follows,except for any incompatibilities listed further below:
+
+* Maintenance releases (such as 1.7.25, 1.7.26) do not introduce any new language features or APIs. They will maintain source-compatibility with each other.
+* Functionality releases and major releases (such as 1.4.0,5.0,6.0,7.0,8.0) maintain upwards but not downwards source-compatibility.
+
+Let's sum up before going on:
+
+* Document those incompatibility cases as much as possible, even some rare circumstances and corner cases.
+* Recommend developer to provide facilities that alert developers to the impact of changes on pre-existing binaries that cannot be recompiled.
+* Generally speaking,Binary Compatibility problems may occur in the following situations:
+ * **Replacing the old with the new**,such as the Java platform Profiling Architecture in the Java SE6,JVMDI was removed and JVMPI was disabled in Java SE 6.
+ * **Combinatorial resolution**,such as Non class files have been moved from rt.jar in Java SE 6.if -Xbootclasspath:<path to rt.jar> is specified,and request any resource files will fail since these resources now reside in a different jar file called resources.jar.
+ * **Behavior change**,such as Java SE 6 will throw an OverlappingFileLockException,if the application attempts to lock the overlapping region other instances of FileChannel that has been locked, in order to realize the backward compatibility, it provides disableSystemWideOverlappingFileLockCheck system properties. again, also in Java SE 6, jar will preserve file modification dates and times during extract, through the sun.tools.jar.useExtractionTime=true to achieve backward compatibility.
+ * **Structure change**,this is also the key problem to solve in **Dependency Mediator** project. generally speaking,modification and deletion belonging to incompatible operations.and more attentions must be paid in the RPC service maintenance, more detailed description,please refer to an [article](http://blog.csdn.net/fengjia10/article/details/7799227) I wrote before. also you can read the the thirteenth chapter of Java language specification.
+
+# Component Compatible Dependency
+
+This article don't talk about **dependency analysis** and **dependency decoupling**,if you care these topics,please read [Dependency Analysis and the Modularisation of Java Programs](http://java.dzone.com/articles/dependency-analysis-and-1).highly recommending you try to dive into open source project [Architecture Explorer](http://xplrarc.massey.ac.nz/),through bcel technology,analyzing and measuring component's dependency relationship.but in my opinion,[Jdepend](http://clarkware.com/software/JDepend.html) was surely the most distinguished tool in dependency analysis and dependency decoupling area.it traverses Java class file directories and generates design quality metrics for each Java package,also allows you to automatically measure the quality of a design in terms of its extensibility, reusability, and maintainability to manage package dependencies effectively.
+
+Let's return,ask yourself\uff0chave you ever been disturbed by the following problems?
+
+* Your service depend a volatile service,volatile means your service depending neither an abstract class nor a interface,this class update at irregular intervals,you often encountered errors such as **NoSuchMethodError**,**NoSuchFieldError** and **NoClassDefFoundError** etc.
+* You are developing a new feature,but this feature can not backward compatible with previous version,
+what should you do ? How do you give a incompatible list in your release report.
+* Java language library is a superb collection of beautiful things,how do you make a rational dependency exclusion and effective mediate various dependency problems when developing.
+
+It is the time to introduce the key idea - **Mediating Components Compatible Dependency**.Specifically,it includs the following basic principles:
+
+* Interface-oriented programming
+* Single component signature protection
+* Single component compatibility protection
+* Multi-component compatibility compile time checking
+
+## Interface-oriented programming
+
+Interface-oriented programming,It's easier said than done.I will cite SLF4J design philosophy explain this principle.
+
+Differing from commons logging,SLF4J use **static binder technology** so as to avoid notorious [classloader problem](http://articles.qos.ch/classloader.html),every concrete log implementation must
+have a class named StaticLoggerBinder and implementing SLF4J spi LoggerFactoryBinder.thus you just depend SLF4J api,not concerned about underlying implementation.some people may asking,why choose SLF4J not commons logging\uff0cI can also depend commons logging api but not concret implementation\uff1fit's not performance but classloader problem.JCL discovery process relies on classloader hacks to find the logging framework at runtime but this mechanism leads to numerous problems including unexpected behavior,hard to debug classloading problems resulting in increased complexity.This is nicely captured by Ceki (the author of Log4J, SLF4J and Logback) in Think again before adopting the commons-logging API (which also mentions memory leaks problems observed with JCL).and this is why SLF4J, which uses static bindings, has been created. Let's see.how to find the concrete realizations in SLF4J:
+
+```java
+// We need to use the name of the StaticLoggerBinder class, but we can't reference
+// the class itself.
+private static String STATIC_LOGGER_BINDER_PATH = "org/slf4j/impl/StaticLoggerBinder.class";
+ /**
+ * It is LoggerFactory's responsibility to track version changes and manage
+ * the compatibility list.
+ * It is assumed that all versions in the 1.6 are mutually compatible.
+ */
+static private final String[] API_COMPATIBILITY_LIST = new String[]{"1.6", "1.7"};
+
+// private constructor prevents instantiation
+private LoggerFactory() {}
+private final static void versionSanityCheck() {
+ try {
+ String requested = StaticLoggerBinder.REQUESTED_API_VERSION;
+ boolean match = false;
+ for (int i = 0; i < API_COMPATIBILITY_LIST.length; i++) {
+ if (requested.startsWith(API_COMPATIBILITY_LIST[i])) {
+ match = true;
+ }
+ }
+ if (!match) {
+ Util.report("The requested version " + requested
+ + " by your slf4j binding is not compatible with "
+ + Arrays.asList(API_COMPATIBILITY_LIST).toString());
+ Util.report("See " + VERSION_MISMATCH + " for further details.");
+ }
+} catch (java.lang.NoSuchFieldError nsfe) {
+ // given our large user base and SLF4J's commitment to backward
+ // compatibility, we cannot cry here. Only for implementations
+ // which willingly declare a REQUESTED_API_VERSION field do we
+ // emit compatibility warnings.
+} catch (Throwable e) {
+ // we should never reach here
+ Util.report("Unexpected problem occured during version sanity check", e);
+ }
+}
+```
+
+you can use these skills at your library.just Let others depend on your interface package,detecting certain implementation statically.
+
+If you have developed an old version library,naming it as 2.x.x.next version,you hope to refactor some implementations,how to process multi-version compatibility?here is a skill from maven dependency plugin:
+
+
+```java
+String hint = isMaven31() ? "maven31" : isMaven2x() ? "maven2" : "maven3";
+
+/**
+* Check the current Maven version to see if it's Maven 2.x.
+*/
+protected static boolean isMaven2x()
+{
+ return !canFindCoreClass( "org.apache.maven.project.DependencyResolutionRequest" ); // Maven 3 specific
+}
+
+/**
+* Check the current Maven version to see if it's Maven 3.1.
+*/
+protected static boolean isMaven31()
+{
+ return canFindCoreClass( "org.eclipse.aether.artifact.Artifact" ); // Maven 3.1 specific
+}
+```
+
+## Single component signature protection
+
+Secondly,Single component signature protection,you can using [Animal Sniffer project](http://mojo.codehaus.org/animal-sniffer/) as well-known Guava library,Animal Sniffer provides tools to assist verifying that classes compiled with a newer JDK/API are compatible with an older JDK/API.
+
+```java
+<plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>animal-sniffer-maven-plugin</artifactId>
+ <configuration>
+ <signature>
+ <groupId>org.codehaus.mojo.signature</groupId>
+ <artifactId>java16-sun</artifactId>
+ <version>1.0</version>
+ </signature>
+ </configuration>
+ <executions>
+ <execution>
+ <id>check-java16-sun</id>
+ <phase>test</phase>
+ <goals>
+ <goal>check</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+```
+
+So from now on, when you performing a mvn test or mvn install operation. If there is an incompatibility in one of the projects, you would find some output like this:
+
+```java
+[INFO] [animal-sniffer:check {execution: check-java-api}]
+[INFO] Checking unresolved references to org.codehaus.mojo.signature:java16-sun:1.0
+[ERROR] Undefined reference: java/lang/String.contains(Ljava/lang/CharSequence;)Z in /home/admin/workspace/kafka/target/classes/org/kafka/tools...Producer.class
+```
+
+More details please see [here](http://blog.gvsig.org/2011/07/25/hunting-api-incompatibilities-with-the-animal-sniffer-project/).
+
+
+## Single component compatibility protection
+
+If your a SOA develper,you almost concerned with single component compatibility protection.[Clirr project] (http://clirr.sourceforge.net/) may be your best choice.Clirr is a tool that checks Java libraries for binary and source compatibility with older releases. Basically you give it two sets of jar files and Clirr dumps out a list of changes in the public api. The Clirr Ant task can be configured to break the build if it detects incompatible api changes. In a continuous integration process Clirr can automatically prevent accidental introduction of binary or source compatibility problems.
+
+How to use it\uff1fjust add some config snippets in your pom :
+
+```java
+<plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>clirr-maven-plugin</artifactId>
+ <version>2.6.1</version>
+</plugin>
+```
+
+then,run `mvn clirr:check`,output may be like this:
+
+```java
+[INFO] --- clirr-maven-plugin:2.6.1:check (default-cli) @ cmq.common ---
+[INFO] artifact com.alibaba.intl.base.cmq:cmq.common: checking for updates from b2bmirror-all
+[INFO] Comparing to version: 2.0.1
+[ERROR] 7005: com.alibaba.mq.client.CompletionListener: Parameter 1 of 'public void onCompletion(com.alibaba.mq.model.CommonMessage)' has changed its type to com.alibaba.mq.common.CommonMessage
+[ERROR] 7005: com.alibaba.mq.client.CompletionListener: Parameter 1 of 'public void onException(com.alibaba.mq.model.CommonMessage, java.lang.Exception)' has changed its type to com.alibaba.mq.common.CommonMessage
+[ERROR] 7005: com.alibaba.mq.client.MessageProducer: Parameter 1 of 'public com.alibaba.mq.client.CommonResult send(com.alibaba.mq.model.CommonMessage)' has changed its type to com.alibaba.mq.common.CommonMessage
+[ERROR] 7005: com.alibaba.mq.client.MessageProducer: Parameter 1 of 'public com.alibaba.mq.client.CommonResult send(com.alibaba.mq.model.CommonMessage, com.alibaba.mq.client.CompletionListener)' has changed its type to com.alibaba.mq.common.CommonMessage
+```
+
+
+
+Some inspiration in **Dependency Mediator** project just come from here.more topics about compatible,you can reference from series article [Evolving Java-based APIs](http://wiki.eclipse.org/index.php/Evolving_Java-based_APIs)
+
+## Multi-component compatibility compile time checking
+
+Last but not least,Multi-component compatibility compile time checking.it's the time to formally recommend project **Dependency Mediator**.
+
+unlike karaf and other lightness modular technology(like osgi,class names do not need to be unique,but the combination of class names and their defining ClassLoader must to be unique),**Dependency Mediator** try to remedy component dependency conflicting problem before the runtime rather than using customized [classLoader](http://www.onjava.com/pub/a/onjava/2005/04/13/dependencies.html) to agree with the coexistence of different version components,thus avoid some well-konwn errors,such as **NoSuchMethodError**,**NoSuchFieldError** and **NoClassDefFoundError** etc.
+
+What is the principle of **Dependency Mediator**\uff1f**Dependency Mediator** try to mediate various component conflicting problems.It uses jar or class as the smallest component unit,that is ComponentEntry:
+
+```java
+public class ComponentEntry implements Comparable<ComponentEntry> {
+ /**
+ * Similar file name
+ */
+ private String pathName;
+ /**
+ * Component name ,such as fully-qualified class name or jar name
+ */
+ private String name;
+ /**
+ * Name of the jar which contains this entry,this field may be null
+ */
+ private String jarName;
+ /**
+ * Jar entry meta info,if it is a jar file,this field may be null
+ */
+ private JarEntry entry;
+ ....
+```
+
+It could scan directory(also including classpath,if you set system property scanClasspath) and POM.if it happen to jar,but you setting checkJars to false,it could detect inner MANIFEST file,using **Build-Jdk** and **Built-By** properties to decide whether duplicated jars.thus,output only report whether including duplicated jar.but if you
+setting checkJars to true or nothing to do with this property,it will further analyzes whether existing incompatible cases.
+
+It also provides a maven plugin,if your project is a web project,you must run mvn insall before next step. Maven plugin also comply with the latest annotation mojo api rather than version 2.x.x doclet api.
+
+**Dependency Mediator** take advantage of **nearest definition** dependency mediation strategy. this determines what version of a dependency will be used when multiple versions of an artifact are encountered. **nearest definition**,means that it will use the version of the closest dependency to your project in the tree of dependencies. You can always guarantee a version by declaring it explicitly in your project's POM. Note that if two dependency versions are at the same depth in the dependency tree, until Maven 2.0.8 it was not defined which one would win, but since Maven 2.0.9 it's the order in the declaration that counts: the first declaration wins.
+**nearest definition** means that the version used will be the closest one to your project in the tree of dependencies, eg. if dependencies for A, B, and C are defined as A -> B -> C -> D 2.0 and A -> E -> D 1.0, then D 1.0 will be used when building A because the path from A to D through E is shorter. You could explicitly add a dependency to D 2.0 in A to force the use of D 2.0.
+
+If you consider the dependency hierarchy as a binary tree structure,Undoubtedly,**nearest definition** means the **minimum depth**. so if you put some higher version dependency library after lower version one(may be transitive dependency),you will face incompatible conflicting problems.Such as :
+
+```java
+[WARNING] Founded conflicting dependency component:org.apache.thrift:libthrift:jar Resolved version is org.apache.thrift:libthrift:jar:0.8.0:compile But found conflicting artifact org.apache.thrift:libthrift:0.9.1
+```
+
+In this case,you could fix this problem through place libthrift 0.9.1 before the library imported through transitive dependency.
+
+of course,all the establishment must meet a premise,that is:
+* **Functionality releases and major releases(such as 1.4.x, 1.5.x,2.x.x,3.x.x) have source compatibility problems.**
+
+In **Dependency Mediator** 2.0 version,you could customize version differentiation strategy,not limited the aformentioned.by the way, also in 2.0,I will improvement compatible dependency mediator, please wait and see.
+
+# Summary
+
+Through this article, you almost calculated the evolution process of the **Dependency Mediator** project.in my opinion,integration with the maven enforcer plugin may be a better choice.After all,I hope **Dependency Mediator** can keep sustainable development.I would try and donate this project to codehaus in the near future.
+
+If you have better idea or improving suggestion, please contact <de...@rocketmq.incubator.apache.org>.
[2/3] incubator-rocketmq-site git commit: Add some default pages.
Posted by yu...@apache.org.
Add some default pages.
Project: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/commit/2041fbe8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/tree/2041fbe8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/diff/2041fbe8
Branch: refs/heads/master
Commit: 2041fbe8e0beed4d3bf6fdb097199944431bde1a
Parents: a18e488
Author: yukon <yu...@apache.org>
Authored: Fri Dec 23 11:43:45 2016 +0800
Committer: yukon <yu...@apache.org>
Committed: Fri Dec 23 11:43:49 2016 +0800
----------------------------------------------------------------------
_pages/404.md | 16 +++
_pages/archive-layout-with-content.md | 218 +++++++++++++++++++++++++++++
_pages/category-archive.html | 16 +++
_pages/collection-archive.html | 23 +++
_pages/tag-archive.html | 16 +++
5 files changed, 289 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/blob/2041fbe8/_pages/404.md
----------------------------------------------------------------------
diff --git a/_pages/404.md b/_pages/404.md
new file mode 100644
index 0000000..9d0f860
--- /dev/null
+++ b/_pages/404.md
@@ -0,0 +1,16 @@
+---
+title: "Page Not Found"
+excerpt: "Page not found. Your pixels are in another canvas."
+sitemap: false
+permalink: /404.html
+---
+
+Sorry, but the page you were trying to view does not exist --- perhaps you can try searching for it below.
+
+<script type="text/javascript">
+ var GOOG_FIXURL_LANG = 'en';
+ var GOOG_FIXURL_SITE = '{{ site.url }}'
+</script>
+<script type="text/javascript"
+ src="//linkhelp.clients.google.com/tbproxy/lh/wm/fixurl.js">
+</script>
http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/blob/2041fbe8/_pages/archive-layout-with-content.md
----------------------------------------------------------------------
diff --git a/_pages/archive-layout-with-content.md b/_pages/archive-layout-with-content.md
new file mode 100644
index 0000000..6c9b737
--- /dev/null
+++ b/_pages/archive-layout-with-content.md
@@ -0,0 +1,218 @@
+---
+title: "Archive Layout with Content"
+layout: archive
+permalink: /archive-layout-with-content/
+---
+
+A variety of common markup showing how the theme styles them.
+
+# Header one
+
+## Header two
+
+### Header three
+
+#### Header four
+
+##### Header five
+
+###### Header six
+
+## Blockquotes
+
+Single line blockquote:
+
+> Stay hungry. Stay foolish.
+
+Multi line blockquote with a cite reference:
+
+> People think focus means saying yes to the thing you've got to focus on. But that's not what it means at all. It means saying no to the hundred other good ideas that there are. You have to pick carefully. I'm actually as proud of the things we haven't done as the things I have done. Innovation is saying no to 1,000 things.
+
+<cite>Steve Jobs</cite> --- Apple Worldwide Developers' Conference, 1997
+{: .small}
+
+## Tables
+
+| Employee | Salary | |
+| -------- | ------ | ------------------------------------------------------------ |
+| [John Doe](#) | $1 | Because that's all Steve Jobs needed for a salary. |
+| [Jane Doe](#) | $100K | For all the blogging she does. |
+| [Fred Bloggs](#) | $100M | Pictures are worth a thousand words, right? So Jane � 1,000. |
+| [Jane Bloggs](#) | $100B | With hair like that?! Enough said. |
+
+| Header1 | Header2 | Header3 |
+|:--------|:-------:|--------:|
+| cell1 | cell2 | cell3 |
+| cell4 | cell5 | cell6 |
+|-----------------------------|
+| cell1 | cell2 | cell3 |
+| cell4 | cell5 | cell6 |
+|=============================|
+| Foot1 | Foot2 | Foot3 |
+
+## Definition Lists
+
+Definition List Title
+: Definition list division.
+
+Startup
+: A startup company or startup is a company or temporary organization designed to search for a repeatable and scalable business model.
+
+#dowork
+: Coined by Rob Dyrdek and his personal body guard Christopher "Big Black" Boykins, "Do Work" works as a self motivator, to motivating your friends.
+
+Do It Live
+: I'll let Bill O'Reilly [explain](https://www.youtube.com/watch?v=O_HyZ5aW76c "We'll Do It Live") this one.
+
+## Unordered Lists (Nested)
+
+ * List item one
+ * List item one
+ * List item one
+ * List item two
+ * List item three
+ * List item four
+ * List item two
+ * List item three
+ * List item four
+ * List item two
+ * List item three
+ * List item four
+
+## Ordered List (Nested)
+
+ 1. List item one
+ 1. List item one
+ 1. List item one
+ 2. List item two
+ 3. List item three
+ 4. List item four
+ 2. List item two
+ 3. List item three
+ 4. List item four
+ 2. List item two
+ 3. List item three
+ 4. List item four
+
+## Buttons
+
+Make any link standout more when applying the `.btn` class.
+
+```html
+<a href="#" class="btn--success">Success Button</a>
+```
+
+[Primary Button](#){: .btn}
+[Success Button](#){: .btn .btn--success}
+[Warning Button](#){: .btn .btn--warning}
+[Danger Button](#){: .btn .btn--danger}
+[Info Button](#){: .btn .btn--info}
+[Inverse Button](#){: .btn .btn--inverse}
+[Light Outline Button](#){: .btn .btn--light-outline}
+
+```markdown
+[Primary Button Text](#link){: .btn}
+[Success Button Text](#link){: .btn .btn--success}
+[Warning Button Text](#link){: .btn .btn--warning}
+[Danger Button Text](#link){: .btn .btn--danger}
+[Info Button Text](#link){: .btn .btn--info}
+[Inverse Button](#link){: .btn .btn--inverse}
+[Light Outline Button](#link){: .btn .btn--light-outline}
+```
+
+[X-Large Button](#){: .btn .btn--x-large}
+[Large Button](#){: .btn .btn--large}
+[Default Button](#){: .btn}
+[Small Button](#){: .btn .btn--small}
+
+```markdown
+[X-Large Button](#link){: .btn .btn--x-large}
+[Large Button](#link){: .btn .btn--large}
+[Default Button](#link){: .btn}
+[Small Button](#link){: .btn .btn--small}
+```
+
+## Notices
+
+**Watch out!** You can also add notices by appending `{: .notice}` to a paragraph.
+{: .notice}
+
+## HTML Tags
+
+### Address Tag
+
+<address>
+ 1 Infinite Loop<br /> Cupertino, CA 95014<br /> United States
+</address>
+
+### Anchor Tag (aka. Link)
+
+This is an example of a [link](http://apple.com "Apple").
+
+### Abbreviation Tag
+
+The abbreviation CSS stands for "Cascading Style Sheets".
+
+*[CSS]: Cascading Style Sheets
+
+### Cite Tag
+
+"Code is poetry." ---<cite>Automattic</cite>
+
+### Code Tag
+
+You will learn later on in these tests that `word-wrap: break-word;` will be your best friend.
+
+### Strike Tag
+
+This tag will let you <strike>strikeout text</strike>.
+
+### Emphasize Tag
+
+The emphasize tag should _italicize_ text.
+
+### Insert Tag
+
+This tag should denote <ins>inserted</ins> text.
+
+### Keyboard Tag
+
+This scarcely known tag emulates <kbd>keyboard text</kbd>, which is usually styled like the `<code>` tag.
+
+### Preformatted Tag
+
+This tag styles large blocks of code.
+
+<pre>
+.post-title {
+ margin: 0 0 5px;
+ font-weight: bold;
+ font-size: 38px;
+ line-height: 1.2;
+ and here's a line of some really, really, really, really long text, just to see how the PRE tag handles it and to find out how it overflows;
+}
+</pre>
+
+### Quote Tag
+
+<q>Developers, developers, developers…</q> –Steve Ballmer
+
+### Strong Tag
+
+This tag shows **bold text**.
+
+### Subscript Tag
+
+Getting our science styling on with H<sub>2</sub>O, which should push the "2" down.
+
+### Superscript Tag
+
+Still sticking with science and Isaac Newton's E = MC<sup>2</sup>, which should lift the 2 up.
+
+### Variable Tag
+
+This allows you to denote <var>variables</var>.
+
+{% for post in site.pages %}
+{% include archive-single.html %}
+{% endfor %}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/blob/2041fbe8/_pages/category-archive.html
----------------------------------------------------------------------
diff --git a/_pages/category-archive.html b/_pages/category-archive.html
new file mode 100644
index 0000000..c9e03e1
--- /dev/null
+++ b/_pages/category-archive.html
@@ -0,0 +1,16 @@
+---
+layout: archive
+permalink: /categories/
+title: "Posts by Category"
+author_profile: true
+---
+
+{% include group-by-array collection=site.posts field="categories" %}
+
+{% for category in group_names %}
+ {% assign posts = group_items[forloop.index0] %}
+ <h2 id="{{ category | slugify }}" class="archive__subtitle">{{ category }}</h2>
+ {% for post in posts %}
+ {% include archive-single.html %}
+ {% endfor %}
+{% endfor %}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/blob/2041fbe8/_pages/collection-archive.html
----------------------------------------------------------------------
diff --git a/_pages/collection-archive.html b/_pages/collection-archive.html
new file mode 100644
index 0000000..80d2816
--- /dev/null
+++ b/_pages/collection-archive.html
@@ -0,0 +1,23 @@
+---
+layout: archive
+title: "Posts by Collection"
+permalink: /collection-archive/
+author_profile: true
+---
+
+{% capture written_label %}'None'{% endcapture %}
+
+{% for collection in site.collections %}
+ {% unless collection.output == false or collection.label == "posts" %}
+ {% capture label %}{{ collection.label }}{% endcapture %}
+ {% if label != written_label %}
+ <h2 id="{{ label | slugify }}" class="archive__subtitle">{{ label }}</h2>
+ {% capture written_label %}{{ label }}{% endcapture %}
+ {% endif %}
+ {% endunless %}
+ {% for post in collection.docs %}
+ {% unless collection.output == false or collection.label == "posts" %}
+ {% include archive-single.html %}
+ {% endunless %}
+ {% endfor %}
+{% endfor %}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/blob/2041fbe8/_pages/tag-archive.html
----------------------------------------------------------------------
diff --git a/_pages/tag-archive.html b/_pages/tag-archive.html
new file mode 100644
index 0000000..df08680
--- /dev/null
+++ b/_pages/tag-archive.html
@@ -0,0 +1,16 @@
+---
+layout: archive
+permalink: /tags/
+title: "Posts by Tags"
+author_profile: true
+---
+
+{% include group-by-array collection=site.posts field="tags" %}
+
+{% for tag in group_names %}
+ {% assign posts = group_items[forloop.index0] %}
+ <h2 id="{{ tag | slugify }}" class="archive__subtitle">{{ tag }}</h2>
+ {% for post in posts %}
+ {% include archive-single.html %}
+ {% endfor %}
+{% endfor %}
\ No newline at end of file
[3/3] incubator-rocketmq-site git commit: Post a blog-How to Support
More Queues in RocketMQ.
Posted by yu...@apache.org.
Post a blog-How to Support More Queues in RocketMQ.
Project: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/commit/665dc7d5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/tree/665dc7d5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/diff/665dc7d5
Branch: refs/heads/master
Commit: 665dc7d5e839dab55465d3c1474746e4e6311004
Parents: 2041fbe
Author: yukon <yu...@apache.org>
Authored: Fri Dec 23 14:33:03 2016 +0800
Committer: yukon <yu...@apache.org>
Committed: Fri Dec 23 14:33:03 2016 +0800
----------------------------------------------------------------------
...23-how-to-support-more-queues-in-rocketmq.md | 53 +++++++++++++++++++
...mastering-component-compatible-dependency.md | 2 -
assets/images/blog/rocketmq-queues.png | Bin 0 -> 46754 bytes
3 files changed, 53 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/blob/665dc7d5/_posts/2016-12-23-how-to-support-more-queues-in-rocketmq.md
----------------------------------------------------------------------
diff --git a/_posts/2016-12-23-how-to-support-more-queues-in-rocketmq.md b/_posts/2016-12-23-how-to-support-more-queues-in-rocketmq.md
new file mode 100644
index 0000000..c413d30
--- /dev/null
+++ b/_posts/2016-12-23-how-to-support-more-queues-in-rocketmq.md
@@ -0,0 +1,53 @@
+---
+title: "How to Support More Queues in RocketMQ?"
+categories:
+ - RocketMQ
+tags:
+ - RocketMQ
+ - Queue
+ - Partition
+ - Message Oriented Middleware
+---
+
+# Summary
+
+Kafka is a distributed streaming platform, which was born from [logging aggregation cases](https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying). It does not need too high concurrency. In some large scale cases in alibaba, we found that the original model has been unable to meet our actual needs. So, we developed a messaging middleware, named RocketMQ, which can handle a broad set of use cases, ranging from traditional publish/subscribe scenario to demandingly high volume realtime transaction system that tolerates no message loss. Now, in alibaba, RocketMQ clusters process more than 500 billion events every day, provide services for more than 3000 core applications.
+
+{% include toc %}
+
+# Partition design in kafka
+1. Producer parallelism of writing is bounded by the number of partitions.
+2. The degree of consumer consumption parallelism, also is bounded by the number of partitions being consumed. Assuming that the number of partitions is 20, then the maximum concurrent consumption of Consumer is 20.
+3. Each Topic consists of a number of fixed number of partitions. Partition number determines the number of Topic that single Broker can support.
+
+More details please refer to [here](http://www.confluent.io/blog/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/).
+
+## Why Kafka can't support more partitions
+1. Each partition stores the whole message data, although each partition is written to the disk is in order, but a number of sequential partition writing at the same time from the aspect of operating system become a random writing.
+2. Due to the scattered data files, it is difficult to use the linux IO Group Commit mechanism.
+
+# How to support more partition in RocketMQ?
+
+
+
+
+1. All message data are stored in CommmitLog files. Complete sequential writing and random read.
+2. ConsumeQueue stores the actual user consumption location information, they are flushed to disk in sequential mode.
+
+> pros\uff1a
+
+1. A very small amount of data on a single consume queue. Lightweight.
+2. Sequential access in disk, avoid disk lock contention, and not incur high disk iowait when more and more queues created.
+
+> cons\uff1a
+
+1. Message consumption will read ConsumeQueue firstly, then read CommitLog if not found.This process brings a certain cost in the worst cases.
+2. CommitLog and ConsumeQueue must be keep completely consistent, increasing the complexity of programming model.
+
+> Design Motivation\uff1a
+
+1. Random read. Read as much as possible to increase the PAGECACHE hit rate, and reduce read io operations. So bigger memory is still better. If too much message accumulation happened, whether the read performance will fall very badly? the answer is negative, reasons are as follows:
+ - [PAGECACHE prefetch](https://en.wikipedia.org/wiki/Cache_prefetching), even if the 1K message, the system will read more data in advance. You may hit the memory in the next read operation.
+ - Random access CommitLog from disk. If set the I/O scheduler to noop in SSD, the read qps will greatly accelerate than others scheduler algorithm.
+2. Because ConsumeQueue stores little information, mainly associated with consumption locations.also, supports random read. Under PAGECACHE prefetch control, read performance almost keep consistent with the main memory, even if in the message accumulation cases. In this particular case\uff0cConsumeQueue will not hinder the read performance.
+3. CommitLog stores all the meta information, including the message data. similar db's redolog. So as long as CommitLog exists, even if the ConsumeQueue data is lost, data can be recovered.
http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/blob/665dc7d5/_posts/2016-12-23-mastering-component-compatible-dependency.md
----------------------------------------------------------------------
diff --git a/_posts/2016-12-23-mastering-component-compatible-dependency.md b/_posts/2016-12-23-mastering-component-compatible-dependency.md
index c39662a..02c0f5e 100644
--- a/_posts/2016-12-23-mastering-component-compatible-dependency.md
+++ b/_posts/2016-12-23-mastering-component-compatible-dependency.md
@@ -9,8 +9,6 @@ tags:
- Maven
---
-# Preface
-
This article mainly includes three parts.at first,I will introduce compatibility principle(more details see [here](http://blog.csdn.net/fengjia10/article/details/7799227)) briefly.followed by a detailed elaborating about Java component compatible dependency,including the interface-oriented programming,single component signature protection,single component compatibility protection and multi-component compatibility compile time checking.Finally is the review and prospect,especially about **Dependency Mediator** project.
{% include toc %}
http://git-wip-us.apache.org/repos/asf/incubator-rocketmq-site/blob/665dc7d5/assets/images/blog/rocketmq-queues.png
----------------------------------------------------------------------
diff --git a/assets/images/blog/rocketmq-queues.png b/assets/images/blog/rocketmq-queues.png
new file mode 100644
index 0000000..228f313
Binary files /dev/null and b/assets/images/blog/rocketmq-queues.png differ