You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@fluo.apache.org by GitBox <gi...@apache.org> on 2018/02/27 19:10:31 UTC
[GitHub] keith-turner closed pull request #130: Updates for 1.2.0 release
keith-turner closed pull request #130: Updates for 1.2.0 release
URL: https://github.com/apache/fluo-website/pull/130
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/_config.yml b/_config.yml
index 0cf13de..2fbd9e8 100644
--- a/_config.yml
+++ b/_config.yml
@@ -54,7 +54,7 @@ defaults:
version: "1.2.0"
minor_release: "1.2"
docs_base: "/docs/fluo/1.2"
- javadoc_base: "https://static.javadoc.io/org.apache.fluo/fluo-api/1.1.0-incubating"
+ javadoc_base: "https://static.javadoc.io/org.apache.fluo/fluo-api/1.2.0"
github_base: "https://github.com/apache/fluo/blob/master"
github_yarn: "https://github.com/apache/fluo-yarn/blob/master"
-
@@ -79,7 +79,7 @@ num_home_posts: 5
color: default
# Fluo specific settings
-latest_fluo_release: "1.1.0-incubating"
+latest_fluo_release: "1.2.0"
latest_recipes_release: "1.1.0-incubating"
# Sets links to external API
diff --git a/_posts/release/2017-09-22-fluo-1.2.0.md b/_posts/release/2018-02-26-fluo-1.2.0.md
similarity index 62%
rename from _posts/release/2017-09-22-fluo-1.2.0.md
rename to _posts/release/2018-02-26-fluo-1.2.0.md
index f99410d..d91e810 100644
--- a/_posts/release/2017-09-22-fluo-1.2.0.md
+++ b/_posts/release/2018-02-26-fluo-1.2.0.md
@@ -1,8 +1,7 @@
---
title: Apache Fluo 1.2.0 released
-date: 2017-09-22 15:00:00 +0000
+date: 2018-02-26 15:00:00 +0000
version: fluo-1.2.0
-draft: true
---
Below are resources for this release:
@@ -20,12 +19,12 @@ package can change at any time. If your project is using Fluo code that falls
outside of the API, then consider [initiating a discussion](/getinvolved/)
about adding it to the API.
-## Significant changes
+## Notable changes
The major changes in 1.2.0 are highlighted here, for the complete list of changes, see the [1.2.0
Milestone] on Github.
-### Fluo now supports multiple ways for running Fluo applications
+### Multiple ways of running applications
Before 1.2.0, Fluo applications could only be run in YARN or as local processes. For 1.2.0, the Fluo tarball
was refactored to support other ways of running Fluo applications such as Docker, Kubernetes & Marathon. This
@@ -38,11 +37,13 @@ refactoring has the following notable changes:
* Code for launching Fluo applications in YARN was moved to its [own project][fluo-yarn] that has its own
[release][fluo-yarn-release] tarball. The Fluo tarball can still be used to launch Fluo applications
in YARN if a `fluo.properties` is created from `fluo.properties.deprecated`.
-* Fluo application jars are now stored in HDFS and application configuration is now stored in Zookeeper. This information used to be stored in a local directory. This change made running Fluo different ways (like YARN or Docker) possible.
+* Fluo application jars are now stored in HDFS and application configuration is now stored in Zookeeper. This information used to be stored in a local directory. This change made running Fluo different ways (like YARN or Docker) possible. The `fluo get-jars` command (added in [db0bdad]) can be used to retrieve application jars.
+* Fluo configuration options can now be specified on the command line with `-o` option. This enables passing Fluo options on the Docker command line.
+* Fluo scripts now support passing Java options. In Docker, its useful to have control over the amount of memory Java uses. This can be done by setting the `FLUO_JAVA_OPTS` env variable on the Docker command line. This was added in commit [4207df4].
Read the [quickstart documentation][quickstart] to learn how to run Fluo applications using these new methods.
-### Fluo now supports read locks.
+### Read locks.
The Percolator paper stated that read locks were expensive and usually not
needed. Therefore in Percolator reads did not acquire a read lock. This
@@ -84,22 +85,44 @@ with `setAlias`. If `addEdge` obtained a write lock, then concurrent calls to
See the [withReadLock javadoc][readlock] for more information.
-### Asynchronous commit code refactored for readability.
-Fluo's commit code is asynchronous in order to support high throughput.
-Before this release the high level commit logic was spread far and wide in the
-code. For this release the commit code was transitioned from Guava's
-ListenableFutre to Java 8's CompletableFuture in [#722]. This transition laid
-the ground work for [#978] which centralized the commit logic. Now the high
-level logic for the commit code is all in one place, making it much easier to
-understand.
+### Simplified Scanner API
+
+Range scanning API was simplified in [c737df6]. Before, scanning over a row was :
+
+```java
+CellScanner scanner = snapshot.scanner().over(Span.exact("row")).build();
+```
+
+After this change, it can be written as :
+
+```java
+CellScanner scanner = snapshot.scanner().over("row").build();
+```
+
+### Streams
-### Addition of Fluo remove command.
+In [2f11089] support for Java 8 streams was added. The following
+computes unique columns in rows with the prefix `abc`.
-Fluo now offers a command to remove an applications data in Zookeeper and
-Accumulo. Work on this issue was done in [#991].
+```java
+Set<Column> cols = snapshot.scanner().overPrefix("abc").build()
+ .stream().map(RowColumnValue::getColumn).collect(Collectors.toSet());
+```
+
+### Remove command.
+
+Can now remove applications data in Zookeeper and
+Accumulo. Work on this issue was done in [#999]. Before
+data had to be manually removed from Zookeeper and Accumulo.
-### Shading libthrift
+### FluoConfiguration Javadoc
+
+Most of the methods in [FluoConfiguration] get or a set a specific property.
+Before it was not easy to know which property. In [cbe44b1] the
+javadoc was updated to mention the properties.
+
+### Shaded libthrift
Fluo uses Apache Thrift for remote procedure calls. Projects using Thrift use
its compiler to generate code. This generated Java code make calls to
@@ -110,16 +133,32 @@ compiler is only guaranteed to work with libthrift 0.9.1.
Accumulo also uses Thrift for its RPCs. When Accumulo and Fluo use different
versions of thrift it can cause serious problems. To avoid these problems,
-in [#995] libthrift was shaded and relocated into the fluo-core jar eliminating Fluo's
+in [082ee8e] libthrift was shaded and relocated into the fluo-core jar eliminating Fluo's
external dependency on libthrift. This means that no matter which version
Accumulo uses, it will not conflict with Fluo's version.
-### Minimum Accumulo version.
+### Asynchronous commit refactored
+
+Fluo's commit code is asynchronous in order to support high throughput.
+Before this release the high level commit logic was spread far and wide in the
+code. For this release the commit code was transitioned from Guava's
+ListenableFutre to Java 8's CompletableFuture in [ca63aaf]. This transition laid
+the ground work for [6bf604f] which centralized the commit logic. Now the high
+level logic for the commit code is all in one place, making it much easier to
+understand.
+
+### Other notable changes
-In [#960] Fluo started using some Accumulo APIs introduced in 1.7.0. Therefore
-Accumulo 1.7.0 is the minimum supported version of Accumulo.
+ * [7c16599] Added a method for efficiently comparing Byte and byte[]
+ * [151c565] Added fluo status command.
+ * [#960] Fluo started using some Accumulo APIs introduced in 1.7.0. Therefore 1.7.0 is now the minimum version of Accumulo.
+ * A few of Fluo's internal caches were made configurable in [650f044], [9e87f47], and [955c86f].
+ * Optimized Fluo's server side code that runs in Accumulo tablet servers in [202fe08] and [51dc912]
+ * Added method to merge Fluo configuration objects in [a349a9d]
## Testing
+
+The Fluo stress test was run twice as documented [here](https://twitter.com/ApacheFluo/status/951112777212530689) and [here](https://lists.apache.org/thread.html/023ebf28a2ebbed88cfbd25f09cc6f4e4a1ba685f22c217ebc238025@%3Cdev.fluo.apache.org%3E).
[procedures]: https://www.apache.org/info/verification
[KEYS]: https://www.apache.org/dist/fluo/KEYS
@@ -139,10 +178,24 @@ Accumulo 1.7.0 is the minimum supported version of Accumulo.
[fluo-yarn-release]: https://fluo.apache.org/release/fluo-yarn-1.0.0
[fluo]: https://github.com/apache/fluo
[quickstart]: https://fluo.apache.org/docs/fluo/1.2/getting-started/quick-start
-[#722]: https://github.com/apache/fluo/issues/722
[842]: https://github.com/apache/fluo/issues/842
[#960]: https://github.com/apache/fluo/issues/960
-[#978]: https://github.com/apache/fluo/issues/978
-[#991]: https://github.com/apache/fluo/issues/991
-[#995]: https://github.com/apache/fluo/issues/995
+[#999]: https://github.com/apache/fluo/issues/991
[readlock]: {{ site.fluo_api_static }}/{{ site.latest_fluo_release }}/org/apache/fluo/api/client/TransactionBase.html#withReadLock--
+[FluoConfiguration]: {{ site.fluo_api_static }}/{{ site.latest_fluo_release }}/org/apache/fluo/api/config/FluoConfiguration.html
+[4207df4]: https://github.com/apache/fluo/commit/4207df4
+[7c16599]: https://github.com/apache/fluo/commit/7c16599
+[151c565]: https://github.com/apache/fluo/commit/151c565
+[db0bdad]: https://github.com/apache/fluo/commit/db0bdad
+[cbe44b1]: https://github.com/apache/fluo/commit/cbe44b1
+[082ee8e]: https://github.com/apache/fluo/commit/082ee8e
+[6bf604f]: https://github.com/apache/fluo/commit/6bf604f
+[ca63aaf]: https://github.com/apache/fluo/commit/ca63aaf
+[c737df6]: https://github.com/apache/fluo/commit/c737df6
+[650f044]: https://github.com/apache/fluo/commit/650f044
+[9e87f47]: https://github.com/apache/fluo/commit/9e87f47
+[955c86f]: https://github.com/apache/fluo/commit/955c86f
+[2f11089]: https://github.com/apache/fluo/commit/2f11089
+[a349a9d]: https://github.com/apache/fluo/commit/a349a9d
+[202fe08]: https://github.com/apache/fluo/commit/202fe08
+[51dc912]: https://github.com/apache/fluo/commit/51dc912
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services