You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by we...@apache.org on 2017/04/18 15:43:18 UTC
arrow git commit: ARROW-844: [Format] Update README documents in
format/
Repository: arrow
Updated Branches:
refs/heads/master 7f20f6e73 -> 38efabea9
ARROW-844: [Format] Update README documents in format/
Added a section reflecting specification maturity and stability.
Author: Wes McKinney <we...@twosigma.com>
Closes #556 from wesm/ARROW-844 and squashes the following commits:
03dbb71 [Wes McKinney] Update README documents in format/
Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/38efabea
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/38efabea
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/38efabea
Branch: refs/heads/master
Commit: 38efabea9bbc8d6386f96a635a95c53ba70e6149
Parents: 7f20f6e
Author: Wes McKinney <we...@twosigma.com>
Authored: Tue Apr 18 11:43:13 2017 -0400
Committer: Wes McKinney <we...@twosigma.com>
Committed: Tue Apr 18 11:43:13 2017 -0400
----------------------------------------------------------------------
format/README.md | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/arrow/blob/38efabea/format/README.md
----------------------------------------------------------------------
diff --git a/format/README.md b/format/README.md
index 048badb..3aa8fdd 100644
--- a/format/README.md
+++ b/format/README.md
@@ -14,16 +14,14 @@
## Arrow specification documents
-> **Work-in-progress specification documents**. These are discussion documents
-> created by the Arrow developers during late 2015 and in no way represents a
-> finalized specification.
-
Currently, the Arrow specification consists of these pieces:
- Metadata specification (see Metadata.md)
- Physical memory layout specification (see Layout.md)
-- Metadata serialized representation (see Message.fbs)
+- Logical Types, Schemas, and Record Batch Metadata (see Schema.fbs)
+- Encapsulated Messages (see Message.fbs)
- Mechanics of messaging between Arrow systems (IPC, RPC, etc.) (see IPC.md)
+- Tensor (Multi-dimensional array) Metadata (see Tensor.fbs)
The metadata currently uses Google's [flatbuffers library][1] for serializing a
couple related pieces of information:
@@ -35,4 +33,16 @@ couple related pieces of information:
schema, and enable a system to send and receive Arrow row batches in a form
that can be precisely disassembled or reconstructed.
+## Arrow Format Maturity and Stability
+
+We have made significant progress hardening the Arrow in-memory format and
+Flatbuffer metadata since the project started in February 2016. We have
+integration tests which verify binary compatibility between the Java and C++
+implementations, for example.
+
+Major versions may still include breaking changes to the memory format or
+metadata, so it is recommended to use the same released version of all
+libraries in your applications for maximum compatibility. Data stored in the
+Arrow IPC formats should not be used for long term storage.
+
[1]: http://github.com/google/flatbuffers