You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by ko...@apache.org on 2023/04/27 07:01:38 UTC
[arrow] branch main updated: GH-35084: [Docs][Format] Add how to change format specification (#35174)
This is an automated email from the ASF dual-hosted git repository.
kou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new 07d02d6cca GH-35084: [Docs][Format] Add how to change format specification (#35174)
07d02d6cca is described below
commit 07d02d6ccaf2521cba102ac910c4def97cd3610c
Author: Sutou Kouhei <ko...@clear-code.com>
AuthorDate: Thu Apr 27 16:01:30 2023 +0900
GH-35084: [Docs][Format] Add how to change format specification (#35174)
### Rationale for this change
It's for easy to refer.
### What changes are included in this PR?
This is based on the discussion:
[DISCUSS] Format changes: process and requirements
https://lists.apache.org/thread/9t0pglrvxjhrt4r4xcsc1zmgmbtr8pxj
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
* Closes: #35084
Lead-authored-by: Sutou Kouhei <ko...@clear-code.com>
Co-authored-by: Sutou Kouhei <ko...@cozmixng.org>
Co-authored-by: Will Jones <wi...@gmail.com>
Co-authored-by: Weston Pace <we...@gmail.com>
Signed-off-by: Sutou Kouhei <ko...@clear-code.com>
---
docs/source/format/Changing.rst | 89 +++++++++++++++++++++++++++++++++++++++++
docs/source/index.rst | 1 +
2 files changed, 90 insertions(+)
diff --git a/docs/source/format/Changing.rst b/docs/source/format/Changing.rst
new file mode 100644
index 0000000000..cd61af3b66
--- /dev/null
+++ b/docs/source/format/Changing.rst
@@ -0,0 +1,89 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+Changing the Apache Arrow Format Specification
+==============================================
+
+**Cross-language** compatibility is important in Apache Arrow. To
+maintain it, we use the following process when dealing with changes to
+the format (files in
+`<https://github.com/apache/arrow/tree/main/format>`_):
+
+* We must discuss and vote on the changes on the public mailing list
+* We must have at least two reference implementations and associated
+ integration tests
+
+These do not have to be done in order. In most cases, having at least
+one draft reference implementation is helpful for design discussion.
+
+.. note::
+
+ We must update the corresponding documentation (files in
+ `<https://github.com/apache/arrow/tree/main/docs/source/format>`_)
+ too.
+
+Discussion and Voting Process
+-----------------------------
+
+Changes to the format should be discussed on the public mailing list.
+Anyone can join the discussion. The discussion should be started by a
+thread in dev@arrow.apache.org with the ``[DISCUSS]`` prefixed
+subject.
+
+.. note::
+
+ We sometimes use ``[Discuss]``, ``DISCUSS:`` or something similar but
+ ``[DISCUSS]`` is recommended.
+
+Here are some examples:
+
+* `[Discuss][Format] Add 32-bit and 64-bit Decimals <https://lists.apache.org/thread/9ynjmjlxm44j2pt443mcr2hmdl7m43yz>`_
+* `[DISCUSS][Format] Starting to do some concrete work on the new "StringView" columnar data type <https://lists.apache.org/thread/dccj1qrozo88qsxx133kcy308qwfwpfm>`_
+
+The voting process is used to verify we have reached consensus. We can
+start a vote for the format changes after we reach consensus in the
+preceding DISCUSS mailing list thread. Similar to discussion threads,
+voting thread must have the subject prefix ``[VOTE]``.
+
+See also: `Apache Voting Process <https://www.apache.org/foundation/voting.html>`_
+
+At Least Two Reference Implementations
+--------------------------------------
+
+We must have at least two reference implementations and associated
+integration tests to confirm whether the format changes are compatible
+across languages and consistent.
+
+Reference implementations must be within complete Arrow
+implementations. For example, the C++ library is acceptable but the
+Python library is not, since it is a wrapper around the C++
+library. Here are candidate implementations:
+
+* The C++ implementation
+* The Java implementation
+* The Rust (arrow-rs) implementation
+* The Go implementation
+
+We can discuss and vote to add more implementations to the list. We
+may use :doc:`../status` to determine which implementations are
+complete.
+
+Versioning
+----------
+
+The format version (which is separate from the library versions) must
+also be incremented as new changed are made. See :doc:`Versioning`.
diff --git a/docs/source/index.rst b/docs/source/index.rst
index 8341b9f354..1813c201d3 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -82,6 +82,7 @@ target environment.**
format/CStreamInterface
format/ADBC
format/Other
+ format/Changing
format/Glossary
.. _toc.development: