You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by al...@apache.org on 2021/05/26 20:26:44 UTC
[arrow-datafusion] branch master updated: Update more docs and also
the developer.md doc (#414)
This is an automated email from the ASF dual-hosted git repository.
alamb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
The following commit(s) were added to refs/heads/master by this push:
new b382829 Update more docs and also the developer.md doc (#414)
b382829 is described below
commit b38282990a3a3ec3c3c3963e96158f879df0ffe2
Author: Jiayu Liu <Ji...@users.noreply.github.com>
AuthorDate: Thu May 27 04:26:33 2021 +0800
Update more docs and also the developer.md doc (#414)
* update dev.md
* update docs
---
DEVELOPERS.md | 17 +++++++++++++++++
datafusion-examples/examples/README.md | 2 +-
dev/README.md | 14 ++++++++++++--
.../crossbow/tests/fixtures/crossbow-success-message.md | 12 ++++++------
dev/benchmarking/README.md | 9 +++++----
dev/release/VERIFY.md | 10 +++++-----
dev/tasks/conda-recipes/README.md | 2 --
dev/tasks/gandiva-jars/README.md | 2 +-
dev/tasks/linux-packages/README.md | 6 +++---
9 files changed, 50 insertions(+), 24 deletions(-)
diff --git a/DEVELOPERS.md b/DEVELOPERS.md
index 60048c8..c2daf3a 100644
--- a/DEVELOPERS.md
+++ b/DEVELOPERS.md
@@ -93,3 +93,20 @@ can be displayed. For example, the following command creates a
```bash
dot -Tpdf < /tmp/plan.dot > /tmp/plan.pdf
```
+
+## How to format `.md` document
+
+We are using `prettier` to format `.md` files.
+
+You can either use `npm i -g prettier` to install it globally or use `npx` to run it as a standalone binary. Using `npx` required a working node environment. Upgrading to the latest prettier is recommended (by adding `--upgrade` to the `npm` command).
+
+```bash
+$ prettier --version
+2.3.0
+```
+
+After you've confirmed your prettier version, you can format all the `.md` files:
+
+```bash
+prettier -w {ballista,datafusion,datafusion-examples,dev,docs,python}/**/*.md
+```
diff --git a/datafusion-examples/examples/README.md b/datafusion-examples/examples/README.md
index 163ef3d..2b24a22 100644
--- a/datafusion-examples/examples/README.md
+++ b/datafusion-examples/examples/README.md
@@ -25,4 +25,4 @@ The examples `csv_sql.rs` and `parquet_sql.rs` demonstrate building a query plan
## Distributed
-The `flight-client.rs` and `flight-server.rs` examples demonstrate how to run DataFusion as a standalone process and execute SQL queries from a client using the Flight protocol.
\ No newline at end of file
+The `flight-client.rs` and `flight-server.rs` examples demonstrate how to run DataFusion as a standalone process and execute SQL queries from a client using the Flight protocol.
diff --git a/dev/README.md b/dev/README.md
index 258792b..2744087 100644
--- a/dev/README.md
+++ b/dev/README.md
@@ -33,14 +33,16 @@ committer.
## How to merge a Pull request
-Please don't merge PRs using the Github Web interface. Instead, set up
-your git clone such as to have a remote named ``apache`` pointing to the
+Please don't merge PRs using the Github Web interface. Instead, set up
+your git clone such as to have a remote named `apache` pointing to the
official Arrow repository:
+
```
git remote add apache git@github.com:apache/arrow.git
```
and then run the following command:
+
```
./dev/merge_arrow_pr.sh
```
@@ -64,10 +66,13 @@ If these aren't supplied, the script will ask you the values of them.
Note that the directory name of your Arrow git clone must be called `arrow`.
example output:
+
```
Which pull request would you like to merge? (e.g. 34):
```
+
Type the pull request number (from https://github.com/apache/arrow/pulls) and hit enter.
+
```
=== Pull Request #X ===
title Blah Blah Blah
@@ -77,7 +82,9 @@ url https://api.github.com/repos/apache/arrow/pulls/X
Proceed with merging pull request #3? (y/n):
```
+
If this looks good, type y and hit enter.
+
```
From git-wip-us.apache.org:/repos/asf/arrow.git
* [new branch] master -> PR_TOOL_MERGE_PR_3_MASTER
@@ -85,8 +92,10 @@ Switched to branch 'PR_TOOL_MERGE_PR_3_MASTER'
Merge complete (local ref PR_TOOL_MERGE_PR_3_MASTER). Push to apache? (y/n):
```
+
A local branch with the merge has been created.
type y and hit enter to push it to apache master
+
```
Counting objects: 67, done.
Delta compression using up to 4 threads.
@@ -115,6 +124,7 @@ Merge hash: 485658a5
Would you like to pick 485658a5 into another branch? (y/n):
```
+
For now just say n as we have 1 branch
## Verifying Release Candidates
diff --git a/dev/archery/archery/crossbow/tests/fixtures/crossbow-success-message.md b/dev/archery/archery/crossbow/tests/fixtures/crossbow-success-message.md
index 1582521..f914287 100644
--- a/dev/archery/archery/crossbow/tests/fixtures/crossbow-success-message.md
+++ b/dev/archery/archery/crossbow/tests/fixtures/crossbow-success-message.md
@@ -2,9 +2,9 @@ Revision: {revision}
Submitted crossbow builds: [{repo} @ {branch}](https://github.com/{repo}/branches/all?query={branch})
-|Task|Status|
-|----|------|
-|docker-cpp-cmake32|[](https://circleci.com/gh/{repo}/tree/{branch}-circle-docker-cpp-cmake32)|
-|wheel-osx-cp36m|[](https://travis-ci.com/{repo}/branches)|
-|wheel-osx-cp37m|[](https://travis-ci.com/{repo}/branches)|
-|wheel-win-cp36m|[](https://ci.appveyor.com/project/{repo}/history)|
+| Task | Status |
+| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| docker-cpp-cmake32 | [](https://circleci.com/gh/{repo}/tree/{branch}-circle-docker-cpp-cmake32) |
+| wheel-osx-cp36m | [](https://travis-ci.com/{repo}/branches) |
+| wheel-osx-cp37m | [](https://travis-ci.com/{repo}/branches) |
+| wheel-win-cp36m | [](https://ci.appveyor.com/project/{repo}/history) |
diff --git a/dev/benchmarking/README.md b/dev/benchmarking/README.md
index c5ddd62..0c49baf 100644
--- a/dev/benchmarking/README.md
+++ b/dev/benchmarking/README.md
@@ -17,7 +17,6 @@
~ under the License.
-->
-
> NOTE: For those deploying this database, Postgres does not by default use
> UTF-8, however it is [required for the jsonb][pg-jsonb] format used in
> some columns to always work. This [stackoverflow post][so-utf8] describes
@@ -81,7 +80,6 @@ database will be set up automatically once the container is running.
To start the containers, be sure to have [Docker installed][docker],
and then run the following from this directory (arrow/dev/benchmarking).
-
```
docker-compose up
```
@@ -111,9 +109,11 @@ The `psql` shell client is bundled with the PostgreSQL core distribution
available from the [Postgres download page][postgres-downloads].
Using the `PG_USER` defined in the `.env` file (currently "benchmark"),
the command to connect to the container is:
+
```shell
psql -h localhost -p 5432 -U benchmark
```
+
There is an example script in [examples/example.sql](examples/example.sql) that
runs some queries against the database. To run it in the psql client, type
the following in the psql command-line interface:
@@ -168,7 +168,7 @@ The script [graphql_submit.sh](./graphql_submit.sh) simplifies submission
to the database via curl. Examples:
```shell
-./graphql_submit.sh benchmarks examples/benchmark_example.json
+./graphql_submit.sh benchmarks examples/benchmark_example.json
./graphql_submit.sh runs examples/benchmark_run_example.json
```
@@ -176,6 +176,7 @@ to the database via curl. Examples:
The output of the query is a JSON object that is hard to read on the command line.
Here is an example query in the shell:
+
```shell
curl -X POST \
-H "Content-Type: application/json" \
@@ -190,6 +191,7 @@ which (if you have previously run the "examples.sql" command) yields
```
Here is an example query using Python:
+
```python
import json
import requests
@@ -219,7 +221,6 @@ for row in response.json()['data']['allEnvironmentViews']['edges']:
> how to do it for Amazon RDS. This [section of the docs][pg-charset]
> states how to do it in general, i.e.: `initdb -E UTF8`.
-
## Quick reference
- String variables `'have single quotes'`
diff --git a/dev/release/VERIFY.md b/dev/release/VERIFY.md
index 5b441ac..ec77bcc 100644
--- a/dev/release/VERIFY.md
+++ b/dev/release/VERIFY.md
@@ -34,11 +34,11 @@ GLib, Java and JavaScript builds on Linux and macOS. Read the comments in
You need the followings to verify C GLib build:
- * GLib
- * GObject Introspection
- * Ruby (not EOL-ed version is required)
- * gobject-introspection gem
- * test-unit gem
+- GLib
+- GObject Introspection
+- Ruby (not EOL-ed version is required)
+- gobject-introspection gem
+- test-unit gem
You can install them by the followings on Debian GNU/Linux and Ubuntu:
diff --git a/dev/tasks/conda-recipes/README.md b/dev/tasks/conda-recipes/README.md
index 39f82f1..074cefe 100644
--- a/dev/tasks/conda-recipes/README.md
+++ b/dev/tasks/conda-recipes/README.md
@@ -54,14 +54,12 @@ related parts (the cloning of arrow and the jinja templated variables) and
moving the matrix definitions like [this][matrix-definition] to the crossbow
[tasks.yml][../tasks.yml] config file.
-
### Porting recipes from crossbow to the upstream feedstocks
Theoretically these recipes should be up to date with the actual version of
Arrow, so during the release procedure the content of these recipes should be
copied to the upstream feedstocks.
-
[arrow-cpp-feedstock]: https://github.com/conda-forge/arrow-cpp-feedstock
[parquet-cpp-feedstock]: https://github.com/conda-forge/parquet-cpp-feedstock
[matrix-definition]: https://github.com/conda-forge/arrow-cpp-feedstock/blob/master/.azure-pipelines/azure-pipelines-linux.yml#L12
diff --git a/dev/tasks/gandiva-jars/README.md b/dev/tasks/gandiva-jars/README.md
index 2f4c694..5de59a0 100644
--- a/dev/tasks/gandiva-jars/README.md
+++ b/dev/tasks/gandiva-jars/README.md
@@ -26,4 +26,4 @@ Do the following to update arrow manylinux docker image for building Gandiva Jar
- Export JAVA_HOME environment variable.
- Then update build_boost.sh under python/manylinux1/scripts to build boost statically.
-Please look at https://github.com/praveenbingo/arrow/tree/buildGandivaDocker that already has these changes.
\ No newline at end of file
+Please look at https://github.com/praveenbingo/arrow/tree/buildGandivaDocker that already has these changes.
diff --git a/dev/tasks/linux-packages/README.md b/dev/tasks/linux-packages/README.md
index cafcc04..a1a14d1 100644
--- a/dev/tasks/linux-packages/README.md
+++ b/dev/tasks/linux-packages/README.md
@@ -21,9 +21,9 @@
## Requirements
- * Ruby
- * Docker
- * Tools to build tar.gz for Apache Arrow C++ and GLib
+- Ruby
+- Docker
+- Tools to build tar.gz for Apache Arrow C++ and GLib
## How to build .deb packages