You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by we...@apache.org on 2017/09/17 17:46:03 UTC
[1/6] arrow git commit: [Release] Update CHANGELOG.md for 0.7.0
[Forced Update!]
Repository: arrow
Updated Branches:
refs/heads/master 40e72f5fb -> 59b24ba3b (forced update)
[Release] Update CHANGELOG.md for 0.7.0
Change-Id: I29a15bf02595ac5de6615b97597bb3ff3c493f9a
Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/b671dccb
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/b671dccb
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/b671dccb
Branch: refs/heads/master
Commit: b671dccbffd69f4c2177ec469e3cd1369ede2af5
Parents: 576fb66
Author: Wes McKinney <we...@twosigma.com>
Authored: Tue Sep 12 17:58:08 2017 -0400
Committer: Wes McKinney <we...@twosigma.com>
Committed: Tue Sep 12 17:58:08 2017 -0400
----------------------------------------------------------------------
CHANGELOG.md | 142 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 142 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/arrow/blob/b671dccb/CHANGELOG.md
----------------------------------------------------------------------
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8fa4e42..4858df8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -17,6 +17,147 @@
under the License.
-->
+# Apache Arrow 0.7.0 (12 September 2017)
+
+## Bug
+
+* ARROW-1302 - C++: ${MAKE} variable not set sometimes on older MacOS installations
+* ARROW-1354 - [Python] Segfault in Table.from\_pandas with Mixed-Type Categories
+* ARROW-1357 - [Python] Data corruption in reading multi-file parquet dataset
+* ARROW-1363 - [C++] IPC writer sends buffer layout for dictionary rather than indices
+* ARROW-1365 - [Python] Remove usage of removed jemalloc\_memory\_pool in Python API docs
+* ARROW-1373 - [Java] Implement get<type>Buffer() methods at the ValueVector interface
+* ARROW-1375 - [C++] Visual Studio 2017 Appveyor builds failing
+* ARROW-1379 - [Java] maven dependency issues - both unused and undeclared
+* ARROW-1407 - Dictionaries can only hold a maximum of 4096 indices
+* ARROW-1411 - [Python] Booleans in Float Columns cause Segfault
+* ARROW-1414 - [GLib] Cast after status check
+* ARROW-1421 - [Python] pyarrow.serialize cannot serialize a Python dict input
+* ARROW-1426 - [Website] The title element of the top page is empty
+* ARROW-1429 - [Python] Error loading parquet file with \_metadata from HDFS
+* ARROW-1430 - [Python] flake8 warnings are not failing CI builds
+* ARROW-1434 - [C++/Python] pyarrow.Array.from\_pandas does not support datetime64[D] arrays
+* ARROW-1435 - [Python] PyArrow not propagating timezone information from Parquet to Python
+* ARROW-1439 - [Packaging] Automate updating RPM in RPM build
+* ARROW-1443 - [Java] Bug on ArrowBuf.setBytes with unsliced ByteBuffers
+* ARROW-1444 - BitVector.splitAndTransfer copies last byte incorrectly
+* ARROW-1446 - Python: Writing more than 2^31 rows from pandas dataframe causes row count overflow error
+* ARROW-1450 - [Python] Raise proper error if custom serialization handler fails
+* ARROW-1452 - [C++] Make UNUSED macro name more unique so it does not conflict with thirdparty projects
+* ARROW-1453 - [Python] Implement WriteTensor for non-contiguous tensors
+* ARROW-1458 - [Python] Document that HadoopFileSystem.mkdir with create\_parents=False has no effect
+* ARROW-1459 - [Python] PyArrow fails to load partitioned parquet files with non-primitive types
+* ARROW-1461 - [C++] Disable builds using LLVM apt packages temporarily
+* ARROW-1467 - [JAVA]: Fix reset() and allocateNew() in Nullable Value Vectors template
+* ARROW-1490 - [Java] Allow Travis CI failures for JDK9 for now
+* ARROW-1493 - [C++] Flush the output stream at the end of each PrettyPrint function
+* ARROW-1495 - [C++] Store shared\_ptr to boxed arrays in RecordBatch
+* ARROW-1507 - [C++] arrow/compute/api.h can't be used without arrow/array.h
+* ARROW-1512 - [Docs] NumericArray has no member named 'raw\_data'
+* ARROW-1514 - [C++] Fix a typo in document
+* ARROW-1527 - Fix Travis JDK9 build
+* ARROW-1531 - [C++] Return ToBytes by value from Decimal128
+* ARROW-1532 - [Python] Referencing an Empty Schema causes a SegFault
+* ARROW-407 - BitVector.copyFromSafe() should re-allocate if necessary instead of returning false
+* ARROW-801 - [JAVA] Provide direct access to underlying buffer memory addresses in consistent way without generating garbage or large amount indirections
+
+## Improvement
+
+* ARROW-1307 - [Python] Add pandas serialization section + Feather API to Sphinx docs
+* ARROW-1317 - [Python] Add function to set Hadoop CLASSPATH
+* ARROW-1331 - [Java] Refactor tests
+* ARROW-1339 - [C++] Use boost::filesystem for handling of platform-specific file path encodings
+* ARROW-1344 - [C++] Calling BufferOutputStream::Write after calling Finish crashes
+* ARROW-1348 - [C++/Python] Add release verification script for Windows
+* ARROW-1351 - Automate updating CHANGELOG.md as part of release scripts
+* ARROW-1352 - [Integration] Improve print formatting for producer, consumer line
+* ARROW-1355 - Make arrow buildable with java9
+* ARROW-1356 - [Website] Add new committers
+* ARROW-1358 - Update source release scripts to account for new SHA checksum policy
+* ARROW-1359 - [Python] Add Parquet writer option to normalize field names for use in Spark
+* ARROW-1366 - [Python] Add instructions for starting the Plasma store when installing pyarrow from wheels
+* ARROW-1372 - [Plasma] Support for storing data in huge pages
+* ARROW-1376 - [C++] RecordBatchStreamReader::Open API is inconsistent with writer
+* ARROW-1381 - [Python] Improve performance of SerializedPyObject.to\_buffer
+* ARROW-1383 - [C++] Support std::vector<bool> in builder vector appends
+* ARROW-1384 - [C++] Add convenience function for serializing a record batch to an IPC message
+* ARROW-1386 - [C++] Unpin CMake version in MSVC build toolchain
+* ARROW-1395 - [C++] Remove APIs deprecated as of 0.5.0 and later versions
+* ARROW-1397 - [Packaging] Use Docker instead of Vagrant
+* ARROW-1401 - [C++] Add extra debugging context to failures in RETURN\_NOT\_OK in debug builds
+* ARROW-1402 - [C++] Possibly deprecate public APIs that use MutableBuffer
+* ARROW-1404 - [Packaging] Build .deb and .rpm on Travis CI
+* ARROW-1405 - [Python] Add logging option for verbose memory allocations
+* ARROW-1406 - [Python] Harden user API for generating serialized schema and record batch messages as memoryview-compatible objects
+* ARROW-1408 - [C++] Refactor and make IPC read / write APIs more consistent, add appropriate deprecations
+* ARROW-1410 - Plasma object store occasionally pauses for a long time
+* ARROW-1412 - [Plasma] Add higher level API for putting and getting Python objects
+* ARROW-1413 - [C++] Add include-what-you-use configuration
+* ARROW-1416 - [Format] Clarify example array in memory layout documentation
+* ARROW-1418 - [Python] Introduce SerializationContext to register custom serialization callbacks
+* ARROW-1419 - [GLib] Suppress sign-conversion warning on Clang
+* ARROW-1427 - [GLib] Add a link to readme of Arrow GLib
+* ARROW-1428 - [C++] Append steps to clone source code to README.mb
+* ARROW-1432 - [C++] Build bundled jemalloc functions with private prefix
+* ARROW-1433 - [C++] Simplify implementation of Array::Slice
+* ARROW-1438 - [Plasma] Pull SerializationContext through PlasmaClient put and get
+* ARROW-1441 - [Site] Add Ruby to Flexible section
+* ARROW-1442 - [Website] Add pointer to nightly conda packages on /install
+* ARROW-1447 - [C++] Round of include-what-you-use include cleanups
+* ARROW-1448 - [Packaging] Support uploading built .deb and .rpm to Bintray
+* ARROW-1449 - Implement Decimal using only Int128
+* ARROW-1451 - [C++] Create arrow/io/api.h
+* ARROW-1460 - [C++] Upgrade clang-format used to LLVM 4.0
+* ARROW-1466 - [C++] Support DecimalArray in arrow::PrettyPrint
+* ARROW-1468 - [C++] Append to PrimitiveBuilder from std::vector<CTYPE>
+* ARROW-1480 - [Python] Improve performance of serializing sets
+* ARROW-1494 - [C++] Document that shared\_ptr returned by RecordBatch::column needs to be retained
+* ARROW-1499 - [Python] Consider adding option to parquet.write\_table that sets options for maximum Spark compatibility
+* ARROW-1505 - [GLib] Simplify arguments check
+* ARROW-1506 - [C++] Support pkg-config for compute modules
+* ARROW-1508 - C++: Add support for FixedSizeBinaryType in DictionaryBuilder
+* ARROW-1511 - [C++] Deprecate arrow::MakePrimitiveArray
+* ARROW-1513 - C++: Add cast from Dictionary to plain arrays
+* ARROW-1515 - [GLib] Detect version directly
+* ARROW-1516 - [GLib] Update document
+* ARROW-1517 - Remove unnecessary temporary in DecimalUtil::ToString function
+* ARROW-1519 - [C++] Move DecimalUtil functions to methods on the Int128 class
+* ARROW-1528 - [GLib] Resolve include dependency
+* ARROW-1530 - [C++] Install arrow/util/parallel.h
+* ARROW-594 - [Python] Provide interface to write pyarrow.Table to a stream
+* ARROW-786 - [Format] In-memory format for 128-bit Decimals, handling of sign bit
+* ARROW-837 - [Python] Expose buffer allocation, FixedSizeBufferWriter
+* ARROW-941 - [Docs] Improve "cold start" integration testing instructions
+
+## New Feature
+
+* ARROW-1156 - [Python] pyarrow.Array.from\_pandas should take a type parameter
+* ARROW-1238 - [Java] Add JSON read/write support for decimals for integration tests
+* ARROW-1364 - [C++] IPC reader and writer specialized for GPU device memory
+* ARROW-1377 - [Python] Add function to assist with benchmarking Parquet scan performance
+* ARROW-1387 - [C++] Set up GPU leaf library build toolchain
+* ARROW-1392 - [C++] Implement reader and writer IO interfaces for GPU buffers
+* ARROW-1396 - [C++] Add PrettyPrint function for Schemas, which also outputs any dictionaries
+* ARROW-1399 - [C++] Add CUDA build version in a public header to help prevent ABI conflicts
+* ARROW-1400 - [Python] Ability to create partitions when writing to Parquet
+* ARROW-1415 - [GLib] Support date32 and date64
+* ARROW-1417 - [Python] Allow more generic filesystem objects to be passed to ParquetDataset
+* ARROW-1462 - [GLib] Support time array
+* ARROW-1479 - [JS] Expand JavaScript implementation
+* ARROW-1481 - [C++] Expose type casts as generic callable object that can write into pre-allocated memory
+* ARROW-1504 - [GLib] Support timestamp
+* ARROW-1510 - [C++] Support cast
+* ARROW-229 - [C++] Implement safe casts for primitive types
+* ARROW-592 - [C++] Provide .deb and .rpm packages
+* ARROW-695 - Integration tests for Decimal types
+* ARROW-696 - [C++] Add JSON read/write support for decimals for integration tests
+* ARROW-759 - [Python] Implement a transient list serialization function that can handle a mix of scalars, lists, ndarrays, dicts
+* ARROW-989 - [Python] Write pyarrow.Table to FileWriter or StreamWriter
+
+## Test
+
+* ARROW-1390 - [Python] Extend tests for python serialization
+
# Apache Arrow 0.6.0 (14 August 2017)
## Bug
@@ -1102,3 +1243,4 @@
* ARROW-260 - TestValueVector.testFixedVectorReallocation and testVariableVectorReallocation are flaky
* ARROW-83 - Add basic test infrastructure for DecimalType
+
[5/6] arrow git commit: ARROW-559: Add release verification script
for Linux
Posted by we...@apache.org.
ARROW-559: Add release verification script for Linux
Since we're accumulating a bunch of components, I started this script which we can refine to make verifying releases easier for others.
I bootstrapped some pieces off https://github.com/apache/parquet-cpp/blob/master/dev/release/verify-release-candidate, very helpful!
This script:
* Checks GPG signature, checksums
* Installs temporary Python install for the duration of these tests
* Builds/install C++ and runs tests (with Python and Plasma)
* Builds parquet-cpp against the Arrow RC
* Python (with Parquet and Plasma extensions)
* C GLib (requires Ruby in PATH and the gems indicated in README)
* Integration tests
* JavaScript (requires NodeJS >= 6.0.0)
There are some potentially snowflake-y aspects to my environment:
* BOOST_ROOT is set to a Boost install location containing libraries built with `-fPIC`. I'm not sure what to do about this one. One maybe better option is to use system level boost and shared libraries
* Maven 3.3.9 is in PATH
* NodeJS 6.11.3 is in PATH
There are probably some other things that Linux users will run into as they run this script.
I had to compile GLib libraries in this since the ones at system level (Ubuntu 14.04) are too old.
cc @kou @xhochy
Author: Wes McKinney <we...@twosigma.com>
Closes #1102 from wesm/ARROW-559 and squashes the following commits:
8fd6530 [Wes McKinney] Use Boost shared libraries
3531927 [Wes McKinney] Add note to dev/README.md
079b5e4 [Wes McKinney] Fix comments
17f7ac0 [Wes McKinney] More fixes, finally works
adb3146 [Wes McKinney] More work on release verification script
86ef171 [Wes McKinney] Start Linux release verification script
Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/59b24ba3
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/59b24ba3
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/59b24ba3
Branch: refs/heads/master
Commit: 59b24ba3b81bb8ead4fa74ff15ebe633d408415e
Parents: 72ad07e
Author: Wes McKinney <we...@twosigma.com>
Authored: Fri Sep 15 12:29:10 2017 -0400
Committer: Wes McKinney <we...@twosigma.com>
Committed: Sun Sep 17 13:45:20 2017 -0400
----------------------------------------------------------------------
dev/README.md | 18 ++
dev/release/verify-release-candidate.sh | 244 +++++++++++++++++++++++++++
2 files changed, 262 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/arrow/blob/59b24ba3/dev/README.md
----------------------------------------------------------------------
diff --git a/dev/README.md b/dev/README.md
index e986abe..91d688d 100644
--- a/dev/README.md
+++ b/dev/README.md
@@ -92,3 +92,21 @@ Merge hash: 485658a5
Would you like to pick 485658a5 into another branch? (y/n):
```
For now just say n as we have 1 branch
+
+## Verifying Release Candidates
+
+We have provided a script to assist with verifying release candidates:
+
+```shell
+bash dev/release/verify-release-candidate.sh 0.7.0 0
+```
+
+Currently this only works on Linux (patches to expand to macOS welcome!). Read
+the script for information about system dependencies.
+
+On Windows, we have a script that verifies C++ and Python (requires Visual
+Studio 2015):
+
+```
+dev/release/verify-release-candidate.bat apache-arrow-0.7.0.tar.gz
+```
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/arrow/blob/59b24ba3/dev/release/verify-release-candidate.sh
----------------------------------------------------------------------
diff --git a/dev/release/verify-release-candidate.sh b/dev/release/verify-release-candidate.sh
new file mode 100755
index 0000000..38680f4
--- /dev/null
+++ b/dev/release/verify-release-candidate.sh
@@ -0,0 +1,244 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+# Requirements
+# - Ruby 2.x
+# - Plus gem dependencies, see c_glib/README
+# - Maven >= 3.3.9
+# - JDK >=7
+# - gcc >= 4.8
+# - nodejs >= 6.0.0 (best way is to use nvm)
+#
+# If using a non-system Boost, set BOOST_ROOT and add Boost libraries to
+# LD_LIBRARY_PATH
+
+case $# in
+ 2) VERSION="$1"
+ RC_NUMBER="$2"
+ ;;
+
+ *) echo "Usage: $0 X.Y.Z RC_NUMBER"
+ exit 1
+ ;;
+esac
+
+set -ex
+
+HERE=$(cd `dirname "${BASH_SOURCE[0]:-$0}"` && pwd)
+
+ARROW_DIST_URL='https://dist.apache.org/repos/dist/dev/arrow'
+
+download_dist_file() {
+ curl -f -O $ARROW_DIST_URL/$1
+}
+
+download_rc_file() {
+ download_dist_file apache-arrow-${VERSION}-rc${RC_NUMBER}/$1
+}
+
+import_gpg_keys() {
+ download_dist_file KEYS
+ gpg --import KEYS
+}
+
+fetch_archive() {
+ local dist_name=$1
+ download_rc_file ${dist_name}.tar.gz
+ download_rc_file ${dist_name}.tar.gz.asc
+ download_rc_file ${dist_name}.tar.gz.md5
+ download_rc_file ${dist_name}.tar.gz.sha512
+ gpg --verify ${dist_name}.tar.gz.asc ${dist_name}.tar.gz
+ gpg --print-md MD5 ${dist_name}.tar.gz | diff - ${dist_name}.tar.gz.md5
+ sha512sum ${dist_name}.tar.gz | diff - ${dist_name}.tar.gz.sha512
+}
+
+setup_tempdir() {
+ cleanup() {
+ rm -fr "$TMPDIR"
+ }
+ trap cleanup EXIT
+ TMPDIR=$(mktemp -d -t "$1.XXXXX")
+}
+
+
+setup_miniconda() {
+ # Setup short-lived miniconda for Python and integration tests
+ MINICONDA_URL=https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
+
+ MINICONDA=`pwd`/test-miniconda
+
+ wget -O miniconda.sh $MINICONDA_URL
+ bash miniconda.sh -b -p $MINICONDA
+ rm -f miniconda.sh
+
+ export PATH=$MINICONDA/bin:$PATH
+
+ conda create -n arrow-test -y -q python=3.6 \
+ nomkl numpy pandas six cython
+ source activate arrow-test
+}
+
+# Build and test C++
+
+test_and_install_cpp() {
+ mkdir cpp/build
+ pushd cpp/build
+
+ cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
+ -DARROW_PLASMA=on \
+ -DARROW_PYTHON=on \
+ -DARROW_BOOST_USE_SHARED=on \
+ -DCMAKE_BUILD_TYPE=release \
+ -DARROW_BUILD_BENCHMARKS=on \
+ ..
+
+ make -j$NPROC
+ make install
+
+ ctest -L unittest
+ popd
+}
+
+# Build and install Parquet master so we can test the Python bindings
+
+install_parquet_cpp() {
+ git clone git@github.com:apache/parquet-cpp.git
+
+ mkdir parquet-cpp/build
+ pushd parquet-cpp/build
+
+ cmake -DCMAKE_INSTALL_PREFIX=$PARQUET_HOME \
+ -DCMAKE_BUILD_TYPE=release \
+ -DPARQUET_BOOST_USE_SHARED=on \
+ -DPARQUET_BUILD_TESTS=off \
+ ..
+
+ make -j$NPROC
+ make install
+
+ popd
+}
+
+# Build and test Python
+
+test_python() {
+ pushd python
+
+ pip install -r requirements.txt
+
+ python setup.py build_ext --inplace --with-parquet --with-plasma
+ py.test pyarrow -v --pdb
+
+ popd
+}
+
+
+test_glib() {
+ # Build and test GLib, requires GLib >= 2.44 , so install that
+ # here
+ GLIB_VERSION=glib-2.53.7
+ GLIB_URL=https://gensho.ftp.acc.umu.se/pub/gnome/sources/glib/2.53/$GLIB_VERSION.tar.xz
+ curl -f -O $GLIB_URL
+ tar xf $GLIB_VERSION.tar.xz
+ pushd $GLIB_VERSION
+
+ ./configure --disable-libelf --enable-libmount=no --prefix=$ARROW_HOME
+ make -j$NPROC
+ make install
+
+ popd
+
+ pushd c_glib
+
+ ./configure --prefix=$ARROW_HOME
+ make -j$NPROC
+ make install
+
+ NO_MAKE=yes test/run-test.sh
+
+ popd
+}
+
+test_js() {
+ pushd js
+ npm install
+ npm run validate
+ popd
+}
+
+# Build and test Java (Requires newer Maven -- I used 3.3.9)
+
+test_package_java() {
+ pushd java
+
+ mvn test
+ mvn package
+
+ popd
+}
+
+# Run integration tests
+test_integration() {
+ JAVA_DIR=`pwd`/java
+ CPP_BUILD_DIR=`pwd`/cpp/build
+
+ export ARROW_JAVA_INTEGRATION_JAR=$JAVA_DIR/tools/target/arrow-tools-$VERSION-jar-with-dependencies.jar
+ export ARROW_CPP_EXE_PATH=$CPP_BUILD_DIR/release
+
+ pushd integration
+
+ python integration_test.py
+
+ popd
+}
+
+setup_tempdir "arrow-$VERSION"
+echo "Working in sandbox $TMPDIR"
+cd $TMPDIR
+
+export ARROW_HOME=$TMPDIR/install
+export PARQUET_HOME=$TMPDIR/install
+export LD_LIBRARY_PATH=$ARROW_HOME/lib:$LD_LIBRARY_PATH
+export PKG_CONFIG_PATH=$ARROW_HOME/lib/pkgconfig:$PKG_CONFIG_PATH
+
+NPROC=$(nproc)
+VERSION=$1
+RC_NUMBER=$2
+
+TARBALL=apache-arrow-$1.tar.gz
+
+import_gpg_keys
+
+DIST_NAME="apache-arrow-${VERSION}"
+fetch_archive $DIST_NAME
+tar xvzf ${DIST_NAME}.tar.gz
+cd ${DIST_NAME}
+
+setup_miniconda
+test_and_install_cpp
+install_parquet_cpp
+test_python
+test_glib
+test_package_java
+test_integration
+test_js
+
+echo 'Release candidate looks good!'
+exit 0
[4/6] arrow git commit: ARROW-1542: [C++] Install packages in
temporary directory in MSVC build verification script
Posted by we...@apache.org.
ARROW-1542: [C++] Install packages in temporary directory in MSVC build verification script
I found that the script did not work due to the remnants of the last time I ran it.
Author: Wes McKinney <we...@twosigma.com>
Closes #1101 from wesm/ARROW-1542 and squashes the following commits:
0718370 [Wes McKinney] Install packages in temporary directory in MSVC build verification script
Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/72ad07e9
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/72ad07e9
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/72ad07e9
Branch: refs/heads/master
Commit: 72ad07e96648cdc3bc7d2c6f48c732dbe3133080
Parents: 96d451f
Author: Wes McKinney <we...@twosigma.com>
Authored: Fri Sep 15 10:48:22 2017 -0400
Committer: Wes McKinney <we...@twosigma.com>
Committed: Sun Sep 17 13:45:20 2017 -0400
----------------------------------------------------------------------
dev/release/verify-release-candidate.bat | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/arrow/blob/72ad07e9/dev/release/verify-release-candidate.bat
----------------------------------------------------------------------
diff --git a/dev/release/verify-release-candidate.bat b/dev/release/verify-release-candidate.bat
index 503f1fd..27157da 100644
--- a/dev/release/verify-release-candidate.bat
+++ b/dev/release/verify-release-candidate.bat
@@ -45,15 +45,18 @@ tar xvf %1.tar.gz -C "C:/tmp/"
set GENERATOR=Visual Studio 14 2015 Win64
set CONFIGURATION=release
set ARROW_SOURCE=C:\tmp\%1
+set INSTALL_DIR=C:\tmp\%1\install
pushd %ARROW_SOURCE%
call activate arrow-verify-release
set ARROW_BUILD_TOOLCHAIN=%CONDA_PREFIX%\Library
-set ARROW_HOME=%CONDA_PREFIX%\Library
set PARQUET_BUILD_TOOLCHAIN=%CONDA_PREFIX%\Library
-set PARQUET_HOME=%CONDA_PREFIX%\Library
+
+set ARROW_HOME=%INSTALL_DIR%
+set PARQUET_HOME=%INSTALL_DIR%
+set PATH=%INSTALL_DIR%\bin;%PATH%
@rem Build and test Arrow C++ libraries
mkdir cpp\build
[3/6] arrow git commit: ARROW-1537: [C++] Support building with full
path install_name on macOS
Posted by we...@apache.org.
ARROW-1537: [C++] Support building with full path install_name on macOS
If you use `@rpath` for install_name (default), you can use the
DYLD_LIBRARY_PATH environment variable to find libarrow.dylib. But the
DYLD_LIBRARY_PATH environment variable isn't inherited to sub process by
System Integration Protection (SIP). It's difficult to use
libarrow.dylib.
You can use full path install_name by -DARROW_INSTALL_NAME_RPATH=OFF
CMake option. If you use it, you can find libarrow.dylib without
DYLD_LIBRARY_PATH environment variable.
Author: Kouhei Sutou <ko...@clear-code.com>
Closes #1100 from kou/cpp-macos-support-install-name and squashes the following commits:
8207ace [Kouhei Sutou] [C++] Support building with full path install_name on macOS
Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/96d451f3
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/96d451f3
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/96d451f3
Branch: refs/heads/master
Commit: 96d451f37eb23d9c9cdc997032d7534a585537ff
Parents: d0b0b74
Author: Kouhei Sutou <ko...@clear-code.com>
Authored: Fri Sep 15 10:47:04 2017 -0400
Committer: Wes McKinney <we...@twosigma.com>
Committed: Sun Sep 17 13:45:20 2017 -0400
----------------------------------------------------------------------
ci/travis_before_script_c_glib.sh | 6 ------
ci/travis_before_script_cpp.sh | 3 ++-
cpp/CMakeLists.txt | 4 ++++
cpp/cmake_modules/BuildUtils.cmake | 9 +++++++--
4 files changed, 13 insertions(+), 9 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/arrow/blob/96d451f3/ci/travis_before_script_c_glib.sh
----------------------------------------------------------------------
diff --git a/ci/travis_before_script_c_glib.sh b/ci/travis_before_script_c_glib.sh
index 533c892..52bfe87 100755
--- a/ci/travis_before_script_c_glib.sh
+++ b/ci/travis_before_script_c_glib.sh
@@ -77,12 +77,6 @@ pushd $ARROW_C_GLIB_DIR
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$ARROW_CPP_INSTALL/lib/pkgconfig
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ARROW_CPP_INSTALL/lib
-if [ $TRAVIS_OS_NAME == "osx" ]; then
- install_name_tool \
- -id $ARROW_CPP_INSTALL/lib/libarrow.dylib \
- $ARROW_CPP_INSTALL/lib/libarrow.dylib
-fi
-
CONFIGURE_OPTIONS="--prefix=$ARROW_C_GLIB_INSTALL"
if [ $TRAVIS_OS_NAME != "osx" ]; then
CONFIGURE_OPTIONS="$CONFIGURE_OPTIONS --enable-gtk-doc"
http://git-wip-us.apache.org/repos/asf/arrow/blob/96d451f3/ci/travis_before_script_cpp.sh
----------------------------------------------------------------------
diff --git a/ci/travis_before_script_cpp.sh b/ci/travis_before_script_cpp.sh
index a613957..d46fa2f 100755
--- a/ci/travis_before_script_cpp.sh
+++ b/ci/travis_before_script_cpp.sh
@@ -70,7 +70,8 @@ if [ $only_library_mode == "yes" ]; then
CMAKE_COMMON_FLAGS="\
$CMAKE_COMMON_FLAGS \
-DARROW_BUILD_TESTS=OFF \
--DARROW_BUILD_UTILITIES=OFF"
+-DARROW_BUILD_UTILITIES=OFF \
+-DARROW_INSTALL_NAME_RPATH=OFF"
fi
# Use Ninja for faster builds when using toolchain
http://git-wip-us.apache.org/repos/asf/arrow/blob/96d451f3/cpp/CMakeLists.txt
----------------------------------------------------------------------
diff --git a/cpp/CMakeLists.txt b/cpp/CMakeLists.txt
index 577a4bb..972132f 100644
--- a/cpp/CMakeLists.txt
+++ b/cpp/CMakeLists.txt
@@ -150,6 +150,10 @@ if("${CMAKE_SOURCE_DIR}" STREQUAL "${CMAKE_CURRENT_SOURCE_DIR}")
"Build Arrow libraries with RATH set to \$ORIGIN"
OFF)
+ option(ARROW_INSTALL_NAME_RPATH
+ "Build Arrow libraries with install_name set to @rpath"
+ ON)
+
option(ARROW_PLASMA
"Build the plasma object store along with Arrow"
OFF)
http://git-wip-us.apache.org/repos/asf/arrow/blob/96d451f3/cpp/cmake_modules/BuildUtils.cmake
----------------------------------------------------------------------
diff --git a/cpp/cmake_modules/BuildUtils.cmake b/cpp/cmake_modules/BuildUtils.cmake
index 6b2be41..8f92d73 100644
--- a/cpp/cmake_modules/BuildUtils.cmake
+++ b/cpp/cmake_modules/BuildUtils.cmake
@@ -176,10 +176,15 @@ function(ADD_ARROW_LIB LIB_NAME)
endif()
if (APPLE)
- set_target_properties(${LIB_NAME}_shared
+ if (ARROW_INSTALL_NAME_RPATH)
+ set(_lib_install_name "@rpath")
+ else()
+ set(_lib_install_name "${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}")
+ endif()
+ set_target_properties(${LIB_NAME}_shared
PROPERTIES
BUILD_WITH_INSTALL_RPATH ON
- INSTALL_NAME_DIR "@rpath")
+ INSTALL_NAME_DIR "${_lib_install_name}")
endif()
endfunction()
[6/6] arrow git commit: ARROW-1529: [GLib] Use Xcode 8.3 on Travis CI
Posted by we...@apache.org.
ARROW-1529: [GLib] Use Xcode 8.3 on Travis CI
Author: Kouhei Sutou <ko...@clear-code.com>
Closes #1092 from kou/glib-travis-macos and squashes the following commits:
291808b2 [Kouhei Sutou] [GLib] Use Xcode 8.3 on Travis CI
Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/d0b0b741
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/d0b0b741
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/d0b0b741
Branch: refs/heads/master
Commit: d0b0b74181fd94c22996014b9a13e953ed7bbadd
Parents: 97f9029
Author: Kouhei Sutou <ko...@clear-code.com>
Authored: Wed Sep 13 09:09:38 2017 -0400
Committer: Wes McKinney <we...@twosigma.com>
Committed: Sun Sep 17 13:45:20 2017 -0400
----------------------------------------------------------------------
.travis.yml | 2 +-
ci/travis_before_script_c_glib.sh | 13 +++++++++++--
2 files changed, 12 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/arrow/blob/d0b0b741/.travis.yml
----------------------------------------------------------------------
diff --git a/.travis.yml b/.travis.yml
index c4a8f41..f6f6400 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -128,7 +128,7 @@ matrix:
script:
- $TRAVIS_BUILD_DIR/ci/travis_script_c_glib.sh
- compiler: clang
- osx_image: xcode6.4
+ osx_image: xcode8.3
os: osx
cache:
addons:
http://git-wip-us.apache.org/repos/asf/arrow/blob/d0b0b741/ci/travis_before_script_c_glib.sh
----------------------------------------------------------------------
diff --git a/ci/travis_before_script_c_glib.sh b/ci/travis_before_script_c_glib.sh
index 6799ae4..533c892 100755
--- a/ci/travis_before_script_c_glib.sh
+++ b/ci/travis_before_script_c_glib.sh
@@ -23,7 +23,9 @@ source $TRAVIS_BUILD_DIR/ci/travis_env_common.sh
if [ $TRAVIS_OS_NAME == "osx" ]; then
brew install gtk-doc autoconf-archive gobject-introspection
- brew upgrade git cmake wget libtool
+ brew upgrade git cmake
+ brew outdated || brew upgrade wget
+ brew outdated || brew upgrade libtool
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/opt/libffi/lib/pkgconfig
else
@@ -36,6 +38,7 @@ gem install test-unit gobject-introspection
if [ $TRAVIS_OS_NAME == "osx" ]; then
brew install lua
+ sudo env PKG_CONFIG_PATH=$PKG_CONFIG_PATH luarocks install lgi
else
git clone \
--quiet \
@@ -47,8 +50,8 @@ else
echo "yes" | ./install.sh > /dev/null
. ~/torch/install/bin/torch-activate
popd
+ luarocks install lgi
fi
-luarocks install lgi
go get github.com/linuxdeepin/go-gir-generator || :
pushd $GOPATH/src/github.com/linuxdeepin/go-gir-generator
@@ -74,6 +77,12 @@ pushd $ARROW_C_GLIB_DIR
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$ARROW_CPP_INSTALL/lib/pkgconfig
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ARROW_CPP_INSTALL/lib
+if [ $TRAVIS_OS_NAME == "osx" ]; then
+ install_name_tool \
+ -id $ARROW_CPP_INSTALL/lib/libarrow.dylib \
+ $ARROW_CPP_INSTALL/lib/libarrow.dylib
+fi
+
CONFIGURE_OPTIONS="--prefix=$ARROW_C_GLIB_INSTALL"
if [ $TRAVIS_OS_NAME != "osx" ]; then
CONFIGURE_OPTIONS="$CONFIGURE_OPTIONS --enable-gtk-doc"
[2/6] arrow git commit: [maven-release-plugin] prepare release
apache-arrow-0.7.0
Posted by we...@apache.org.
[maven-release-plugin] prepare release apache-arrow-0.7.0
Change-Id: Ib082c383446182630abbb6c8e2e154006ff4560f
Project: http://git-wip-us.apache.org/repos/asf/arrow/repo
Commit: http://git-wip-us.apache.org/repos/asf/arrow/commit/97f9029c
Tree: http://git-wip-us.apache.org/repos/asf/arrow/tree/97f9029c
Diff: http://git-wip-us.apache.org/repos/asf/arrow/diff/97f9029c
Branch: refs/heads/master
Commit: 97f9029ce835dfc2655ca91b9820a2e6aed89107
Parents: b671dcc
Author: Wes McKinney <we...@twosigma.com>
Authored: Tue Sep 12 18:01:14 2017 -0400
Committer: Wes McKinney <we...@twosigma.com>
Committed: Tue Sep 12 18:01:14 2017 -0400
----------------------------------------------------------------------
java/format/pom.xml | 2 +-
java/memory/pom.xml | 2 +-
java/pom.xml | 4 ++--
java/tools/pom.xml | 2 +-
java/vector/pom.xml | 2 +-
5 files changed, 6 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/arrow/blob/97f9029c/java/format/pom.xml
----------------------------------------------------------------------
diff --git a/java/format/pom.xml b/java/format/pom.xml
index 647b716..684ed3c 100644
--- a/java/format/pom.xml
+++ b/java/format/pom.xml
@@ -15,7 +15,7 @@
<parent>
<artifactId>arrow-java-root</artifactId>
<groupId>org.apache.arrow</groupId>
- <version>0.7.0-SNAPSHOT</version>
+ <version>0.7.0</version>
</parent>
<artifactId>arrow-format</artifactId>
http://git-wip-us.apache.org/repos/asf/arrow/blob/97f9029c/java/memory/pom.xml
----------------------------------------------------------------------
diff --git a/java/memory/pom.xml b/java/memory/pom.xml
index 1c4eef7..dcd3572 100644
--- a/java/memory/pom.xml
+++ b/java/memory/pom.xml
@@ -14,7 +14,7 @@
<parent>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-java-root</artifactId>
- <version>0.7.0-SNAPSHOT</version>
+ <version>0.7.0</version>
</parent>
<artifactId>arrow-memory</artifactId>
<name>Arrow Memory</name>
http://git-wip-us.apache.org/repos/asf/arrow/blob/97f9029c/java/pom.xml
----------------------------------------------------------------------
diff --git a/java/pom.xml b/java/pom.xml
index c4c301d..acd5626 100644
--- a/java/pom.xml
+++ b/java/pom.xml
@@ -20,7 +20,7 @@
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-java-root</artifactId>
- <version>0.7.0-SNAPSHOT</version>
+ <version>0.7.0</version>
<packaging>pom</packaging>
<name>Apache Arrow Java Root POM</name>
@@ -44,7 +44,7 @@
<connection>scm:git:https://git-wip-us.apache.org/repos/asf/arrow.git</connection>
<developerConnection>scm:git:https://git-wip-us.apache.org/repos/asf/arrow.git</developerConnection>
<url>https://github.com/apache/arrow</url>
- <tag>HEAD</tag>
+ <tag>apache-arrow-0.7.0</tag>
</scm>
<mailingLists>
http://git-wip-us.apache.org/repos/asf/arrow/blob/97f9029c/java/tools/pom.xml
----------------------------------------------------------------------
diff --git a/java/tools/pom.xml b/java/tools/pom.xml
index 48b507a..a1a6308 100644
--- a/java/tools/pom.xml
+++ b/java/tools/pom.xml
@@ -14,7 +14,7 @@
<parent>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-java-root</artifactId>
- <version>0.7.0-SNAPSHOT</version>
+ <version>0.7.0</version>
</parent>
<artifactId>arrow-tools</artifactId>
<name>Arrow Tools</name>
http://git-wip-us.apache.org/repos/asf/arrow/blob/97f9029c/java/vector/pom.xml
----------------------------------------------------------------------
diff --git a/java/vector/pom.xml b/java/vector/pom.xml
index bf33823..dd1a0ea 100644
--- a/java/vector/pom.xml
+++ b/java/vector/pom.xml
@@ -14,7 +14,7 @@
<parent>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-java-root</artifactId>
- <version>0.7.0-SNAPSHOT</version>
+ <version>0.7.0</version>
</parent>
<artifactId>arrow-vector</artifactId>
<name>Arrow Vectors</name>