You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by ko...@apache.org on 2022/02/06 11:29:14 UTC
[arrow] branch master updated: ARROW-13185: [MATLAB] Create a single MEX gateway function which delegates to specific C++ functions
This is an automated email from the ASF dual-hosted git repository.
kou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/master by this push:
new 9c5e4b3 ARROW-13185: [MATLAB] Create a single MEX gateway function which delegates to specific C++ functions
9c5e4b3 is described below
commit 9c5e4b3e76b0a4de75f0cced731de8a5582c571e
Author: Fiona La <fi...@mathworks.com>
AuthorDate: Sun Feb 6 20:27:22 2022 +0900
ARROW-13185: [MATLAB] Create a single MEX gateway function which delegates to specific C++ functions
## Overview
This pull request implements a more scalable and sustainable approach to organizing the C++ functionality that needs to be exposed to MATLAB. It adds a new singular MEX gateway function, `mexfcn('<cpp-function-name>', <cpp-function-argument-1>, ..., <cpp-function-argument-N>)`, which delegates to specific C++ functions. To make use of `mexfcn`, the directory containing `mexfcn.<mex-extension>` must be added to the MATLAB path.
Advantages to this approach:
- Organization
- All C++ functions that are exposed to MATLAB are registered in one location
- Reduce the complexity of managing and linking many MEX files and ensuring that they are added to the MATLAB path
- Reduce cognitive load for adding new functions
- Avoid polluting the source tree with build artifacts
- Reduce build times, building a single MEX function is faster than building potentially hundreds
- Reduce binary bloat caused by creating a separate MEX file for every MEX function
- Enable flexibility in terms of where C++ implementation files live
## Implementation
1. One MEX function, `mexfcn`, defined in `matlab/src/cpp/arrow/matlab/mex/mex_util.h`, dispatches to individual C++ implementation files.
For example, to invoke featherread functionality from MATLAB, that is implemented in C++:
``` matlab
>> [variables, metadata] = mexfcn('featherread', 'featherfile.feather');
```
2. Functionality implemented in C++ that we want to expose in MATLAB is registered in a function map in the file `matlab/src/cpp/arrow/matlab/mex/mex_functions.h`.
3. Restructured source tree layout and performed general code clean up in preparation for feature implementation work:
3.1. Split source code to matlab/src/matlab and matlab/src/cpp
3.2 Make packages, namespaces, and directories consistent in terms of naming and hierarchy for simplifying navigation and header inclusion.
3.3 Renamed the MATLAB package name `mlarrow` to `arrow` as the `ml` is superfluous.
4. Refactored `matlab/CMakeLists.txt`:
4.1. Build shared library, `arrow_matlab`, that contains C++ functionality for the interface.
4.2. macOS: explicitly add the path of `arrow` to the `rpath` of `arrow_matlab`, as paths of libraries output by imported targets are not automatically included.
4.3. Windows:
- Copy shared library, `arrow.dll` to the directory where the MEX function lives.
- Add the path to MATLAB and GTest shared libraries to the ctest `ENVIRONMENT` when building tests.
- Specify the release version of MSVC Runtime Libraries for all targets created in the CMake file.
5. Enable the `install` target:
5.1. Utilize [`CMAKE_INSTALL_PREFIX`](https://cmake.org/cmake/help/v3.0/variable/CMAKE_INSTALL_PREFIX.html) to allow users to customize the install location. The platform-specific default install locations will be used if the user does not specify a custom value.
5.2. Once installed, the interface's source files and libraries are relocatable, on all platforms.
5.3. As part of the install step, add the path to the install directory to the [MATLAB Search Path](https://uk.mathworks.com/help/matlab/matlab_env/what-is-the-matlab-search-path.html) by:
- Option 1: Call `addpath` and `savepath` to modify the `pathdef.m` file that MATLAB uses on startup. This option is on by default. However, it can only be used if CMake has the appropriate permissions to modify `pathdef.m`. This option is toggled on and off by the flag: `MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH`.
- Option 2: Add an `addpath` command to the `startup.m` file located at the [`userpath`](https://uk.mathworks.com/help/matlab/matlab_env/what-is-the-matlab-search-path.html#:~:text=on%20Search%20Path.-,userpath%20Folder%20on%20the%20Search%20Path,-The%20userpath%20folder). This option can be used if a user does not have the permissions to modify the `pathdef.m` file. This option is off, by default, and is toggled on and off by the flag: `MATLAB_ADD_INSTALL_DIR_TO_STARTUP_FILE`.
- Option 3: Add the path to the install directory to the [`MATLABPATH`](https://uk.mathworks.com/help/matlab/matlab_env/add-folders-to-matlab-search-path-at-startup.html#btpajlw) environment variable. The paths listed in `MATLABPATH` are added to the MATLAB Search Path on start up.
- The MATLAB Actions install of MATLAB used during CI does not have `userpath` set, nor do we have edit permissions for `pathdef.m`. Therefore, we use the third option and set the `MATLABPATH` environment variable in `matlab.yml`.
## Testing
Qualified `CMakeLists.txt` changes by building and running all tests:
- On Windows 10 (Ninja and Visual Studio), macOS 11.5 (Make and Ninja), and Debian 10 (Make and Ninja)
- Configurations: build both Arrow and GTest, use provided `ARROW_HOME`, use provided `GTEST_ROOT`, use both `ARROW_HOME` and `GTEST_ROOT`.
## Future Directions
1. Investigate why the default CMake behavior does not link the test executables against the correct MSVC Runtime libraries (ie. `ucrtbase.dll` versus `ucrtbased.dll`) when building with Ninja on Windows.
2. Add support for specifying function names and arguments as MATLAB strings to `mexfcn`. Currently, only character vectors are supported.
3. Refactor `mexfcn` to use [MATLAB Data Arrays](https://uk.mathworks.com/help/matlab/matlab-data-array.html) (MDAs) and [C++ Mex](https://uk.mathworks.com/help/matlab/cpp-mex-file-applications.html).
4. Investigate running MATLAB tests using [`matlab_add_unit_test`](https://cmake.org/cmake/help/latest/module/FindMatlab.html#command:matlab_add_unit_test).
## Notes
1. Thank you for all of your help on this pull request, @sgilmore10 and @kevingurney!
Closes #12004 from lafiona/ARROW_13185
Lead-authored-by: Fiona La <fi...@mathworks.com>
Co-authored-by: sgilmore <sg...@mathworks.com>
Co-authored-by: lafiona <fi...@gmail.com>
Co-authored-by: Kevin Gurney <kg...@mathworks.com>
Signed-off-by: Sutou Kouhei <ko...@clear-code.com>
---
.github/workflows/matlab.yml | 4 +
ci/scripts/matlab_build.sh | 5 +-
matlab/CMakeLists.txt | 462 ++++++++++++++++-----
.../arrow/matlab/api/visibility.h} | 22 +-
.../arrow/matlab/feather/feather_functions.cc} | 32 +-
.../arrow/matlab/feather/feather_functions.h} | 16 +-
.../arrow/matlab/feather}/feather_reader.cc | 19 +-
.../arrow/matlab/feather}/feather_reader.h | 10 +-
.../arrow/matlab/feather}/feather_writer.cc | 19 +-
.../arrow/matlab/feather}/feather_writer.h | 10 +-
.../{ => cpp/arrow/matlab/feather}/matlab_traits.h | 2 -
.../arrow/matlab/feather}/util/handle_status.cc | 4 +-
.../arrow/matlab/feather}/util/handle_status.h | 5 +-
.../matlab/feather}/util/unicode_conversion.cc | 18 +-
.../matlab/feather}/util/unicode_conversion.h | 8 +-
.../arrow/matlab/mex/mex_functions.h} | 28 +-
matlab/src/cpp/arrow/matlab/mex/mex_util.cc | 53 +++
.../arrow/matlab/mex/mex_util.h} | 24 +-
.../arrow/matlab/mex/mex_util_test.cc} | 16 +-
.../arrow/matlab/mex/mexfcn.cc} | 19 +-
.../+arrow}/+util/createMetadataStruct.m | 0
.../+arrow}/+util/createVariableStruct.m | 0
.../+util/makeValidMATLABTableVariableNames.m | 0
.../+arrow}/+util/table2mlarrow.m | 2 +-
matlab/src/{ => matlab}/featherread.m | 4 +-
matlab/src/{ => matlab}/featherwrite.m | 5 +-
matlab/test/tfeather.m | 15 +-
matlab/test/tfeathermex.m | 25 +-
matlab/test/tmexfcn.m | 61 +++
.../test/util/createVariablesAndMetadataStructs.m | 2 +-
matlab/test/util/featherMEXRoundTrip.m | 4 +-
matlab/tools/UpdateMatlabSearchPath.cmake | 54 +++
matlab/tools/addInstallDirToSearchPath.m | 54 +++
33 files changed, 746 insertions(+), 256 deletions(-)
diff --git a/.github/workflows/matlab.yml b/.github/workflows/matlab.yml
index cbca09d..00d953e 100644
--- a/.github/workflows/matlab.yml
+++ b/.github/workflows/matlab.yml
@@ -61,6 +61,10 @@ jobs:
# errors will occur. To work around this issue, we can explicitly
# force MATLAB to use the system libstdc++.so via LD_PRELOAD.
LD_PRELOAD: /usr/lib/x86_64-linux-gnu/libstdc++.so.6
+
+ # Add the installation directory to the MATLAB Search Path by
+ # setting the MATLABPATH environment variable.
+ MATLABPATH: matlab/install/arrow_matlab
uses: matlab-actions/run-tests@v1
with:
select-by-folder: matlab/test
diff --git a/ci/scripts/matlab_build.sh b/ci/scripts/matlab_build.sh
index 5e9bdd2..6568052 100755
--- a/ci/scripts/matlab_build.sh
+++ b/ci/scripts/matlab_build.sh
@@ -23,7 +23,8 @@ set -ex
base_dir=${1}
source_dir=${base_dir}/matlab
build_dir=${base_dir}/matlab/build
+install_dir=${base_dir}/matlab/install
-cmake -S ${source_dir} -B ${build_dir} -G Ninja -D MATLAB_BUILD_TESTS=ON
-cmake --build ${build_dir} --config Release
+cmake -S ${source_dir} -B ${build_dir} -G Ninja -D MATLAB_BUILD_TESTS=ON -D CMAKE_INSTALL_PREFIX=${install_dir} -D MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH=OFF
+cmake --build ${build_dir} --config Release --target install
ctest --test-dir ${build_dir}
diff --git a/matlab/CMakeLists.txt b/matlab/CMakeLists.txt
index d4da7e8..a331521 100644
--- a/matlab/CMakeLists.txt
+++ b/matlab/CMakeLists.txt
@@ -22,6 +22,7 @@ function(build_arrow)
set(options BUILD_GTEST)
set(one_value_args)
set(multi_value_args)
+
cmake_parse_arguments(ARG
"${options}"
"${one_value_args}"
@@ -31,23 +32,40 @@ function(build_arrow)
message(SEND_ERROR "Error: unrecognized arguments: ${ARG_UNPARSED_ARGUMENTS}")
endif()
+ # If Arrow needs to be built, the default location will be within the build tree.
+ set(ARROW_PREFIX "${CMAKE_CURRENT_BINARY_DIR}/arrow_ep-prefix")
+
if(WIN32)
- set(ARROW_IMPORTED_TYPE IMPORTED_IMPLIB)
- set(ARROW_LIBRARY_SUFFIX ${CMAKE_IMPORT_LIBRARY_SUFFIX})
+ # The shared library is located in the "bin" directory.
+ set(ARROW_SHARED_LIBRARY_DIR "${ARROW_PREFIX}/bin")
+
+ # Imported libraries are used
+ set(ARROW_IMPORT_LIB_FILENAME
+ "${CMAKE_IMPORT_LIBRARY_PREFIX}arrow${CMAKE_IMPORT_LIBRARY_SUFFIX}")
+ set(ARROW_IMPORT_LIB "${ARROW_PREFIX}/lib/${ARROW_IMPORT_LIB_FILENAME}")
else()
- set(ARROW_IMPORTED_TYPE IMPORTED_LOCATION)
- set(ARROW_LIBRARY_SUFFIX ${CMAKE_SHARED_LIBRARY_SUFFIX})
+ # The shared library is located in the "lib" directory.
+ set(ARROW_SHARED_LIBRARY_DIR "${ARROW_PREFIX}/lib")
endif()
- set(ARROW_PREFIX "${CMAKE_CURRENT_BINARY_DIR}/arrow_ep-prefix")
- set(ARROW_INCLUDE_DIR "${ARROW_PREFIX}/include")
- set(ARROW_LIBRARY_DIR "${ARROW_PREFIX}/lib")
- set(ARROW_SHARED_LIB
- "${ARROW_LIBRARY_DIR}/${CMAKE_SHARED_LIBRARY_PREFIX}arrow${ARROW_LIBRARY_SUFFIX}")
+ set(ARROW_SHARED_LIB_FILENAME
+ "${CMAKE_SHARED_LIBRARY_PREFIX}arrow${CMAKE_SHARED_LIBRARY_SUFFIX}")
+ set(ARROW_SHARED_LIB "${ARROW_SHARED_LIBRARY_DIR}/${ARROW_SHARED_LIB_FILENAME}")
+
set(ARROW_BINARY_DIR "${CMAKE_CURRENT_BINARY_DIR}/arrow_ep-build")
set(ARROW_CMAKE_ARGS "-DCMAKE_INSTALL_PREFIX=${ARROW_PREFIX}"
"-DCMAKE_INSTALL_LIBDIR=lib" "-DARROW_BUILD_STATIC=OFF")
- set(ARROW_BUILD_BYPRODUCTS "${ARROW_SHARED_LIB}")
+ set(ARROW_INCLUDE_DIR "${ARROW_PREFIX}/include")
+
+ # The output libraries need to be guaranteed to be available for linking the test
+ # executables.
+ if(WIN32)
+ # On Windows, add the Arrow link library as a BUILD_BYPRODUCTS for arrow_ep.
+ set(ARROW_BUILD_BYPRODUCTS "${ARROW_IMPORT_LIB}")
+ else()
+ # On Linux and macOS, add the Arrow shared library as a BUILD_BYPRODUCTS for arrow_ep.
+ set(ARROW_BUILD_BYPRODUCTS "${ARROW_SHARED_LIB}")
+ endif()
# Building the Arrow C++ libraries and bundled GoogleTest binaries requires ExternalProject.
include(ExternalProject)
@@ -59,14 +77,14 @@ function(build_arrow)
externalproject_add(arrow_ep
SOURCE_DIR "${CMAKE_SOURCE_DIR}/../cpp"
BINARY_DIR "${ARROW_BINARY_DIR}"
- CMAKE_ARGS ${ARROW_CMAKE_ARGS}
- BUILD_BYPRODUCTS ${ARROW_BUILD_BYPRODUCTS})
+ CMAKE_ARGS "${ARROW_CMAKE_ARGS}"
+ BUILD_BYPRODUCTS "${ARROW_BUILD_BYPRODUCTS}")
set(ARROW_LIBRARY_TARGET arrow_shared)
# If find_package has already found a valid Arrow installation, then
# we don't want to link against the newly built arrow_shared library.
- # However, we still need create a library target to trigger building
+ # However, we still need to create a library target to trigger building
# of the arrow_ep target, which will ultimately build the bundled
# GoogleTest binaries.
if(Arrow_FOUND)
@@ -76,108 +94,96 @@ function(build_arrow)
file(MAKE_DIRECTORY "${ARROW_INCLUDE_DIR}")
add_library(${ARROW_LIBRARY_TARGET} SHARED IMPORTED)
set_target_properties(${ARROW_LIBRARY_TARGET}
- PROPERTIES ${ARROW_IMPORTED_TYPE} ${ARROW_SHARED_LIB}
- INTERFACE_INCLUDE_DIRECTORIES ${ARROW_INCLUDE_DIR})
+ PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${ARROW_INCLUDE_DIR}
+ IMPORTED_LOCATION ${ARROW_SHARED_LIB})
+ if(WIN32)
+ # On Windows, IMPORTED_IMPLIB is set to the location of arrow.lib, which is
+ # for linking arrow_matlab against the Arrow C++ library.
+ set_target_properties(${ARROW_LIBRARY_TARGET} PROPERTIES IMPORTED_IMPLIB
+ ${ARROW_IMPORT_LIB})
+ endif()
add_dependencies(${ARROW_LIBRARY_TARGET} arrow_ep)
if(ARG_BUILD_GTEST)
build_gtest()
endif()
-
endfunction()
macro(enable_gtest)
+ set(ARROW_GTEST_PREFIX "${ARROW_BINARY_DIR}/googletest_ep-prefix")
+ set(ARROW_GTEST_MAIN_PREFIX "${ARROW_BINARY_DIR}/googletest_ep-prefix")
+
if(WIN32)
- set(ARROW_GTEST_IMPORTED_TYPE IMPORTED_IMPLIB)
- set(ARROW_GTEST_MAIN_IMPORTED_TYPE IMPORTED_IMPLIB)
+ set(ARROW_GTEST_SHARED_LIB_DIR "${ARROW_GTEST_PREFIX}/bin")
+ set(ARROW_GTEST_MAIN_SHARED_LIB_DIR "${ARROW_GTEST_MAIN_PREFIX}/bin")
- set(ARROW_GTEST_LIBRARY_SUFFIX ${CMAKE_IMPORT_LIBRARY_SUFFIX})
- set(ARROW_GTEST_MAIN_LIBRARY_SUFFIX ${CMAKE_IMPORT_LIBRARY_SUFFIX})
- else()
- set(ARROW_GTEST_IMPORTED_TYPE IMPORTED_LOCATION)
- set(ARROW_GTEST_MAIN_IMPORTED_TYPE IMPORTED_LOCATION)
+ set(ARROW_GTEST_LINK_LIB_DIR "${ARROW_GTEST_PREFIX}/lib")
+ set(ARROW_GTEST_LINK_LIB
+ "${ARROW_GTEST_LINK_LIB_DIR}/${CMAKE_IMPORT_LIBRARY_PREFIX}gtest${CMAKE_IMPORT_LIBRARY_SUFFIX}"
+ )
- set(ARROW_GTEST_LIBRARY_SUFFIX ${CMAKE_SHARED_LIBRARY_SUFFIX})
- set(ARROW_GTEST_MAIN_LIBRARY_SUFFIX ${CMAKE_SHARED_LIBRARY_SUFFIX})
+ set(ARROW_GTEST_MAIN_LINK_LIB_DIR "${ARROW_GTEST_MAIN_PREFIX}/lib")
+ set(ARROW_GTEST_MAIN_LINK_LIB
+ "${ARROW_GTEST_MAIN_LINK_LIB_DIR}/${CMAKE_IMPORT_LIBRARY_PREFIX}gtest_main${CMAKE_IMPORT_LIBRARY_SUFFIX}"
+ )
+ else()
+ set(ARROW_GTEST_SHARED_LIB_DIR "${ARROW_GTEST_PREFIX}/lib")
+ set(ARROW_GTEST_MAIN_SHARED_LIB_DIR "${ARROW_GTEST_MAIN_PREFIX}/lib")
endif()
- set(ARROW_GTEST_PREFIX "${ARROW_BINARY_DIR}/googletest_ep-prefix")
set(ARROW_GTEST_INCLUDE_DIR "${ARROW_GTEST_PREFIX}/include")
- set(ARROW_GTEST_LIBRARY_DIR "${ARROW_GTEST_PREFIX}/lib")
set(ARROW_GTEST_SHARED_LIB
- "${ARROW_GTEST_LIBRARY_DIR}/${CMAKE_SHARED_LIBRARY_PREFIX}gtest${ARROW_GTEST_LIBRARY_SUFFIX}"
+ "${ARROW_GTEST_SHARED_LIB_DIR}/${CMAKE_SHARED_LIBRARY_PREFIX}gtest${CMAKE_SHARED_LIBRARY_SUFFIX}"
)
- set(ARROW_GTEST_MAIN_PREFIX "${ARROW_BINARY_DIR}/googletest_ep-prefix")
set(ARROW_GTEST_MAIN_INCLUDE_DIR "${ARROW_GTEST_MAIN_PREFIX}/include")
- set(ARROW_GTEST_MAIN_LIBRARY_DIR "${ARROW_GTEST_MAIN_PREFIX}/lib")
set(ARROW_GTEST_MAIN_SHARED_LIB
- "${ARROW_GTEST_MAIN_LIBRARY_DIR}/${CMAKE_SHARED_LIBRARY_PREFIX}gtest_main${ARROW_GTEST_MAIN_LIBRARY_SUFFIX}"
+ "${ARROW_GTEST_MAIN_SHARED_LIB_DIR}/${CMAKE_SHARED_LIBRARY_PREFIX}gtest_main${CMAKE_SHARED_LIBRARY_SUFFIX}"
)
list(APPEND ARROW_CMAKE_ARGS "-DARROW_BUILD_TESTS=ON")
- list(APPEND ARROW_BUILD_BYPRODUCTS "${ARROW_GTEST_SHARED_LIB}"
- "${ARROW_GTEST_MAIN_SHARED_LIB}")
+
+ # The appropriate libraries need to be guaranteed to be available when linking the test
+ # executables.
+ if(WIN32)
+ # On Windows, add the gtest link libraries as BUILD_BYPRODUCTS for arrow_ep.
+ list(APPEND ARROW_BUILD_BYPRODUCTS "${ARROW_GTEST_LINK_LIB}"
+ "${ARROW_GTEST_MAIN_LINK_LIB}")
+ else()
+ # On Linux and macOS, add the gtest shared libraries as BUILD_BYPRODUCTS for arrow_ep.
+ list(APPEND ARROW_BUILD_BYPRODUCTS "${ARROW_GTEST_SHARED_LIB}"
+ "${ARROW_GTEST_MAIN_SHARED_LIB}")
+ endif()
endmacro()
# Build the GoogleTest binaries that are bundled with the Arrow C++ libraries.
macro(build_gtest)
- set(ARROW_GTEST_INCLUDE_DIR "${ARROW_GTEST_PREFIX}/include")
- set(ARROW_GTEST_MAIN_INCLUDE_DIR "${ARROW_GTEST_MAIN_PREFIX}/include")
-
file(MAKE_DIRECTORY "${ARROW_GTEST_INCLUDE_DIR}")
- if(WIN32)
- set(ARROW_GTEST_RUNTIME_DIR "${ARROW_GTEST_PREFIX}/bin")
- set(ARROW_GTEST_MAIN_RUNTIME_DIR "${ARROW_GTEST_MAIN_PREFIX}/bin")
- set(ARROW_GTEST_RUNTIME_SUFFIX "${CMAKE_SHARED_LIBRARY_SUFFIX}")
- set(ARROW_GTEST_MAIN_RUNTIME_SUFFIX "${CMAKE_SHARED_LIBRARY_SUFFIX}")
- set(ARROW_GTEST_RUNTIME_LIB
- "${ARROW_GTEST_RUNTIME_DIR}/${CMAKE_SHARED_LIBRARY_PREFIX}gtest${ARROW_GTEST_RUNTIME_SUFFIX}"
- )
- set(ARROW_GTEST_MAIN_RUNTIME_LIB
- "${ARROW_GTEST_MAIN_RUNTIME_DIR}/${CMAKE_SHARED_LIBRARY_PREFIX}gtest_main${ARROW_GTEST_MAIN_RUNTIME_SUFFIX}"
- )
-
- # Multi-Configuration generators (e.g. Visual Studio or XCode) place their build artifacts
- # in a subdirectory named ${CMAKE_BUILD_TYPE} by default, where ${CMAKE_BUILD_TYPE} varies
- # depending on the chosen build configuration (e.g. Release or Debug).
- get_property(GENERATOR_IS_MULTI_CONFIG_VALUE GLOBAL
- PROPERTY GENERATOR_IS_MULTI_CONFIG)
- if(GENERATOR_IS_MULTI_CONFIG_VALUE)
- set(MATLAB_TESTS_DIR "${CMAKE_BINARY_DIR}/$<CONFIG>")
- else()
- set(MATLAB_TESTS_DIR "${CMAKE_BINARY_DIR}")
- endif()
-
- # We need to copy the gtest and gtest_main runtime DLLs into the directory where the
- # MATLAB C++ tests reside, since Windows requires that runtime DLLs are in the same
- # directory as the executables that depend on them (or on the %PATH%).
- externalproject_add_step(arrow_ep copy
- COMMAND ${CMAKE_COMMAND} -E make_directory
- ${MATLAB_TESTS_DIR}
- COMMAND ${CMAKE_COMMAND} -E copy ${ARROW_GTEST_RUNTIME_LIB}
- ${MATLAB_TESTS_DIR}
- COMMAND ${CMAKE_COMMAND} -E copy
- ${ARROW_GTEST_MAIN_RUNTIME_LIB} ${MATLAB_TESTS_DIR}
- DEPENDEES install)
- endif()
-
+ # Create target GTest::gtest
add_library(GTest::gtest SHARED IMPORTED)
set_target_properties(GTest::gtest
- PROPERTIES ${ARROW_GTEST_IMPORTED_TYPE} ${ARROW_GTEST_SHARED_LIB}
+ PROPERTIES IMPORTED_LOCATION ${ARROW_GTEST_SHARED_LIB}
INTERFACE_INCLUDE_DIRECTORIES
${ARROW_GTEST_INCLUDE_DIR})
+ if(WIN32)
+ set_target_properties(GTest::gtest PROPERTIES IMPORTED_IMPLIB ${ARROW_GTEST_LINK_LIB})
+ endif()
+ add_dependencies(GTest::gtest arrow_ep)
+
+ # Create target GTest::gtest_main
add_library(GTest::gtest_main SHARED IMPORTED)
set_target_properties(GTest::gtest_main
- PROPERTIES ${ARROW_GTEST_MAIN_IMPORTED_TYPE}
- ${ARROW_GTEST_MAIN_SHARED_LIB}
+ PROPERTIES IMPORTED_LOCATION ${ARROW_GTEST_MAIN_SHARED_LIB}
INTERFACE_INCLUDE_DIRECTORIES
${ARROW_GTEST_MAIN_INCLUDE_DIR})
+ if(WIN32)
+ set_target_properties(GTest::gtest_main PROPERTIES IMPORTED_IMPLIB
+ ${ARROW_GTEST_MAIN_LINK_LIB})
+ endif()
- add_dependencies(GTest::gtest arrow_ep)
add_dependencies(GTest::gtest_main arrow_ep)
endmacro()
@@ -188,9 +194,15 @@ string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" MLARROW_BASE_VERSION "${MLARROW_V
project(mlarrow VERSION "${MLARROW_BASE_VERSION}")
+# On Windows, set the global variable that determines the MSVC runtime library that is
+# used by targets created in this CMakeLists.txt file.
+if(WIN32)
+ set(CMAKE_MSVC_RUNTIME_LIBRARY "MultiThreadedDLL")
+endif()
+
option(MATLAB_BUILD_TESTS "Build the C++ tests for the MATLAB interface" OFF)
-# Grab CMAKE Modules from the CPP interface
+# Grab CMAKE Modules from the CPP interface.
set(CPP_CMAKE_MODULES "${CMAKE_SOURCE_DIR}/../cpp/cmake_modules")
if(EXISTS "${CPP_CMAKE_MODULES}")
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CPP_CMAKE_MODULES})
@@ -198,10 +210,21 @@ endif()
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_SOURCE_DIR}/cmake_modules)
+# Multi-Configuration generators (e.g. Visual Studio or XCode) place their build artifacts
+# in a subdirectory named ${CMAKE_BUILD_TYPE} by default, where ${CMAKE_BUILD_TYPE} varies
+# depending on the chosen build configuration (e.g. Release or Debug).
+get_property(GENERATOR_IS_MULTI_CONFIG_VALUE GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG)
+if(GENERATOR_IS_MULTI_CONFIG_VALUE)
+ set(MATLAB_BUILD_OUTPUT_DIR "${CMAKE_BINARY_DIR}/$<CONFIG>")
+else()
+ set(MATLAB_BUILD_OUTPUT_DIR "${CMAKE_BINARY_DIR}")
+endif()
+
# Only build the MATLAB interface C++ tests if MATLAB_BUILD_TESTS=ON.
if(MATLAB_BUILD_TESTS)
# find_package(GTest) supports custom GTEST_ROOT as well as package managers.
find_package(GTest)
+
if(NOT GTest_FOUND)
# find_package(Arrow) supports custom ARROW_HOME as well as package
# managers.
@@ -212,12 +235,36 @@ if(MATLAB_BUILD_TESTS)
# C++ libraries that are built from source.
build_arrow(BUILD_GTEST)
else()
+ # On Windows, IMPORTED_LOCATION needs to be set to indicate where the shared
+ # libraries live when GTest is found.
+ if(WIN32)
+ set(GTEST_SHARED_LIB_DIR "${GTEST_ROOT}/bin")
+ set(GTEST_SHARED_LIBRARY_FILENAME
+ "${CMAKE_SHARED_LIBRARY_PREFIX}gtest${CMAKE_SHARED_LIBRARY_SUFFIX}")
+ set(GTEST_SHARED_LIBRARY_LIB
+ "${GTEST_SHARED_LIB_DIR}/${GTEST_SHARED_LIBRARY_FILENAME}")
+
+ set(GTEST_MAIN_SHARED_LIB_DIR "${GTEST_ROOT}/bin")
+ set(GTEST_MAIN_SHARED_LIBRARY_FILENAME
+ "${CMAKE_SHARED_LIBRARY_PREFIX}gtest_main${CMAKE_SHARED_LIBRARY_SUFFIX}")
+ set(GTEST_MAIN_SHARED_LIBRARY_LIB
+ "${GTEST_MAIN_SHARED_LIB_DIR}/${GTEST_MAIN_SHARED_LIBRARY_FILENAME}")
+
+ set_target_properties(GTest::gtest PROPERTIES IMPORTED_LOCATION
+ "${GTEST_SHARED_LIBRARY_LIB}")
+
+ set_target_properties(GTest::gtest_main
+ PROPERTIES IMPORTED_LOCATION
+ "${GTEST_MAIN_SHARED_LIBRARY_LIB}")
+ endif()
+
find_package(Arrow)
if(NOT Arrow_FOUND)
# Trigger an automatic build of the Arrow C++ libraries.
build_arrow()
endif()
endif()
+
else()
find_package(Arrow)
if(NOT Arrow_FOUND)
@@ -226,42 +273,48 @@ else()
endif()
# MATLAB is Required
-find_package(Matlab REQUIRED)
+find_package(Matlab REQUIRED COMPONENTS MAIN_PROGRAM)
-# Construct the absolute path to featherread's source files
-set(featherread_sources featherreadmex.cc feather_reader.cc util/handle_status.cc
- util/unicode_conversion.cc)
-list(TRANSFORM featherread_sources PREPEND ${CMAKE_SOURCE_DIR}/src/)
+message(STATUS "Mex Library: ${Matlab_MEX_LIBRARY}")
+message(STATUS "Mex Include Folder: ${Matlab_INCLUDE_DIRS}")
-# Build featherreadmex MEX binary
-matlab_add_mex(R2018a
- NAME featherreadmex
- SRC ${featherread_sources}
- LINK_TO arrow_shared)
+set(CPP_SOURCE_DIR ${CMAKE_SOURCE_DIR}/src/cpp)
+set(MATLAB_SOURCE_DIR ${CMAKE_SOURCE_DIR}/src/cpp/arrow/matlab)
+
+set(arrow_matlab_sources
+ mex/mex_util.cc
+ feather/feather_reader.cc
+ feather/feather_writer.cc
+ feather/feather_functions.cc
+ feather/util/handle_status.cc
+ feather/util/unicode_conversion.cc)
+list(TRANSFORM arrow_matlab_sources PREPEND ${CPP_SOURCE_DIR}/arrow/matlab/)
+
+add_library(arrow_matlab SHARED ${arrow_matlab_sources})
+
+# Declare a dependency on arrow_shared (libarrow.so/dylib/dll).
+target_link_libraries(arrow_matlab arrow_shared)
+
+# Declare a dependency on the MEX shared library (libmex.so/dylib/dll).
+target_link_libraries(arrow_matlab ${Matlab_MX_LIBRARY})
+target_link_libraries(arrow_matlab ${Matlab_MEX_LIBRARY})
-# Construct the absolute path to featherwrite's source files
-set(featherwrite_sources featherwritemex.cc feather_writer.cc util/handle_status.cc
- util/unicode_conversion.cc)
-list(TRANSFORM featherwrite_sources PREPEND ${CMAKE_SOURCE_DIR}/src/)
+# Include the MATLAB MEX headers.
+target_include_directories(arrow_matlab PRIVATE ${Matlab_INCLUDE_DIRS})
+target_include_directories(arrow_matlab PRIVATE ${CPP_SOURCE_DIR})
+target_include_directories(arrow_matlab PRIVATE ${ARROW_INCLUDE_DIR})
+target_compile_definitions(arrow_matlab PRIVATE ARROW_MATLAB_EXPORTING)
-# Build featherwritemex MEX binary
+set(mexfcn_sources mex/mexfcn.cc)
+list(TRANSFORM mexfcn_sources PREPEND ${CPP_SOURCE_DIR}/arrow/matlab/)
+
+# Build mexfcn MEX binary.
matlab_add_mex(R2018a
- NAME featherwritemex
- SRC ${featherwrite_sources}
- LINK_TO arrow_shared)
+ NAME mexfcn
+ SRC ${mexfcn_sources}
+ LINK_TO arrow_matlab)
-# Ensure the MEX binaries are placed in the src directory on all platforms
-if(WIN32)
- set_target_properties(featherreadmex PROPERTIES RUNTIME_OUTPUT_DIRECTORY
- $<1:${CMAKE_SOURCE_DIR}/src>)
- set_target_properties(featherwritemex PROPERTIES RUNTIME_OUTPUT_DIRECTORY
- $<1:${CMAKE_SOURCE_DIR}/src>)
-else()
- set_target_properties(featherreadmex PROPERTIES LIBRARY_OUTPUT_DIRECTORY
- $<1:${CMAKE_SOURCE_DIR}/src>)
- set_target_properties(featherwritemex PROPERTIES LIBRARY_OUTPUT_DIRECTORY
- $<1:${CMAKE_SOURCE_DIR}/src>)
-endif()
+target_include_directories(mexfcn PRIVATE ${CPP_SOURCE_DIR})
# ##############################################################################
# C++ Tests
@@ -273,10 +326,203 @@ if(MATLAB_BUILD_TESTS)
# Define a test executable target. TODO: Remove the placeholder test. This is
# just for testing GoogleTest integration.
add_executable(placeholder_test ${CMAKE_SOURCE_DIR}/src/placeholder_test.cc)
+ add_executable(mex_util_test ${CPP_SOURCE_DIR}/arrow/matlab/mex/mex_util_test.cc)
+
# Declare a dependency on the GTest::gtest and GTest::gtest_main IMPORTED
# targets.
target_link_libraries(placeholder_test GTest::gtest GTest::gtest_main)
- # Add a test target.
+ # Declare a dependency on the GTest::gtest and GTest::gtest_main IMPORTED
+ # targets.
+ target_link_libraries(mex_util_test GTest::gtest GTest::gtest_main)
+ target_link_libraries(mex_util_test arrow_matlab)
+
+ # Include the MATLAB MEX headers.
+ target_include_directories(mex_util_test PRIVATE ${Matlab_INCLUDE_DIRS})
+ # Include the C++ source headers.
+ target_include_directories(mex_util_test PRIVATE ${CPP_SOURCE_DIR})
+
+ # Add test targets for C++ tests.
add_test(PlaceholderTestTarget placeholder_test)
+ add_test(CheckNumArgsTestTarget mex_util_test)
+
+ # On macOS, add the directory of libarrow.dylib to the $DYLD_LIBRARY_PATH for
+ # running CheckNumArgsTestTarget.
+ if(APPLE)
+ get_target_property(ARROW_SHARED_LIB arrow_shared IMPORTED_LOCATION)
+ get_filename_component(ARROW_SHARED_LIB_DIR ${ARROW_SHARED_LIB} DIRECTORY)
+
+ set_tests_properties(CheckNumArgsTestTarget
+ PROPERTIES ENVIRONMENT
+ "DYLD_LIBRARY_PATH=${ARROW_SHARED_LIB_DIR}")
+ endif()
+
+ # On Windows:
+ # Add the directory of gtest.dll and gtest_main.dll to the %PATH% for running
+ # all tests.
+ # Add the directory of libmx.dll, libmex.dll, and libarrow.dll to the %PATH% for running
+ # CheckNumArgsTestTarget.
+ # Note: When appending to the path using set_test_properties' ENVIRONMENT property,
+ # make sure that we escape ';' to prevent CMake from interpreting the input as
+ # a list of strings.
+ if(WIN32)
+ get_target_property(GTEST_SHARED_LIB GTest::gtest IMPORTED_LOCATION)
+ get_filename_component(GTEST_SHARED_LIB_DIR ${GTEST_SHARED_LIB} DIRECTORY)
+
+ get_target_property(GTEST_MAIN_SHARED_LIB GTest::gtest_main IMPORTED_LOCATION)
+ get_filename_component(GTEST_MAIN_SHARED_LIB_DIR ${GTEST_MAIN_SHARED_LIB} DIRECTORY)
+
+ set_tests_properties(PlaceholderTestTarget
+ PROPERTIES ENVIRONMENT
+ "PATH=${GTEST_SHARED_LIB_DIR}\;${GTEST_MAIN_SHARED_LIB_DIR}\;$ENV{PATH}"
+ )
+
+ get_target_property(ARROW_SHARED_LIB arrow_shared IMPORTED_LOCATION)
+ get_filename_component(ARROW_SHARED_LIB_DIR ${ARROW_SHARED_LIB} DIRECTORY)
+
+ set(MATLAB_DLL_DEPENDENCIES_DIR "${Matlab_ROOT_DIR}/bin/win64")
+
+ set_tests_properties(CheckNumArgsTestTarget
+ PROPERTIES ENVIRONMENT
+ "PATH=${ARROW_SHARED_LIB_DIR}\;${MATLAB_DLL_DEPENDENCIES_DIR}\;${GTEST_SHARED_LIB_DIR}\;${GTEST_MAIN_SHARED_LIB_DIR}\;$ENV{PATH}"
+ )
+ endif()
+endif()
+
+# ##############################################################################
+# Install
+# ##############################################################################
+# Create a subdirectory at CMAKE_INSTALL_PREFIX to install the interface.
+set(CMAKE_INSTALL_DIR "${CMAKE_INSTALL_PREFIX}/arrow_matlab")
+
+# Install MATLAB source files.
+# On macOS, exclude '.DS_Store' files in the source tree from installation.
+install(DIRECTORY "${CMAKE_SOURCE_DIR}/src/matlab/"
+ DESTINATION ${CMAKE_INSTALL_DIR}
+ PATTERN ".DS_Store" EXCLUDE)
+
+# Install arrow_matlab and mexfcn.
+# Use the RUNTIME output artifact keyword for Windows.
+# Use the LIBRARY output artifact keyword for macOS and Linux.
+install(TARGETS arrow_matlab mexfcn
+ RUNTIME DESTINATION ${CMAKE_INSTALL_DIR}
+ LIBRARY DESTINATION ${CMAKE_INSTALL_DIR})
+
+get_target_property(ARROW_SHARED_LIB arrow_shared IMPORTED_LOCATION)
+get_filename_component(ARROW_SHARED_LIB_DIR ${ARROW_SHARED_LIB} DIRECTORY)
+get_filename_component(ARROW_SHARED_LIB_FILENAME ${ARROW_SHARED_LIB} NAME_WE)
+
+if(WIN32)
+ # On Windows, arrow.dll must be installed to to CMAKE_INSTALL_DIR regardless of whether
+ # Arrow_FOUND is true or false.
+ install(FILES ${ARROW_SHARED_LIB} DESTINATION "${CMAKE_INSTALL_DIR}")
+endif()
+
+# On macOS, use the RPATH values below for runtime dependency resolution. This enables
+# relocation of the installation directory.
+if(APPLE)
+ # Setting INSTALL_RPATH_USE_LINK_PATH to true will add the paths to external dependencies
+ # to the RPATH of arrow_matlab and mexfcn, including the MATLAB dependencies.
+ # If Arrow_FOUND is true, this also includes the path to arrow_shared.
+ set_target_properties(arrow_matlab mexfcn PROPERTIES INSTALL_RPATH_USE_LINK_PATH TRUE)
+
+ # Add @loader_path to the RPATH of mexfcn so that libarrow_matlab.dylib can be found
+ # at runtime.
+ set_target_properties(mexfcn PROPERTIES INSTALL_RPATH "@loader_path")
+
+ if(NOT Arrow_FOUND)
+ # If Arrow_FOUND is false, Arrow is built by the arrow_shared target and needs
+ # to be copied to CMAKE_INSTALL_DIR. The DIRECTORY install command is used to
+ # install libarrow.dylib (symlink) and the real files it points to.
+ #
+ # The subfolders cmake and pkgconfig are excluded as they will be empty.
+ # Note: The following CMake Issue suggests enabling an option to exclude all
+ # folders that would be empty after installation:
+ # https://gitlab.kitware.com/cmake/cmake/-/issues/17122
+ install(DIRECTORY "${ARROW_SHARED_LIB_DIR}/"
+ DESTINATION ${CMAKE_INSTALL_DIR}
+ FILES_MATCHING
+ REGEX "${ARROW_SHARED_LIB_FILENAME}\\..*dylib"
+ PATTERN "cmake" EXCLUDE
+ PATTERN "pkgconfig" EXCLUDE)
+
+ # Add @loader_path to the RPATH of arrow_matlab so that libarrow.dylib can be found
+ # at runtime.
+ set_target_properties(arrow_matlab PROPERTIES INSTALL_RPATH "@loader_path")
+ endif()
+endif()
+
+# On Linux, use the RUNPATH values below for runtime dependency resolution. This enables
+# relocation of the installation directory.
+if(UNIX
+ AND NOT APPLE
+ AND NOT CYGWIN)
+ # Setting INSTALL_RPATH_USE_LINK_PATH to true will add the paths to external dependencies
+ # to the RUNPATH of arrow_matlab and mexfcn, including the MATLAB dependencies.
+ # If Arrow_FOUND is true, this also includes the path to arrow_shared.
+ set_target_properties(arrow_matlab mexfcn PROPERTIES INSTALL_RPATH_USE_LINK_PATH TRUE)
+
+ # Add $ORIGIN to the RUNPATH of mexfcn so that libarrow_matlab.so can be found
+ # at runtime.
+ set_target_properties(mexfcn PROPERTIES INSTALL_RPATH $ORIGIN)
+
+ if(NOT Arrow_FOUND)
+ # If Arrow_FOUND is false, Arrow is built by the arrow_shared target and needs
+ # to be copied to CMAKE_INSTALL_DIR. The DIRECTORY install command is used to
+ # install libarrow.so (symlink) and the real files it points to.
+ #
+ # The subfolders cmake and pkgconfig are excluded as they will be empty.
+ # Note: The following CMake Issue suggests enabling an option to exclude all
+ # folders that would be empty after installation:
+ # https://gitlab.kitware.com/cmake/cmake/-/issues/17122
+ install(DIRECTORY "${ARROW_SHARED_LIB_DIR}/"
+ DESTINATION ${CMAKE_INSTALL_DIR}
+ FILES_MATCHING
+ REGEX "${ARROW_SHARED_LIB_FILENAME}\\.so.*"
+ PATTERN "cmake" EXCLUDE
+ PATTERN "pkgconfig" EXCLUDE)
+
+ # Add $ORIGIN to the RUNPATH of arrow_matlab so that libarrow.so can be found
+ # at runtime.
+ set_target_properties(arrow_matlab PROPERTIES INSTALL_RPATH $ORIGIN)
+ endif()
+endif()
+
+# MATLAB_ADD_INSTALL_DIR_TO_STARTUP_FILE toggles whether an addpath command to add the install
+# directory path to the MATLAB Search Path is added to the startup.m file located in the MATLAB
+# userpath directory.
+option(MATLAB_ADD_INSTALL_DIR_TO_STARTUP_FILE
+ "Sets whether the path to the install directory should be added to the startup.m file located at the MATLAB userpath"
+ OFF)
+
+# If MATLAB_ADD_INSTALL_DIR_TO_STARTUP_FILE is specified ON and MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH
+# is not specified, then set MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH=OFF.
+if(MATLAB_ADD_INSTALL_DIR_TO_STARTUP_FILE AND NOT DEFINED
+ MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH)
+ set(MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH OFF)
+endif()
+
+# MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH toggles whether the path to the install directory should
+# be directly added to the MATLAB Search Path.
+option(MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH
+ "Sets whether the path to the install directory should be directly added to the MATLAB Search Path"
+ ON)
+
+if(MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH OR MATLAB_ADD_INSTALL_DIR_TO_STARTUP_FILE)
+ set(TOOLS_DIR "${CMAKE_SOURCE_DIR}/tools")
+
+ # Pass Matlab_MAIN_PROGRAM, TOOLS_DIR, and INSTALL_DIR to the install step
+ # code/script execution scope.
+ install(CODE "set(Matlab_MAIN_PROGRAM \"${Matlab_MAIN_PROGRAM}\")")
+ install(CODE "set(TOOLS_DIR \"${TOOLS_DIR}\")")
+ install(CODE "set(INSTALL_DIR \"${CMAKE_INSTALL_DIR}\")")
+ install(CODE "set(MATLAB_ADD_INSTALL_DIR_TO_STARTUP_FILE \"${MATLAB_ADD_INSTALL_DIR_TO_STARTUP_FILE}\")"
+ )
+ install(CODE "set(MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH \"${MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH}\")"
+ )
+
+ # Call the CMake script that runs the MATLAB function to add the install directory
+ # to the MATLAB Search Path or add a command to the MATLAB startup file to add the
+ # install directory to the MATLAB Search Path.
+ install(SCRIPT "${TOOLS_DIR}/UpdateMatlabSearchPath.cmake")
endif()
diff --git a/matlab/src/util/unicode_conversion.h b/matlab/src/cpp/arrow/matlab/api/visibility.h
similarity index 71%
copy from matlab/src/util/unicode_conversion.h
copy to matlab/src/cpp/arrow/matlab/api/visibility.h
index fa905cb..51efac9 100644
--- a/matlab/src/util/unicode_conversion.h
+++ b/matlab/src/cpp/arrow/matlab/api/visibility.h
@@ -17,16 +17,12 @@
#pragma once
-#include <string>
-#include <mex.h>
-
-namespace arrow {
-namespace matlab {
-namespace util {
-// Converts a UTF-8 encoded std::string to a heap-allocated UTF-16 encoded
-// mxCharArray.
-mxArray* ConvertUTF8StringToUTF16CharMatrix(const std::string& utf8_string);
-} // namespace util
-} // namespace matlab
-} // namespace arrow
-
+#if defined(_WIN32) || defined(__CYGWIN__)
+#ifdef ARROW_MATLAB_EXPORTING
+#define ARROW_MATLAB_EXPORT __declspec(dllexport)
+#else
+#define ARROW_MATLAB_EXPORT __declspec(dllimport)
+#endif
+#else // Not Windows
+#define ARROW_MATLAB_EXPORT __attribute__((visibility("default")))
+#endif
diff --git a/matlab/src/featherreadmex.cc b/matlab/src/cpp/arrow/matlab/feather/feather_functions.cc
similarity index 57%
copy from matlab/src/featherreadmex.cc
copy to matlab/src/cpp/arrow/matlab/feather/feather_functions.cc
index b52b8a9..f5b5a39 100644
--- a/matlab/src/featherreadmex.cc
+++ b/matlab/src/cpp/arrow/matlab/feather/feather_functions.cc
@@ -15,23 +15,41 @@
// specific language governing permissions and limitations
// under the License.
-#include <string>
+#include "feather_functions.h"
-#include <mex.h>
+#include <string>
#include "feather_reader.h"
+#include "feather_writer.h"
#include "util/handle_status.h"
-// MEX gateway function. This is the entry point for featherreadmex.cpp.
-void mexFunction(int nlhs, mxArray* plhs[], int nrhs, const mxArray* prhs[]) {
+namespace arrow {
+namespace matlab {
+namespace feather {
+
+void featherwrite(int nlhs, mxArray* plhs[], int nrhs, const mxArray* prhs[]) {
+ const std::string filename{mxArrayToUTF8String(prhs[0])};
+
+ // Open a Feather file at the provided file path for writing.
+ std::shared_ptr<FeatherWriter> feather_writer{nullptr};
+ util::HandleStatus(FeatherWriter::Open(filename, &feather_writer));
+
+ // Write the Feather file table variables and table metadata from MATLAB.
+ util::HandleStatus(feather_writer->WriteVariables(prhs[1], prhs[2]));
+}
+
+void featherread(int nlhs, mxArray* plhs[], int nrhs, const mxArray* prhs[]) {
const std::string filename{mxArrayToUTF8String(prhs[0])};
// Read the given Feather file into memory.
- std::shared_ptr<arrow::matlab::FeatherReader> feather_reader{nullptr};
- arrow::matlab::util::HandleStatus(
- arrow::matlab::FeatherReader::Open(filename, &feather_reader));
+ std::shared_ptr<FeatherReader> feather_reader{nullptr};
+ util::HandleStatus(FeatherReader::Open(filename, &feather_reader));
// Return the Feather file table variables and table metadata to MATLAB.
plhs[0] = feather_reader->ReadVariables();
plhs[1] = feather_reader->ReadMetadata();
}
+
+} // namespace feather
+} // namespace matlab
+} // namespace arrow
\ No newline at end of file
diff --git a/matlab/src/util/unicode_conversion.h b/matlab/src/cpp/arrow/matlab/feather/feather_functions.h
similarity index 76%
copy from matlab/src/util/unicode_conversion.h
copy to matlab/src/cpp/arrow/matlab/feather/feather_functions.h
index fa905cb..c055e95 100644
--- a/matlab/src/util/unicode_conversion.h
+++ b/matlab/src/cpp/arrow/matlab/feather/feather_functions.h
@@ -17,16 +17,18 @@
#pragma once
-#include <string>
#include <mex.h>
+#include "arrow/matlab/api/visibility.h"
+
namespace arrow {
namespace matlab {
-namespace util {
-// Converts a UTF-8 encoded std::string to a heap-allocated UTF-16 encoded
-// mxCharArray.
-mxArray* ConvertUTF8StringToUTF16CharMatrix(const std::string& utf8_string);
-} // namespace util
+namespace feather {
+
+ARROW_MATLAB_EXPORT void featherwrite(int nlhs, mxArray* plhs[], int nrhs, const mxArray* prhs[]);
+
+ARROW_MATLAB_EXPORT void featherread(int nlhs, mxArray* plhs[], int nrhs, const mxArray* prhs[]);
+
+} // namespace feather
} // namespace matlab
} // namespace arrow
-
diff --git a/matlab/src/feather_reader.cc b/matlab/src/cpp/arrow/matlab/feather/feather_reader.cc
similarity index 98%
rename from matlab/src/feather_reader.cc
rename to matlab/src/cpp/arrow/matlab/feather/feather_reader.cc
index 1cbb505..413b7f1 100644
--- a/matlab/src/feather_reader.cc
+++ b/matlab/src/cpp/arrow/matlab/feather/feather_reader.cc
@@ -15,9 +15,6 @@
// specific language governing permissions and limitations
// under the License.
-#include <algorithm>
-#include <cmath>
-
#include "feather_reader.h"
#include <arrow/array/array_base.h>
@@ -33,17 +30,21 @@
#include <arrow/util/bitmap_visit.h>
#include <mex.h>
+#include <algorithm>
+#include <cmath>
+
#include "matlab_traits.h"
#include "util/handle_status.h"
#include "util/unicode_conversion.h"
namespace arrow {
namespace matlab {
+namespace feather {
namespace internal {
// Read the name of variable i from the Feather file as a mxArray*.
mxArray* ReadVariableName(const std::string& column_name) {
- return matlab::util::ConvertUTF8StringToUTF16CharMatrix(column_name);
+ return util::ConvertUTF8StringToUTF16CharMatrix(column_name);
}
template <typename ArrowDataType>
@@ -57,8 +58,7 @@ mxArray* ReadNumericVariableData(const std::shared_ptr<Array>& column) {
mxArray* variable_data =
mxCreateNumericMatrix(column->length(), 1, matlab_class_id, mxREAL);
- auto arrow_numeric_array =
- std::static_pointer_cast<ArrowArrayType>(column);
+ auto arrow_numeric_array = std::static_pointer_cast<ArrowArrayType>(column);
// Get a raw pointer to the Arrow array data.
const MatlabType* source = arrow_numeric_array->raw_values();
@@ -182,10 +182,10 @@ Status FeatherReader::Open(const std::string& filename,
// Open file with given filename as a ReadableFile.
ARROW_ASSIGN_OR_RAISE(auto readable_file, io::ReadableFile::Open(filename));
-
+
// Open the Feather file for reading with a TableReader.
ARROW_ASSIGN_OR_RAISE(auto reader, ipc::feather::Reader::Open(readable_file));
-
+
// Set the internal reader_ object.
(*feather_reader)->reader_ = reader;
@@ -273,5 +273,6 @@ mxArray* FeatherReader::ReadVariables() {
return variables;
}
+} // namespace feather
} // namespace matlab
-} // namespace arrow
+} // namespace arrow
\ No newline at end of file
diff --git a/matlab/src/feather_reader.h b/matlab/src/cpp/arrow/matlab/feather/feather_reader.h
similarity index 97%
rename from matlab/src/feather_reader.h
rename to matlab/src/cpp/arrow/matlab/feather/feather_reader.h
index 197e470..0e2089d 100644
--- a/matlab/src/feather_reader.h
+++ b/matlab/src/cpp/arrow/matlab/feather/feather_reader.h
@@ -17,16 +17,17 @@
#pragma once
-#include <memory>
-#include <string>
-
#include <arrow/ipc/feather.h>
#include <arrow/status.h>
#include <arrow/type.h>
#include <matrix.h>
+#include <memory>
+#include <string>
+
namespace arrow {
namespace matlab {
+namespace feather {
class FeatherReader {
public:
@@ -71,5 +72,6 @@ class FeatherReader {
std::string description_;
};
+} // namespace feather
} // namespace matlab
-} // namespace arrow
+} // namespace arrow
\ No newline at end of file
diff --git a/matlab/src/feather_writer.cc b/matlab/src/cpp/arrow/matlab/feather/feather_writer.cc
similarity index 98%
rename from matlab/src/feather_writer.cc
rename to matlab/src/cpp/arrow/matlab/feather/feather_writer.cc
index 1a76ada..f67d8dc 100644
--- a/matlab/src/feather_writer.cc
+++ b/matlab/src/cpp/arrow/matlab/feather/feather_writer.cc
@@ -15,10 +15,6 @@
// specific language governing permissions and limitations
// under the License.
-#include <cmath>
-#include <functional> /* for std::multiplies */
-#include <numeric> /* for std::accumulate */
-
#include "feather_writer.h"
#include <arrow/array.h>
@@ -33,11 +29,16 @@
#include <arrow/util/key_value_metadata.h>
#include <mex.h>
+#include <cmath>
+#include <functional> /* for std::multiplies */
+#include <numeric> /* for std::accumulate */
+
#include "matlab_traits.h"
#include "util/handle_status.h"
namespace arrow {
namespace matlab {
+namespace feather {
namespace internal {
// Returns the arrow::DataType that corresponds to the input type string
@@ -279,7 +280,8 @@ Status FeatherWriter::Open(const std::string& filename,
*feather_writer = std::shared_ptr<FeatherWriter>(new FeatherWriter());
// Open a FileOutputStream corresponding to the provided filename.
- ARROW_ASSIGN_OR_RAISE((*feather_writer)->file_output_stream_,
+ ARROW_ASSIGN_OR_RAISE(
+ (*feather_writer)->file_output_stream_,
io::FileOutputStream::Open(filename, &((*feather_writer)->file_output_stream_)));
return Status::OK();
}
@@ -331,15 +333,15 @@ Status FeatherWriter::WriteVariables(const mxArray* variables, const mxArray* me
auto datatype = internal::ConvertMatlabTypeStringToArrowDataType(type_str);
auto field = std::make_shared<arrow::Field>(name_str, datatype);
- ARROW_ASSIGN_OR_RAISE(std::shared_ptr<ResizableBuffer> validity_bitmap,
+ ARROW_ASSIGN_OR_RAISE(
+ std::shared_ptr<ResizableBuffer> validity_bitmap,
arrow::AllocateResizableBuffer(internal::BitPackedLength(num_rows_)));
// Populate bit-packed arrow::Buffer using validity data in the mxArray*.
internal::BitPackBuffer(valid, validity_bitmap);
// Wrap mxArray data in an arrow::Array of the equivalent type.
- auto array =
- internal::WriteVariableData(data, type_str, validity_bitmap);
+ auto array = internal::WriteVariableData(data, type_str, validity_bitmap);
// Verify that the arrow::Array has the right number of elements.
internal::ValidateNumRows(array->length(), num_rows_);
@@ -362,5 +364,6 @@ Status FeatherWriter::WriteVariables(const mxArray* variables, const mxArray* me
return ipc::feather::WriteTable(*table, file_output_stream_.get(), write_props);
}
+} // namespace feather
} // namespace matlab
} // namespace arrow
diff --git a/matlab/src/feather_writer.h b/matlab/src/cpp/arrow/matlab/feather/feather_writer.h
similarity index 97%
rename from matlab/src/feather_writer.h
rename to matlab/src/cpp/arrow/matlab/feather/feather_writer.h
index a35b143..0042d4a 100644
--- a/matlab/src/feather_writer.h
+++ b/matlab/src/cpp/arrow/matlab/feather/feather_writer.h
@@ -17,16 +17,17 @@
#pragma once
-#include <memory>
-#include <string>
-
#include <arrow/ipc/feather.h>
#include <arrow/status.h>
#include <arrow/type.h>
#include <matrix.h>
+#include <memory>
+#include <string>
+
namespace arrow {
namespace matlab {
+namespace feather {
class FeatherWriter {
public:
@@ -64,5 +65,6 @@ class FeatherWriter {
std::shared_ptr<arrow::io::OutputStream> file_output_stream_;
};
+} // namespace feather
} // namespace matlab
-} // namespace arrow
+} // namespace arrow
\ No newline at end of file
diff --git a/matlab/src/matlab_traits.h b/matlab/src/cpp/arrow/matlab/feather/matlab_traits.h
similarity index 99%
rename from matlab/src/matlab_traits.h
rename to matlab/src/cpp/arrow/matlab/feather/matlab_traits.h
index a76539f..8ef5aea 100644
--- a/matlab/src/matlab_traits.h
+++ b/matlab/src/cpp/arrow/matlab/feather/matlab_traits.h
@@ -18,7 +18,6 @@
#pragma once
#include <arrow/type.h>
-
#include <matrix.h>
namespace arrow {
@@ -100,4 +99,3 @@ struct MatlabTraits<Int64Type> {
} // namespace matlab
} // namespace arrow
-
diff --git a/matlab/src/util/handle_status.cc b/matlab/src/cpp/arrow/matlab/feather/util/handle_status.cc
similarity index 98%
rename from matlab/src/util/handle_status.cc
rename to matlab/src/cpp/arrow/matlab/feather/util/handle_status.cc
index f1c3b7f..86d6c1c 100644
--- a/matlab/src/util/handle_status.cc
+++ b/matlab/src/cpp/arrow/matlab/feather/util/handle_status.cc
@@ -16,11 +16,11 @@
// under the License.
#include <arrow/status.h>
-
#include <mex.h>
namespace arrow {
namespace matlab {
+namespace feather {
namespace util {
void HandleStatus(const Status& status) {
@@ -86,6 +86,8 @@ void HandleStatus(const Status& status) {
}
}
}
+
} // namespace util
+} // namespace feather
} // namespace matlab
} // namespace arrow
diff --git a/matlab/src/util/handle_status.h b/matlab/src/cpp/arrow/matlab/feather/util/handle_status.h
similarity index 96%
rename from matlab/src/util/handle_status.h
rename to matlab/src/cpp/arrow/matlab/feather/util/handle_status.h
index 7212114..3025b3b 100644
--- a/matlab/src/util/handle_status.h
+++ b/matlab/src/cpp/arrow/matlab/feather/util/handle_status.h
@@ -21,12 +21,15 @@
namespace arrow {
namespace matlab {
+namespace feather {
namespace util {
+
// Terminates execution and returns to the MATLAB prompt,
// displaying an error message if the given status
// indicates that an error has occurred.
void HandleStatus(const Status& status);
+
} // namespace util
+} // namespace feather
} // namespace matlab
} // namespace arrow
-
diff --git a/matlab/src/util/unicode_conversion.cc b/matlab/src/cpp/arrow/matlab/feather/util/unicode_conversion.cc
similarity index 91%
rename from matlab/src/util/unicode_conversion.cc
rename to matlab/src/cpp/arrow/matlab/feather/util/unicode_conversion.cc
index 01c2e4b..623943d 100644
--- a/matlab/src/util/unicode_conversion.cc
+++ b/matlab/src/cpp/arrow/matlab/feather/util/unicode_conversion.cc
@@ -15,13 +15,14 @@
// specific language governing permissions and limitations
// under the License.
-#include <locale> /* for std::wstring_convert */
-#include <codecvt> /* for std::codecvt_utf8_utf16 */
-
#include "unicode_conversion.h"
+#include <codecvt> /* for std::codecvt_utf8_utf16 */
+#include <locale> /* for std::wstring_convert */
+
namespace arrow {
namespace matlab {
+namespace feather {
namespace util {
mxArray* ConvertUTF8StringToUTF16CharMatrix(const std::string& utf8_string) {
@@ -29,7 +30,7 @@ mxArray* ConvertUTF8StringToUTF16CharMatrix(const std::string& utf8_string) {
const char* string_start = utf8_string.c_str();
const char* string_end = string_start + utf8_string.length();
- // Due to this issue on MSVC: https://stackoverflow.com/q/32055357 we cannot
+ // Due to this issue on MSVC: https://stackoverflow.com/q/32055357 we cannot
// directly use a destination type of char16_t.
#if _MSC_VER >= 1900
using CharType = int16_t;
@@ -43,7 +44,7 @@ mxArray* ConvertUTF8StringToUTF16CharMatrix(const std::string& utf8_string) {
try {
utf16_string = code_converter.from_bytes(string_start, string_end);
} catch (...) {
- // In the case that any error occurs, just try returning a string in the
+ // In the case that any error occurs, just try returning a string in the
// user's current locale instead.
return mxCreateString(string_start);
}
@@ -52,12 +53,13 @@ mxArray* ConvertUTF8StringToUTF16CharMatrix(const std::string& utf8_string) {
const mwSize dimensions[2] = {1, utf16_string.size()};
mxArray* character_matrix = mxCreateCharArray(2, dimensions);
mxChar* character_matrix_pointer = mxGetChars(character_matrix);
- std::copy(utf16_string.data(), utf16_string.data() + utf16_string.size(),
- character_matrix_pointer);
+ std::copy(utf16_string.data(), utf16_string.data() + utf16_string.size(),
+ character_matrix_pointer);
return character_matrix;
}
} // namespace util
+} // namespace feather
} // namespace matlab
-} // namespace arrow
+} // namespace arrow
\ No newline at end of file
diff --git a/matlab/src/util/unicode_conversion.h b/matlab/src/cpp/arrow/matlab/feather/util/unicode_conversion.h
similarity index 96%
copy from matlab/src/util/unicode_conversion.h
copy to matlab/src/cpp/arrow/matlab/feather/util/unicode_conversion.h
index fa905cb..8a41f08 100644
--- a/matlab/src/util/unicode_conversion.h
+++ b/matlab/src/cpp/arrow/matlab/feather/util/unicode_conversion.h
@@ -17,16 +17,20 @@
#pragma once
-#include <string>
#include <mex.h>
+#include <string>
+
namespace arrow {
namespace matlab {
+namespace feather {
namespace util {
+
// Converts a UTF-8 encoded std::string to a heap-allocated UTF-16 encoded
// mxCharArray.
mxArray* ConvertUTF8StringToUTF16CharMatrix(const std::string& utf8_string);
+
} // namespace util
+} // namespace feather
} // namespace matlab
} // namespace arrow
-
diff --git a/matlab/src/featherwritemex.cc b/matlab/src/cpp/arrow/matlab/mex/mex_functions.h
similarity index 55%
rename from matlab/src/featherwritemex.cc
rename to matlab/src/cpp/arrow/matlab/mex/mex_functions.h
index d8f90ba..aa09fc8 100644
--- a/matlab/src/featherwritemex.cc
+++ b/matlab/src/cpp/arrow/matlab/mex/mex_functions.h
@@ -16,21 +16,25 @@
// under the License.
#include <string>
+#include <functional>
+#include <unordered_map>
#include <mex.h>
-#include "feather_writer.h"
-#include "util/handle_status.h"
+#include "arrow/matlab/feather/feather_functions.h"
-// MEX gateway function. This is the entry point for featherwritemex.cc.
-void mexFunction(int nlhs, mxArray* plhs[], int nrhs, const mxArray* prhs[]) {
- const std::string filename{mxArrayToUTF8String(prhs[0])};
+namespace arrow {
+namespace matlab {
+namespace mex {
+
+using namespace arrow::matlab::feather;
- // Open a Feather file at the provided file path for writing.
- std::shared_ptr<arrow::matlab::FeatherWriter> feather_writer{nullptr};
- arrow::matlab::util::HandleStatus(
- arrow::matlab::FeatherWriter::Open(filename, &feather_writer));
+using mex_fcn_t =
+ std::function<void(int nlhs, mxArray* plhs[], int nrhs, const mxArray* prhs[])>;
- // Write the Feather file table variables and table metadata from MATLAB.
- arrow::matlab::util::HandleStatus(feather_writer->WriteVariables(prhs[1], prhs[2]));
-}
+static const std::unordered_map<std::string, mex_fcn_t> FUNCTION_MAP = {
+ {"featherread", featherread}, {"featherwrite", featherwrite}};
+
+} // namespace mex
+} // namespace matlab
+} // namespace arrow
\ No newline at end of file
diff --git a/matlab/src/cpp/arrow/matlab/mex/mex_util.cc b/matlab/src/cpp/arrow/matlab/mex/mex_util.cc
new file mode 100644
index 0000000..ba6e87c
--- /dev/null
+++ b/matlab/src/cpp/arrow/matlab/mex/mex_util.cc
@@ -0,0 +1,53 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "mex_util.h"
+
+namespace arrow {
+namespace matlab {
+namespace mex {
+
+void checkNumArgs(int nrhs) {
+ if (nrhs < 1) {
+ mexErrMsgIdAndTxt("MATLAB:arrow:minrhs",
+ "'mexfcn' requires at least one input argument, which must be the "
+ "name of the C++ MEX to invoke.");
+ }
+}
+
+std::string get_function_name(const mxArray* input) {
+ std::string opname;
+ if (!mxIsChar(input)) {
+ mexErrMsgIdAndTxt("MATLAB:arrow:FunctionNameDataType",
+ "The first input argument to 'mexfcn' must be a character vector.");
+ }
+ const char* c_str = mxArrayToUTF8String(input);
+ return std::string{c_str};
+}
+
+mex_fcn_t lookup_function(const std::string& function_name) {
+ auto kv_pair = FUNCTION_MAP.find(function_name);
+ if (kv_pair == FUNCTION_MAP.end()) {
+ mexErrMsgIdAndTxt("MATLAB:arrow:UnknownMEXFunction", "Unrecognized MEX function '%s'",
+ function_name.c_str());
+ }
+ return kv_pair->second;
+}
+
+} // namespace mex
+} // namespace matlab
+} // namespace arrow
\ No newline at end of file
diff --git a/matlab/src/util/unicode_conversion.h b/matlab/src/cpp/arrow/matlab/mex/mex_util.h
similarity index 68%
copy from matlab/src/util/unicode_conversion.h
copy to matlab/src/cpp/arrow/matlab/mex/mex_util.h
index fa905cb..8e02084 100644
--- a/matlab/src/util/unicode_conversion.h
+++ b/matlab/src/cpp/arrow/matlab/mex/mex_util.h
@@ -18,15 +18,25 @@
#pragma once
#include <string>
+#include <functional>
+#include <unordered_map>
+
#include <mex.h>
+#include "mex_functions.h"
+
+#include "arrow/matlab/api/visibility.h"
+
namespace arrow {
namespace matlab {
-namespace util {
-// Converts a UTF-8 encoded std::string to a heap-allocated UTF-16 encoded
-// mxCharArray.
-mxArray* ConvertUTF8StringToUTF16CharMatrix(const std::string& utf8_string);
-} // namespace util
-} // namespace matlab
-} // namespace arrow
+namespace mex {
+ARROW_MATLAB_EXPORT void checkNumArgs(int nrhs);
+
+ARROW_MATLAB_EXPORT std::string get_function_name(const mxArray* input);
+
+ARROW_MATLAB_EXPORT mex_fcn_t lookup_function(const std::string& function_name);
+
+} // namespace mex
+} // namespace matlab
+} // namespace arrow
diff --git a/matlab/src/util/unicode_conversion.h b/matlab/src/cpp/arrow/matlab/mex/mex_util_test.cc
similarity index 77%
rename from matlab/src/util/unicode_conversion.h
rename to matlab/src/cpp/arrow/matlab/mex/mex_util_test.cc
index fa905cb..8f2c65c 100644
--- a/matlab/src/util/unicode_conversion.h
+++ b/matlab/src/cpp/arrow/matlab/mex/mex_util_test.cc
@@ -15,18 +15,16 @@
// specific language governing permissions and limitations
// under the License.
-#pragma once
+#include <gtest/gtest.h>
-#include <string>
-#include <mex.h>
+#include "mex_util.h"
namespace arrow {
namespace matlab {
-namespace util {
-// Converts a UTF-8 encoded std::string to a heap-allocated UTF-16 encoded
-// mxCharArray.
-mxArray* ConvertUTF8StringToUTF16CharMatrix(const std::string& utf8_string);
-} // namespace util
+namespace mex {
+TEST(CheckNumArgsTests, TooFewArgsError) {
+ EXPECT_THROW(checkNumArgs(0), std::exception);
+}
+} // namespace mex
} // namespace matlab
} // namespace arrow
-
diff --git a/matlab/src/featherreadmex.cc b/matlab/src/cpp/arrow/matlab/mex/mexfcn.cc
similarity index 59%
rename from matlab/src/featherreadmex.cc
rename to matlab/src/cpp/arrow/matlab/mex/mexfcn.cc
index b52b8a9..c1205b0 100644
--- a/matlab/src/featherreadmex.cc
+++ b/matlab/src/cpp/arrow/matlab/mex/mexfcn.cc
@@ -15,23 +15,14 @@
// specific language governing permissions and limitations
// under the License.
-#include <string>
-
#include <mex.h>
-#include "feather_reader.h"
-#include "util/handle_status.h"
+#include "mex_util.h"
-// MEX gateway function. This is the entry point for featherreadmex.cpp.
void mexFunction(int nlhs, mxArray* plhs[], int nrhs, const mxArray* prhs[]) {
- const std::string filename{mxArrayToUTF8String(prhs[0])};
-
- // Read the given Feather file into memory.
- std::shared_ptr<arrow::matlab::FeatherReader> feather_reader{nullptr};
- arrow::matlab::util::HandleStatus(
- arrow::matlab::FeatherReader::Open(filename, &feather_reader));
+ using namespace arrow::matlab::mex;
- // Return the Feather file table variables and table metadata to MATLAB.
- plhs[0] = feather_reader->ReadVariables();
- plhs[1] = feather_reader->ReadMetadata();
+ checkNumArgs(nrhs);
+ auto fcn = lookup_function(get_function_name(prhs[0]));
+ fcn(nlhs, plhs, nrhs - 1, ++prhs);
}
diff --git a/matlab/src/+mlarrow/+util/createMetadataStruct.m b/matlab/src/matlab/+arrow/+util/createMetadataStruct.m
similarity index 100%
rename from matlab/src/+mlarrow/+util/createMetadataStruct.m
rename to matlab/src/matlab/+arrow/+util/createMetadataStruct.m
diff --git a/matlab/src/+mlarrow/+util/createVariableStruct.m b/matlab/src/matlab/+arrow/+util/createVariableStruct.m
similarity index 100%
rename from matlab/src/+mlarrow/+util/createVariableStruct.m
rename to matlab/src/matlab/+arrow/+util/createVariableStruct.m
diff --git a/matlab/src/+mlarrow/+util/makeValidMATLABTableVariableNames.m b/matlab/src/matlab/+arrow/+util/makeValidMATLABTableVariableNames.m
similarity index 100%
rename from matlab/src/+mlarrow/+util/makeValidMATLABTableVariableNames.m
rename to matlab/src/matlab/+arrow/+util/makeValidMATLABTableVariableNames.m
diff --git a/matlab/src/+mlarrow/+util/table2mlarrow.m b/matlab/src/matlab/+arrow/+util/table2mlarrow.m
similarity index 99%
rename from matlab/src/+mlarrow/+util/table2mlarrow.m
rename to matlab/src/matlab/+arrow/+util/table2mlarrow.m
index 36e4d1d..391b060 100644
--- a/matlab/src/+mlarrow/+util/table2mlarrow.m
+++ b/matlab/src/matlab/+arrow/+util/table2mlarrow.m
@@ -43,7 +43,7 @@ function [variables, metadata] = table2mlarrow(t)
% implied. See the License for the specific language governing
% permissions and limitations under the License.
-import mlarrow.util.*;
+import arrow.util.*;
% Struct array representing the underlying data of each variable
% in the given table.
diff --git a/matlab/src/featherread.m b/matlab/src/matlab/featherread.m
similarity index 97%
rename from matlab/src/featherread.m
rename to matlab/src/matlab/featherread.m
index 31bc426..1d77cd3 100644
--- a/matlab/src/featherread.m
+++ b/matlab/src/matlab/featherread.m
@@ -23,7 +23,7 @@ function t = featherread(filename)
% specific language governing permissions and limitations
% under the License.
-import mlarrow.util.*;
+import arrow.util.*;
% Validate input arguments.
narginchk(1, 1);
@@ -46,7 +46,7 @@ end
% Read table variables and metadata from the given Feather file using
% libarrow.
-[variables, metadata] = featherreadmex(filename);
+[variables, metadata] = mexfcn('featherread', filename);
% Make valid MATLAB table variable names out of any of the
% Feather table column names that are not valid MATLAB table
diff --git a/matlab/src/featherwrite.m b/matlab/src/matlab/featherwrite.m
similarity index 94%
rename from matlab/src/featherwrite.m
rename to matlab/src/matlab/featherwrite.m
index eeedf26..361d7b6 100644
--- a/matlab/src/featherwrite.m
+++ b/matlab/src/matlab/featherwrite.m
@@ -23,7 +23,7 @@ function featherwrite(filename, t)
% specific language governing permissions and limitations
% under the License.
-import mlarrow.util.table2mlarrow;
+import arrow.util.table2mlarrow;
% Validate input arguments.
narginchk(2, 2);
@@ -39,6 +39,5 @@ end
[variables, metadata] = table2mlarrow(t);
% Write the table to a Feather file.
-featherwritemex(filename, variables, metadata);
-
+mexfcn('featherwrite', filename, variables, metadata);
end
diff --git a/matlab/test/tfeather.m b/matlab/test/tfeather.m
index 625a3a5..7d87176 100755
--- a/matlab/test/tfeather.m
+++ b/matlab/test/tfeather.m
@@ -17,21 +17,14 @@ classdef tfeather < matlab.unittest.TestCase
% permissions and limitations under the License.
methods(TestClassSetup)
-
function addFeatherFunctionsToMATLABPath(testCase)
import matlab.unittest.fixtures.PathFixture
% Add Feather test utilities to the MATLAB path.
testCase.applyFixture(PathFixture('util'));
- % Add featherread and featherwrite to the MATLAB path.
- testCase.applyFixture(PathFixture(fullfile('..', 'src')));
- % featherreadmex must be on the MATLAB path.
- testCase.assertTrue(~isempty(which('featherreadmex')), ...
- '''featherreadmex'' must be on the MATLAB path. Use ''addpath'' to add folders to the MATLAB path.');
- % featherwritemex must be on the MATLAB path.
- testCase.assertTrue(~isempty(which('featherwritemex')), ...
- '''featherwritemex'' must be on to the MATLAB path. Use ''addpath'' to add folders to the MATLAB path.');
+ % mexfcn must be on the MATLAB path.
+ testCase.assertTrue(~isempty(which('mexfcn')), ...
+ '''mexfcn'' must be on the MATLAB path. Use ''addpath'' to add folders to the MATLAB path.');
end
-
end
methods(TestMethodSetup)
@@ -226,7 +219,5 @@ classdef tfeather < matlab.unittest.TestCase
expectedTable = featherRoundTrip(filename, actualTable);
testCase.verifyNotEqual(actualTable, expectedTable);
end
-
end
-
end
diff --git a/matlab/test/tfeathermex.m b/matlab/test/tfeathermex.m
index 77070ad..f502b05 100644
--- a/matlab/test/tfeathermex.m
+++ b/matlab/test/tfeathermex.m
@@ -22,16 +22,10 @@ classdef tfeathermex < matlab.unittest.TestCase
import matlab.unittest.fixtures.PathFixture
% Add Feather test utilities to the MATLAB path.
testCase.applyFixture(PathFixture('util'));
- % Add featherread and featherwrite to the MATLAB path.
- testCase.applyFixture(PathFixture(fullfile('..', 'src')));
- % featherreadmex must be on the MATLAB path.
- testCase.assertTrue(~isempty(which('featherreadmex')), ...
- '''featherreadmex'' must be on the MATLAB path. Use ''addpath'' to add folders to the MATLAB path.');
- % featherwritemex must be on the MATLAB path.
- testCase.assertTrue(~isempty(which('featherwritemex')), ...
- '''featherwritemex'' must be on to the MATLAB path. Use ''addpath'' to add folders to the MATLAB path.');
+ % mexfcn must be on the MATLAB path.
+ testCase.assertTrue(~isempty(which('mexfcn')), ...
+ '''mexfcn'' must be on the MATLAB path. Use ''addpath'' to add folders to the MATLAB path.');
end
-
end
methods(TestMethodSetup)
@@ -40,7 +34,6 @@ classdef tfeathermex < matlab.unittest.TestCase
import matlab.unittest.fixtures.WorkingFolderFixture;
testCase.applyFixture(WorkingFolderFixture);
end
-
end
methods(Test)
@@ -57,11 +50,11 @@ classdef tfeathermex < matlab.unittest.TestCase
filename = fullfile(pwd, 'temp.feather');
% Create a table with an invalid MATLAB table variable name.
- invalidVariable = mlarrow.util.createVariableStruct('double', 1, true, '@');
- validVariable = mlarrow.util.createVariableStruct('double', 1, true, 'Valid');
+ invalidVariable = arrow.util.createVariableStruct('double', 1, true, '@');
+ validVariable = arrow.util.createVariableStruct('double', 1, true, 'Valid');
variables = [invalidVariable, validVariable];
- metadata = mlarrow.util.createMetadataStruct(1, 2);
- featherwritemex(filename, variables, metadata);
+ metadata = arrow.util.createMetadataStruct(1, 2);
+ mexfcn('featherwrite', filename, variables, metadata);
t = featherread(filename);
testCase.verifyEqual(t.Properties.VariableNames{1}, 'x_');
@@ -69,8 +62,6 @@ classdef tfeathermex < matlab.unittest.TestCase
testCase.verifyEqual(t.Properties.VariableDescriptions{1}, 'Original variable name: ''@''');
testCase.verifyEqual(t.Properties.VariableDescriptions{2}, '');
- end
-
+ end
end
-
end
diff --git a/matlab/test/tmexfcn.m b/matlab/test/tmexfcn.m
new file mode 100644
index 0000000..ebaec65
--- /dev/null
+++ b/matlab/test/tmexfcn.m
@@ -0,0 +1,61 @@
+classdef tmexfcn < matlab.unittest.TestCase
+ % Tests for mexfcn.
+
+ % Licensed to the Apache Software Foundation (ASF) under one or more
+ % contributor license agreements. See the NOTICE file distributed with
+ % this work for additional information regarding copyright ownership.
+ % The ASF licenses this file to you under the Apache License, Version
+ % 2.0 (the "License"); you may not use this file except in compliance
+ % with the License. You may obtain a copy of the License at
+ %
+ % http://www.apache.org/licenses/LICENSE-2.0
+ %
+ % Unless required by applicable law or agreed to in writing, software
+ % distributed under the License is distributed on an "AS IS" BASIS,
+ % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ % implied. See the License for the specific language governing
+ % permissions and limitations under the License.
+
+ methods(TestClassSetup)
+ function addFeatherFunctionsToMATLABPath(testCase)
+ import matlab.unittest.fixtures.PathFixture
+ % Add Feather test utilities to the MATLAB path.
+ testCase.applyFixture(PathFixture('util'));
+ % mexfcn must be on the MATLAB path.
+ testCase.assertTrue(~isempty(which('mexfcn')), ...
+ '''mexfcn'' must be on the MATLAB path. Use ''addpath'' to add folders to the MATLAB path.');
+ end
+ end
+
+ methods(Test)
+ function UnknownMEXFunctionError(testCase)
+ % Verifies mexfcn throws an error if an unkown MEX function name
+ % is passed to it.
+ errID = "MATLAB:arrow:UnknownMEXFunction";
+ fcn = @()mexfcn('NotAFunction');
+ testCase.verifyError(fcn, errID);
+ end
+
+ function TooFewInputArguementsError(testCase)
+ % Verifies mexfcn throws an error if zero input arguments are
+ % passed it.
+ errID = "MATLAB:arrow:minrhs";
+ fcn = @()mexfcn;
+ testCase.verifyError(fcn, errID);
+ end
+
+ function InvalidFunctionNameDataTypeError(testCase)
+ % Verifies mexfcn throws an error if the first input argument is
+ % not a character vector.
+ errID = "MATLAB:arrow:FunctionNameDataType";
+ fcn = @()mexfcn(1);
+ testCase.verifyError(fcn, errID);
+
+ fcn = @()mexfcn(categorical);
+ testCase.verifyError(fcn, errID);
+
+ fcn = @()mexfcn(datetime(2021, 9, 10));
+ testCase.verifyError(fcn, errID);
+ end
+ end
+end
diff --git a/matlab/test/util/createVariablesAndMetadataStructs.m b/matlab/test/util/createVariablesAndMetadataStructs.m
index 0c60cbf..a26bfe4 100644
--- a/matlab/test/util/createVariablesAndMetadataStructs.m
+++ b/matlab/test/util/createVariablesAndMetadataStructs.m
@@ -17,7 +17,7 @@ function [variables, metadata] = createVariablesAndMetadataStructs()
% implied. See the License for the specific language governing
% permissions and limitations under the License.
-import mlarrow.util.*;
+import arrow.util.*;
type = 'uint8';
data = uint8([1; 2; 3]);
diff --git a/matlab/test/util/featherMEXRoundTrip.m b/matlab/test/util/featherMEXRoundTrip.m
index 49ab183..021e0f6 100644
--- a/matlab/test/util/featherMEXRoundTrip.m
+++ b/matlab/test/util/featherMEXRoundTrip.m
@@ -17,6 +17,6 @@ function [variablesOut, metadataOut] = featherMEXRoundTrip(filename, variablesIn
% implied. See the License for the specific language governing
% permissions and limitations under the License.
-featherwritemex(filename, variablesIn, metadataIn);
-[variablesOut, metadataOut] = featherreadmex(filename);
+mexfcn('featherwrite', filename, variablesIn, metadataIn);
+[variablesOut, metadataOut] = mexfcn('featherread', filename);
end
\ No newline at end of file
diff --git a/matlab/tools/UpdateMatlabSearchPath.cmake b/matlab/tools/UpdateMatlabSearchPath.cmake
new file mode 100644
index 0000000..178bb48
--- /dev/null
+++ b/matlab/tools/UpdateMatlabSearchPath.cmake
@@ -0,0 +1,54 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Convert Matlab_MAIN_PROGRAM, INSTALL_DIR and TOOLS_DIR to use OS native path notation.
+file(TO_NATIVE_PATH ${Matlab_MAIN_PROGRAM} NATIVE_MATLAB_MAIN_PROGRAM)
+file(TO_NATIVE_PATH ${INSTALL_DIR} NATIVE_INSTALL_DIR)
+file(TO_NATIVE_PATH ${TOOLS_DIR} NATIVE_TOOLS_DIR)
+
+# Initialize an instance of MATLAB and call the MATLAB function, addInstallDirToSearchPath,
+# defined in ${NATIVE_TOOLS_DIR}.
+# Flags to pass to MATLAB:
+# -sd: startup directory for the MATLAB
+# -batch: non-interactive script execution
+execute_process(COMMAND "${NATIVE_MATLAB_MAIN_PROGRAM}" -sd "${NATIVE_TOOLS_DIR}" -batch
+ "addInstallDirToSearchPath('${INSTALL_DIR}', '${MATLAB_ADD_INSTALL_DIR_TO_SEARCH_PATH}', '${MATLAB_ADD_INSTALL_DIR_TO_STARTUP_FILE}')"
+ RESULT_VARIABLE MATLAB_EXIT_CODE)
+
+if(MATLAB_EXIT_CODE EQUAL "1")
+ # Get path to the default MATLAB pathdef.m file.
+ set(MATLAB_PATHDEF_FILE "${Matlab_MAIN_PROGRAM}/toolbox/local/pathdef.m")
+ file(TO_NATIVE_PATH ${MATLAB_PATHDEF_FILE} NATIVE_MATLAB_PATHDEF_FILE)
+
+ message(FATAL_ERROR "Failed to add the installation directory, ${NATIVE_INSTALL_DIR}, to the MATLAB Search Path. This may be due to the current user lacking the necessary filesystem permissions to modify ${NATIVE_MATLAB_PATHDEF_FILE}. In order to complete the installation process, ${NATIVE_INSTALL_DIR} must be added to the MATLAB Search Path using the \"addpath\" and \"savepath\" MATLAB commands or by resolving the permissions issues and re-running the CMake install target."
+ )
+endif()
+
+if(MATLAB_EXIT_CODE EQUAL "2")
+ message(FATAL_ERROR "Failed to append to the MATLAB startup.m file located at the MATLAB userpath directory. fopen() failed to open the file. In order to complete the installation process, ${NATIVE_INSTALL_DIR} must be added to the MATLAB Search Path using the \"addpath\" and \"savepath\" MATLAB commands."
+ )
+endif()
+
+if(MATLAB_EXIT_CODE EQUAL "3")
+ message(FATAL_ERROR "Failed to append to the MATLAB startup.m file located at the MATLAB userpath directory. fwrite() failed to write to the file. In order to complete the installation process, ${NATIVE_INSTALL_DIR} must be added to the MATLAB Search Path using the \"addpath\" and \"savepath\" MATLAB commands."
+ )
+endif()
+
+if(MATLAB_EXIT_CODE EQUAL "4")
+ message(FATAL_ERROR "Failed to append to the MATLAB startup.m file located at the MATLAB userpath directory. fclose() failed to close the file. In order to complete the installation process, ${NATIVE_INSTALL_DIR} must be added to the MATLAB Search Path using the \"addpath\" and \"savepath\" MATLAB commands."
+ )
+endif()
diff --git a/matlab/tools/addInstallDirToSearchPath.m b/matlab/tools/addInstallDirToSearchPath.m
new file mode 100644
index 0000000..6725343
--- /dev/null
+++ b/matlab/tools/addInstallDirToSearchPath.m
@@ -0,0 +1,54 @@
+function addInstallDirToSearchPath(installDirPath, addInstallDirToSearchPath, addInstallDirToStartupFile)
+ % addInstallDirToSearchPath Add the input path, INSTALLDIRPATH, to the
+ % MATLAB Search Path and save.
+ %
+ % Licensed to the Apache Software Foundation (ASF) under one or more
+ % contributor license agreements. See the NOTICE file distributed with
+ % this work for additional information regarding copyright ownership.
+ % The ASF licenses this file to you under the Apache License, Version
+ % 2.0 (the "License"); you may not use this file except in compliance
+ % with the License. You may obtain a copy of the License at
+ %
+ % http://www.apache.org/licenses/LICENSE-2.0
+ %
+ % Unless required by applicable law or agreed to in writing, software
+ % distributed under the License is distributed on an "AS IS" BASIS,
+ % WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ % implied. See the License for the specific language governing
+ % permissions and limitations under the License.
+
+ if addInstallDirToSearchPath == "ON"
+ addpath(installDirPath);
+ status = savepath(fullfile(matlabroot, "toolbox", "local", "pathdef.m"));
+
+ % Return exit code 1 to indicate savepath failure and 0 to indicate the path has
+ % been saved successfully.
+ if status == 0
+ disp("Sucessfully added installation directory to the MATLAB Search Path: " + installDirPath);
+ quit(0);
+ else
+ quit(1);
+ end
+ end
+
+ if addInstallDirToStartupFile == "ON"
+ fid = fopen(fullfile(userpath, "startup.m"), "a");
+ if fid > 2
+ count = fwrite(fid, "addpath(""" + installDirPath + """);");
+ if count == 0
+ % fwrite failed.
+ quit(3);
+ end
+ status = fclose(fid);
+ if status ~= 0
+ % fclose failed.
+ quit(4);
+ end
+ else
+ % fopen failed.
+ quit(2);
+ end
+ disp("Sucessfully appended an addpath command to the MATLAB startup.m file located at the userpath to add installation directory to the MATLAB Search Path: " + installDirPath);
+ quit(0);
+ end
+end