[jira] [Created] (ARROW-2872) [Python] Add pytest mark to opt into TensorFlow unit tests

["Wes McKinney (JIRA)" <ji...@apache.org>]

Wes McKinney created ARROW-2872:
-----------------------------------

             Summary: [Python] Add pytest mark to opt into TensorFlow unit tests
                 Key: ARROW-2872
                 URL: https://issues.apache.org/jira/browse/ARROW-2872
             Project: Apache Arrow
          Issue Type: Bug
          Components: Python
            Reporter: Wes McKinney
             Fix For: 0.10.0


After pulling in ARROW-1744, I found this a little bit unfriendly: 

{code}
pyarrow/tests/test_plasma_tf_op.py::test_plasma_tf_op FAILED                   [ 82%]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured stdout >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
TensorFlow version: 1.8.0
Compiling Plasma TensorFlow Op...
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured stderr >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+++ dirname /home/wesm/code/arrow/python/pyarrow/tensorflow/build.sh
++ cd /home/wesm/code/arrow/python/pyarrow/tensorflow
++ pwd
+ PYARROW_TENSORFLOW_DIR=/home/wesm/code/arrow/python/pyarrow/tensorflow
++ python -c 'import tensorflow as tf; print(" ".join(tf.sysconfig.get_compile_flags()))'
+ TF_CFLAGS='-I/home/wesm/miniconda/envs/arrow-dev/lib/python3.6/site-packages/tensorflow/include -D_GLIBCXX_USE_CXX11_ABI=0'
++ python -c 'import tensorflow as tf; print(" ".join(tf.sysconfig.get_link_flags()))'
+ TF_LFLAGS='-L/home/wesm/miniconda/envs/arrow-dev/lib/python3.6/site-packages/tensorflow -ltensorflow_framework'
++ uname
+ '[' Linux == Darwin ']'
+ NDEBUG=-DNDEBUG
++ pkg-config --cflags --libs plasma arrow arrow-python
+ g++ -std=c++11 -g -shared /home/wesm/code/arrow/python/pyarrow/tensorflow/plasma_op.cc -o /home/wesm/code/arrow/python/pyarrow/tensorflow/plasma_op.so -DNDEBUG -I/home/wesm/local/include -I/home/wesm/miniconda/envs/arrow-dev/include/python3.6m -I/home/wesm/local/include -L/home/wesm/local/lib -lplasma -larrow_python -larrow -fPIC -I/home/wesm/miniconda/envs/arrow-dev/lib/python3.6/site-packages/tensorflow/include -D_GLIBCXX_USE_CXX11_ABI=0 -L/home/wesm/miniconda/envs/arrow-dev/lib/python3.6/site-packages/tensorflow -ltensorflow_framework -O2
/home/wesm/code/arrow/python/pyarrow/tensorflow/plasma_op.cc:33:10: fatal error: arrow/adapters/tensorflow/convert.h: No such file or directory
 #include "arrow/adapters/tensorflow/convert.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

use_gpu = False

    @pytest.mark.plasma
    def test_plasma_tf_op(use_gpu=False):
        import pyarrow.plasma as plasma
    
>       plasma.build_plasma_tensorflow_op()

pyarrow/tests/test_plasma_tf_op.py:89: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pyarrow/plasma.py:56: in build_plasma_tensorflow_op
    subprocess.check_call(["bash", script_path])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

popenargs = (['bash', '/home/wesm/code/arrow/python/pyarrow/tensorflow/build.sh'],)
kwargs = {}, retcode = 1
cmd = ['bash', '/home/wesm/code/arrow/python/pyarrow/tensorflow/build.sh']

    def check_call(*popenargs, **kwargs):
        """Run command with arguments.  Wait for command to complete.  If
        the exit code was zero then return, otherwise raise
        CalledProcessError.  The CalledProcessError object will have the
        return code in the returncode attribute.
    
        The arguments are the same as for the call function.  Example:
    
        check_call(["ls", "-l"])
        """
        retcode = call(*popenargs, **kwargs)
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
>           raise CalledProcessError(retcode, cmd)
E           subprocess.CalledProcessError: Command '['bash', '/home/wesm/code/arrow/python/pyarrow/tensorflow/build.sh']' returned non-zero exit status 1.

../../../miniconda/envs/arrow-dev/lib/python3.6/subprocess.py:291: CalledProcessError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/wesm/miniconda/envs/arrow-dev/lib/python3.6/subprocess.py(291)check_call()
-> raise CalledProcessError(retcode, cmd)
{code}

If you pass {{-DARROW_PLASMA=ON}} but do not also pass {{-DARROW_TENSORFLOW=ON}} then this occurs



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)