[GitHub] [arrow] pchaoda opened a new issue #12743: [Python]OSError: Unable to load libjvm. when used in windows

[GitBox <gi...@apache.org>]

pchaoda opened a new issue #12743:
URL: https://github.com/apache/arrow/issues/12743


   Hi all,
   I am using pyarrow==7.0.0 to connect hdfs.
   It run well with linux,but unfortunately get error in windows.
   I have set `JAVA_HOME`, `HADOOPHOME`,`ARROW_LIBHDFS_DIR`
   JAVA_HOME=C:\Users\think\Desktop\python-SDK-green\jdk18
   HADOOP_HOME=C:\Users\think\Downloads\hadoop-2.10.1.tar\hadoop-2.10.1\hadoop-2.10.1
   ARROW_LIBHDFS_DIR=C:\Users\think\Desktop\python-SDK-green\hadoop_client\lib\native;C:\Users\think\Desktop\python-SDK-green\jdk18\jre\bin\server;
   when I am using pyarrow.hdfs.connect(), I am getting the error:
   ```bash
   Traceback (most recent call last):
     File "C:\Users\think\Desktop\python-SDK-green\python\test.py", line 7, in <module>
       data_provider = DataProvider()
     File "C:\Users\think\Desktop\python-SDK-green\python\lib\site-packages\nescqdata\MarketData\dataProvider.py", line 15, in __init__
       super(DataProvider, self).__init__(dfs)
     File "C:\Users\think\Desktop\python-SDK-green\python\lib\site-packages\nescqdata\baseDataProvider.py", line 53, in __init__
       self.dfs = pa.hdfs.connect() if dfs is None else dfs
     File "C:\Users\think\Desktop\python-SDK-green\python\lib\site-packages\pyarrow\hdfs.py", line 227, in connect
       return _connect(
     File "C:\Users\think\Desktop\python-SDK-green\python\lib\site-packages\pyarrow\hdfs.py", line 237, in _connect
       fs = HadoopFileSystem(host=host, port=port, user=user,
     File "C:\Users\think\Desktop\python-SDK-green\python\lib\site-packages\pyarrow\hdfs.py", line 49, in __init__
       self._connect(host, port, user, kerb_ticket, extra_conf)
     File "pyarrow\_hdfsio.pyx", line 85, in pyarrow._hdfsio.HadoopFileSystem._connect
     File "pyarrow\error.pxi", line 114, in pyarrow.lib.check_status
   OSError: Unable to load libjvm: �Ҳ���ָ����ģ�顣
   ```
   and by the way, before I got this error,hdfs.py was modified to avoid another problem by add shell=True
   ```bash
     File "C:\Users\think\Desktop\python-SDK-green\python\lib\site-packages\pyarrow\hdfs.py", line 145, in _maybe_set_hadoop_classpath
       classpath = _hadoop_classpath_glob(hadoop_bin)
     File "C:\Users\think\Desktop\python-SDK-green\python\lib\site-packages\pyarrow\hdfs.py", line 172, in _hadoop_classpath_glob
       return subprocess.check_output(hadoop_classpath_args)
     File "subprocess.py", line 424, in check_output
       return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
     File "subprocess.py", line 505, in run
       with Popen(*popenargs, **kwargs) as process:
     File "subprocess.py", line 951, in __init__
       self._execute_child(args, executable, preexec_fn, close_fds,
     File "subprocess.py", line 1420, in _execute_child
       hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
   OSError: [WinError 193] %1 不是有效的 Win32 应用程序。
   ```
   Thank you !!!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pchaoda commented on issue #12743: [Python]OSError: Unable to load libjvm. when used in windows

[GitBox <gi...@apache.org>]

pchaoda commented on issue #12743:
URL: https://github.com/apache/arrow/issues/12743#issuecomment-1081553531


   ```c
   #ifdef _WIN32
     ARROW_ASSIGN_OR_RAISE(search_prefixes, MakeFilenameVector({""}));
     ARROW_ASSIGN_OR_RAISE(search_suffixes,
                           MakeFilenameVector({"/jre/bin/server", "/bin/server"}));
     file_name = "jvm.dll";
   ```
   in `/jre/bin/server` directory exist jvm.dll
   OSError: Unable to load libjvm: �Ҳ���ָ����ģ�顣
   and I can not tell what is the code after `OSError: Unable to load libjvm:`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org