You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "nero (Jira)" <ji...@apache.org> on 2021/11/22 07:52:00 UTC
[jira] [Updated] (ARROW-14787) read an HDFS file by line failed when the open_mode is "rb"
[ https://issues.apache.org/jira/browse/ARROW-14787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
nero updated ARROW-14787:
-------------------------
Description:
Hi there,
I found some problems when I use `{*}fsspec`{*} to read an HDFS file by line when the open_mode is "rb". It works fine when the *open_mode is "r"* or the {*}file is located locally{*}.
some snippets:
{code:java}
import fsspec
hdfs_file_path = "hdfs://xxxxxx"
with fsspec.open(hdfs_file_path, "rb") as f:
# raise UnspportedOperation
f.readline() {code}
Error logs:
/opt/conda/lib/python3.7/site-packages/pyarrow/io.pxi in pyarrow.lib.NativeFile.readline()
UnsupportedOperation:
was:
Hi there,
I found some problems when I use `{*}fsspec`{*} to read an HDFS file by line when the open_mode is "rb". It works fine when the *open_mode is "r"* or the {*}file is located locally{*}.
some snippets:
{code:java}
import fsspec
hdfs_file_path = "hdfs://xxxxxx"
with fsspec.open(hdfs_file_path, "rb") as f:
# raise UnspportedOperation
f.readline() {code}
Error logs:
/opt/conda/lib/python3.7/site-packages/pyarrow/io.pxi in pyarrow.lib.NativeFile.readline()
UnsupportedOperation:
> read an HDFS file by line failed when the open_mode is "rb"
> -----------------------------------------------------------
>
> Key: ARROW-14787
> URL: https://issues.apache.org/jira/browse/ARROW-14787
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 6.0.0
> Environment: System: Ubuntu 18.04
> fsspec: 2021.10.1
> pyarrow: 6.0.0
> Reporter: nero
> Priority: Blocker
>
> Hi there,
> I found some problems when I use `{*}fsspec`{*} to read an HDFS file by line when the open_mode is "rb". It works fine when the *open_mode is "r"* or the {*}file is located locally{*}.
> some snippets:
> {code:java}
> import fsspec
> hdfs_file_path = "hdfs://xxxxxx"
> with fsspec.open(hdfs_file_path, "rb") as f:
> # raise UnspportedOperation
> f.readline() {code}
>
> Error logs:
>
> /opt/conda/lib/python3.7/site-packages/pyarrow/io.pxi in pyarrow.lib.NativeFile.readline()
> UnsupportedOperation:
--
This message was sent by Atlassian Jira
(v8.20.1#820001)