You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@arrow.apache.org by Tom Augspurger <to...@gmail.com> on 2019/11/26 14:34:58 UTC

Is FileSystem._isfilestore considered public?

Hi,

In https://github.com/dask/dask/issues/5526, we're seeing an issue stemming
from a hack to ensure compatibility for Pyarrow. The details aren't too
important. The core of the issue is that the Pyarrow parquet writer makes a
couple checks for `FileSystem._isfilestore` via `_mkdir_if_not_exists`,
e.g. in
https://github.com/apache/arrow/blob/207b3507be82e92ebf29ec7d6d3b0bb86091c09a/python/pyarrow/parquet.py#L1349-L1350
.

Is it OK for my FileSystem subclass to override _isfilestore? Is it
considered public?

Thanks,

Tom

Re: Is FileSystem._isfilestore considered public?

Posted by Antoine Pitrou <an...@python.org>.

Generally speaking, this API is obsolete (though not formally deprecated
yet).  So we don't envision to change it significantly in the future.

We hope that in the near future the near pyarrow FileSystem API will be
usable directly pyarrow.parquet.

Regards

Antoine.


Le 26/11/2019 à 15:34, Tom Augspurger a écrit :
> Hi,
> 
> In https://github.com/dask/dask/issues/5526, we're seeing an issue stemming
> from a hack to ensure compatibility for Pyarrow. The details aren't too
> important. The core of the issue is that the Pyarrow parquet writer makes a
> couple checks for `FileSystem._isfilestore` via `_mkdir_if_not_exists`,
> e.g. in
> https://github.com/apache/arrow/blob/207b3507be82e92ebf29ec7d6d3b0bb86091c09a/python/pyarrow/parquet.py#L1349-L1350
> .
> 
> Is it OK for my FileSystem subclass to override _isfilestore? Is it
> considered public?
> 
> Thanks,
> 
> Tom
>