[jira] [Created] (ARROW-7965) [Python] Hold a reference to the dataset factory for later reuse

["Krisztian Szucs (Jira)" <ji...@apache.org>]

Krisztian Szucs created ARROW-7965:
--------------------------------------

             Summary: [Python] Hold a reference to the dataset factory for later reuse
                 Key: ARROW-7965
                 URL: https://issues.apache.org/jira/browse/ARROW-7965
             Project: Apache Arrow
          Issue Type: Improvement
          Components: Python
            Reporter: Krisztian Szucs


Provide a more intuitive way to construct nested dataset:

```python
# instead of using confusing factory function
dataset([
     factory("s3://old-taxi-data", format="parquet"),
     factory("local/path/to/new/data", format="csv")
])

# let the user to construct a new dataset directly from dataset objects
dataset([ 
    dataset("s3://old-taxi-data", format="parquet"),
    dataset("local/path/to/new/data", format="csv")
])
```

In the future we might want to introduce a new Dataset class which wraps functionality of both the dataset actory and the materialized dataset enabling optimizations over rediscovery of already materialized datasets. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)