You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "Daniel Rodriguez (JIRA)" <ji...@apache.org> on 2019/01/20 04:55:00 UTC

[jira] [Commented] (ARROW-4208) [CI/Python] Have automatized tests for S3

    [ https://issues.apache.org/jira/browse/ARROW-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16747342#comment-16747342 ] 

Daniel Rodriguez commented on ARROW-4208:
-----------------------------------------

Another option here would be to use [minio|https://www.minio.io/] (an S3 API compatible server). I have use is successfully to test [s3contents|https://github.com/danielfrg/s3contents/].

> [CI/Python] Have automatized tests for S3
> -----------------------------------------
>
>                 Key: ARROW-4208
>                 URL: https://issues.apache.org/jira/browse/ARROW-4208
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Continuous Integration, Python
>            Reporter: Krisztian Szucs
>            Priority: Major
>              Labels: s3
>
> Currently We don't run S3 integration tests regularly. 
> Possible solutions:
> - mock it within python/pytest
> - simply run the s3 tests with an S3 credential provided
> - create a hdfs-integration like docker-compose setup and run an S3 mock server (e.g.: https://github.com/adobe/S3Mock, https://github.com/jubos/fake-s3, https://github.com/gaul/s3proxy, https://github.com/jserver/mock-s3)
> For more see discussion https://github.com/apache/arrow/pull/3286



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)