You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/04/07 02:46:25 UTC
[GitHub] [druid] maytasm opened a new pull request #9632: Test file format
extensions for inputSource (orc, parquet)
maytasm opened a new pull request #9632: Test file format extensions for inputSource (orc, parquet)
URL: https://github.com/apache/druid/pull/9632
Test file format extensions for inputSource (orc, parquet)
### Description
This PR adds integration test coverage on inputSource using inputFormat of type ORC and Parquet. Since the different inputSources share code from AbstractInputSource, it is sufficient to add these new integration tests to just one inputSource. I have chosen to add these new integration tests to the hdfs inputSource. (Note that other inputSource such as S3, GCS, Azure, hdfs, local already have integration tests covering inputFormat of json type.)
The benefits of putting these new integration tests in the hdfs inputSource are
- 1) It is possible to add hdfs inputSource to our Travis/CI as hadoop cluster along with the necessary data files can be setup with the docker test script (while it is possible for cloud input sources)
- 2) It is easier to maintain and setup complex data files directory that is needed for the ingestionSpec using the docker test script (while it is more prone to error if left to user to setup manually)
This PR has:
- [x] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/licenses.yaml)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths.
- [x] added integration tests.
- [ ] been tested in a test Druid cluster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis merged pull request #9632: Test file format
extensions for inputSource (orc, parquet)
Posted by GitBox <gi...@apache.org>.
clintropolis merged pull request #9632: Test file format extensions for inputSource (orc, parquet)
URL: https://github.com/apache/druid/pull/9632
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org