You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kudu.apache.org by Adar Lieber-Dembo <ad...@cloudera.com> on 2017/10/06 22:53:40 UTC
Re: RFC: using C++ mini cluster implementation in Java and Python tests
To close the loop on this, I just merged a new action for the Kudu CLI
which can be used to create and control mini-clusters. Here's the
commit: https://github.com/apache/kudu/commit/761ce10b797c4f6345f0b84cfe30bcc92e346893
I also have a patch out for review that adapts the Java tests to use
this new action, and I expect that to be merged soon. If anyone's
interested in doing the same for the Python tests, I'm happy to review
your work.
On Mon, Aug 28, 2017 at 7:53 AM, Jordan Birdsell
<jo...@gmail.com> wrote:
> Agreed, I too am hoping to have some time free up again in the near future
> for additional work on the python client.
>
> On Mon, Aug 28, 2017 at 10:26 AM Wes McKinney <we...@gmail.com> wrote:
>
>> +1. I hope to find some time in the next 6-12 months to hack on the
>> Python client and add Arrow support at the C++ client level for faster
>> scans and inserts, and making testing simpler and more consistent
>> would make things easier to test across C++ and Python.
>>
>> - Wes
>>
>> On Mon, Aug 28, 2017 at 2:22 AM, Adar Lieber-Dembo <ad...@cloudera.com>
>> wrote:
>> > Over the weekend I've been hacking on an idea that's been discussed in
>> > the past: replacing our language-specific mini cluster support with
>> > the more robust and featureful C++ mini cluster. Besides reducing the
>> > maintenance burden for non-C++ language support, this has the added
>> > benefit of providing a single deliverable that can set up and tear
>> > down mini clusters for use in tests, something that may be useful to
>> > other projects wishing to integrate with Kudu.
>> >
>> > Since this affects a pretty broad swath of Kudu developers, I thought
>> > it'd be prudent to broadcast the idea to a larger audience. If you're
>> > interested, please have a look at my proof of concept, which
>> > demonstrates the idea for Java unit tests:
>> > http://gerrit.cloudera.org:8080/7853.
>>
Re: RFC: using C++ mini cluster implementation in Java and Python tests
Posted by Adar Lieber-Dembo <ad...@cloudera.com>.
The Python patch wasn't as much as work as I thought, so I ended up
doing it myself: https://gerrit.cloudera.org/c/8236.
On Fri, Oct 6, 2017 at 3:53 PM, Adar Lieber-Dembo <ad...@cloudera.com> wrote:
> To close the loop on this, I just merged a new action for the Kudu CLI
> which can be used to create and control mini-clusters. Here's the
> commit: https://github.com/apache/kudu/commit/761ce10b797c4f6345f0b84cfe30bcc92e346893
>
> I also have a patch out for review that adapts the Java tests to use
> this new action, and I expect that to be merged soon. If anyone's
> interested in doing the same for the Python tests, I'm happy to review
> your work.
>
>
> On Mon, Aug 28, 2017 at 7:53 AM, Jordan Birdsell
> <jo...@gmail.com> wrote:
>> Agreed, I too am hoping to have some time free up again in the near future
>> for additional work on the python client.
>>
>> On Mon, Aug 28, 2017 at 10:26 AM Wes McKinney <we...@gmail.com> wrote:
>>
>>> +1. I hope to find some time in the next 6-12 months to hack on the
>>> Python client and add Arrow support at the C++ client level for faster
>>> scans and inserts, and making testing simpler and more consistent
>>> would make things easier to test across C++ and Python.
>>>
>>> - Wes
>>>
>>> On Mon, Aug 28, 2017 at 2:22 AM, Adar Lieber-Dembo <ad...@cloudera.com>
>>> wrote:
>>> > Over the weekend I've been hacking on an idea that's been discussed in
>>> > the past: replacing our language-specific mini cluster support with
>>> > the more robust and featureful C++ mini cluster. Besides reducing the
>>> > maintenance burden for non-C++ language support, this has the added
>>> > benefit of providing a single deliverable that can set up and tear
>>> > down mini clusters for use in tests, something that may be useful to
>>> > other projects wishing to integrate with Kudu.
>>> >
>>> > Since this affects a pretty broad swath of Kudu developers, I thought
>>> > it'd be prudent to broadcast the idea to a larger audience. If you're
>>> > interested, please have a look at my proof of concept, which
>>> > demonstrates the idea for Java unit tests:
>>> > http://gerrit.cloudera.org:8080/7853.
>>>