You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Cory Monty <co...@getbraintree.com> on 2016/02/01 18:40:59 UTC
DataStreamUtils and Scala
Hey there,
We were using DataStreamUtils.collect in Scala for automated testing, which
only works because of `DataStream.getJavaStream` accessor in the Scala
version of `DataStream`. However, a recent commit (
https://github.com/apache/flink/commit/086acf681f01f2da530c04289e0682c56f98a378)
removed `getJavaStream` and replaced the method with a private version that
we can no longer access from our tests.
Do you have an alternative version of DataStreamUtils that does not require
a Java DataStream?
Cheers,
Cory
Re: DataStreamUtils and Scala
Posted by Márton Balassi <ba...@gmail.com>.
Opened the PR. [1] Will merge the re-add "getJavaStream()" method commit
as soon as travis passes if no objections, the second approach can be
discussed on github.
[1] https://github.com/apache/flink/pull/1574
Best,
Marton
On Mon, Feb 1, 2016 at 10:56 PM, Márton Balassi <ba...@gmail.com>
wrote:
> I'll do the fix(es) tomorrow morning.
>
> Best,
>
> Marton
>
> On Mon, Feb 1, 2016, 20:05 Cory Monty <co...@getbraintree.com> wrote:
>
>> Thanks for the quick response!
>>
>> Either solution works for us.
>>
>> On Mon, Feb 1, 2016 at 12:07 PM, Stephan Ewen <se...@apache.org> wrote:
>>
>>> I would actually re-add the "getJavaStream()" method. There are probably
>>> other cases when people need it, and it does not hurt to expose it.
>>>
>>> Also, it allows you to combine programs written partly against the Java
>>> and Scala API (if you would ever want to do that).
>>>
>>> Stephan
>>>
>>>
>>> On Mon, Feb 1, 2016 at 6:53 PM, Márton Balassi <balassi.marton@gmail.com
>>> > wrote:
>>>
>>>> Hey Cory,
>>>>
>>>> Sorry, I did not mean to break your code. One solution that I could
>>>> suggest is to do it the way we have it for the batch api, namely having a
>>>> scala version for DataStreamUtils too. It might be placed under
>>>> flink-contrib for the time being.
>>>>
>>>> Would that solution fit your needs?
>>>>
>>>> On Mon, Feb 1, 2016 at 6:40 PM, Cory Monty <cory.monty@getbraintree.com
>>>> > wrote:
>>>>
>>>>> Hey there,
>>>>>
>>>>> We were using DataStreamUtils.collect in Scala for automated testing,
>>>>> which only works because of `DataStream.getJavaStream` accessor in the
>>>>> Scala version of `DataStream`. However, a recent commit (
>>>>> https://github.com/apache/flink/commit/086acf681f01f2da530c04289e0682c56f98a378)
>>>>> removed `getJavaStream` and replaced the method with a private version that
>>>>> we can no longer access from our tests.
>>>>>
>>>>> Do you have an alternative version of DataStreamUtils that does not
>>>>> require a Java DataStream?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Cory
>>>>>
>>>>
>>>>
>>>
>>
Re: DataStreamUtils and Scala
Posted by Márton Balassi <ba...@gmail.com>.
I'll do the fix(es) tomorrow morning.
Best,
Marton
On Mon, Feb 1, 2016, 20:05 Cory Monty <co...@getbraintree.com> wrote:
> Thanks for the quick response!
>
> Either solution works for us.
>
> On Mon, Feb 1, 2016 at 12:07 PM, Stephan Ewen <se...@apache.org> wrote:
>
>> I would actually re-add the "getJavaStream()" method. There are probably
>> other cases when people need it, and it does not hurt to expose it.
>>
>> Also, it allows you to combine programs written partly against the Java
>> and Scala API (if you would ever want to do that).
>>
>> Stephan
>>
>>
>> On Mon, Feb 1, 2016 at 6:53 PM, Márton Balassi <ba...@gmail.com>
>> wrote:
>>
>>> Hey Cory,
>>>
>>> Sorry, I did not mean to break your code. One solution that I could
>>> suggest is to do it the way we have it for the batch api, namely having a
>>> scala version for DataStreamUtils too. It might be placed under
>>> flink-contrib for the time being.
>>>
>>> Would that solution fit your needs?
>>>
>>> On Mon, Feb 1, 2016 at 6:40 PM, Cory Monty <co...@getbraintree.com>
>>> wrote:
>>>
>>>> Hey there,
>>>>
>>>> We were using DataStreamUtils.collect in Scala for automated testing,
>>>> which only works because of `DataStream.getJavaStream` accessor in the
>>>> Scala version of `DataStream`. However, a recent commit (
>>>> https://github.com/apache/flink/commit/086acf681f01f2da530c04289e0682c56f98a378)
>>>> removed `getJavaStream` and replaced the method with a private version that
>>>> we can no longer access from our tests.
>>>>
>>>> Do you have an alternative version of DataStreamUtils that does not
>>>> require a Java DataStream?
>>>>
>>>> Cheers,
>>>>
>>>> Cory
>>>>
>>>
>>>
>>
>
Re: DataStreamUtils and Scala
Posted by Cory Monty <co...@getbraintree.com>.
Thanks for the quick response!
Either solution works for us.
On Mon, Feb 1, 2016 at 12:07 PM, Stephan Ewen <se...@apache.org> wrote:
> I would actually re-add the "getJavaStream()" method. There are probably
> other cases when people need it, and it does not hurt to expose it.
>
> Also, it allows you to combine programs written partly against the Java
> and Scala API (if you would ever want to do that).
>
> Stephan
>
>
> On Mon, Feb 1, 2016 at 6:53 PM, Márton Balassi <ba...@gmail.com>
> wrote:
>
>> Hey Cory,
>>
>> Sorry, I did not mean to break your code. One solution that I could
>> suggest is to do it the way we have it for the batch api, namely having a
>> scala version for DataStreamUtils too. It might be placed under
>> flink-contrib for the time being.
>>
>> Would that solution fit your needs?
>>
>> On Mon, Feb 1, 2016 at 6:40 PM, Cory Monty <co...@getbraintree.com>
>> wrote:
>>
>>> Hey there,
>>>
>>> We were using DataStreamUtils.collect in Scala for automated testing,
>>> which only works because of `DataStream.getJavaStream` accessor in the
>>> Scala version of `DataStream`. However, a recent commit (
>>> https://github.com/apache/flink/commit/086acf681f01f2da530c04289e0682c56f98a378)
>>> removed `getJavaStream` and replaced the method with a private version that
>>> we can no longer access from our tests.
>>>
>>> Do you have an alternative version of DataStreamUtils that does not
>>> require a Java DataStream?
>>>
>>> Cheers,
>>>
>>> Cory
>>>
>>
>>
>
Re: DataStreamUtils and Scala
Posted by Stephan Ewen <se...@apache.org>.
I would actually re-add the "getJavaStream()" method. There are probably
other cases when people need it, and it does not hurt to expose it.
Also, it allows you to combine programs written partly against the Java and
Scala API (if you would ever want to do that).
Stephan
On Mon, Feb 1, 2016 at 6:53 PM, Márton Balassi <ba...@gmail.com>
wrote:
> Hey Cory,
>
> Sorry, I did not mean to break your code. One solution that I could
> suggest is to do it the way we have it for the batch api, namely having a
> scala version for DataStreamUtils too. It might be placed under
> flink-contrib for the time being.
>
> Would that solution fit your needs?
>
> On Mon, Feb 1, 2016 at 6:40 PM, Cory Monty <co...@getbraintree.com>
> wrote:
>
>> Hey there,
>>
>> We were using DataStreamUtils.collect in Scala for automated testing,
>> which only works because of `DataStream.getJavaStream` accessor in the
>> Scala version of `DataStream`. However, a recent commit (
>> https://github.com/apache/flink/commit/086acf681f01f2da530c04289e0682c56f98a378)
>> removed `getJavaStream` and replaced the method with a private version that
>> we can no longer access from our tests.
>>
>> Do you have an alternative version of DataStreamUtils that does not
>> require a Java DataStream?
>>
>> Cheers,
>>
>> Cory
>>
>
>
Re: DataStreamUtils and Scala
Posted by Márton Balassi <ba...@gmail.com>.
Hey Cory,
Sorry, I did not mean to break your code. One solution that I could suggest
is to do it the way we have it for the batch api, namely having a scala
version for DataStreamUtils too. It might be placed under flink-contrib for
the time being.
Would that solution fit your needs?
On Mon, Feb 1, 2016 at 6:40 PM, Cory Monty <co...@getbraintree.com>
wrote:
> Hey there,
>
> We were using DataStreamUtils.collect in Scala for automated testing,
> which only works because of `DataStream.getJavaStream` accessor in the
> Scala version of `DataStream`. However, a recent commit (
> https://github.com/apache/flink/commit/086acf681f01f2da530c04289e0682c56f98a378)
> removed `getJavaStream` and replaced the method with a private version that
> we can no longer access from our tests.
>
> Do you have an alternative version of DataStreamUtils that does not
> require a Java DataStream?
>
> Cheers,
>
> Cory
>