You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Kenneth Knowles <ke...@apache.org> on 2019/02/15 19:29:37 UTC
[ANNOUNCE] Apache Beam 2.10.0 released!
The Apache Beam team is pleased to announce the release of version 2.10.0!
Apache Beam is an open source unified programming model to define and
execute data processing pipelines, including ETL, batch and stream
(continuous) processing. See https://beam.apache.org
You can download the release here:
https://beam.apache.org/get-started/downloads/
This release includes bugfixes, features, and improvements detailed on the
Beam blog: https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html
Thanks to everyone who contributed to this release, and we hope you enjoy
using Beam 2.10.0.
-- Kenneth Knowles, on behalf of The Apache Beam team
Re: [ANNOUNCE] Apache Beam 2.10.0 released!
Posted by Alexey Romanenko <ar...@gmail.com>.
Hi Kant,
Not yet, the work is in progress. Feel free to join a channel #beam-spark on Slack for more updates/details.
> On 18 Feb 2019, at 12:30, kant kodali <ka...@gmail.com> wrote:
>
> Great! Does this version include Spark Structured Streaming?
>
> On Mon, Feb 18, 2019 at 3:23 AM Matt Casters <mattcasters@gmail.com <ma...@gmail.com>> wrote:
> Hi Beam-ers,
>
> Congratulations on the release!
> I did a quick upgrade and tried to run my test ETL jobs on Direct & Flink runners with great success.
> DataFlow however is throwing all sorts of errors. For example:
>
> * Handler for GET /v1.27/images/gcr.io/cloud-dataflow/v1beta3/beam-java-batch:beam-2.10.0/json <http://gcr.io/cloud-dataflow/v1beta3/beam-java-batch:beam-2.10.0/json> returned error: No such image: gcr.io/cloud-dataflow/v1beta3/beam-java-batch:beam-2.10.0 <http://gcr.io/cloud-dataflow/v1beta3/beam-java-batch:beam-2.10.0>"
> * while reading 'google-dockercfg' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg <http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg>"
> * Error syncing pod..."
>
> The job gets stuck after starting a worker and after an hour or so it gives up with a failure. 2.9.0 runs fine.
>
> I realize I've been quick to update this weekend but it would be great if I could get a feel of whether I'm messing up something in my Maven pom.xml or if integration with Beam 2.10.0 isn't done yet on DataFlow.
>
> Thanks in advance!
>
> Matt
>
>
>
> Op vr 15 feb. 2019 om 20:29 schreef Kenneth Knowles <kenn@apache.org <ma...@apache.org>>:
> The Apache Beam team is pleased to announce the release of version 2.10.0!
>
> Apache Beam is an open source unified programming model to define and
> execute data processing pipelines, including ETL, batch and stream
> (continuous) processing. See https://beam.apache.org <https://beam.apache.org/>
>
> You can download the release here:
>
> https://beam.apache.org/get-started/downloads/ <https://beam.apache.org/get-started/downloads/>
>
> This release includes bugfixes, features, and improvements detailed on the Beam blog: https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html>
>
> Thanks to everyone who contributed to this release, and we hope you enjoy
> using Beam 2.10.0.
>
> -- Kenneth Knowles, on behalf of The Apache Beam team
Re: [ANNOUNCE] Apache Beam 2.10.0 released!
Posted by kant kodali <ka...@gmail.com>.
Great! Does this version include Spark Structured Streaming?
On Mon, Feb 18, 2019 at 3:23 AM Matt Casters <ma...@gmail.com> wrote:
> Hi Beam-ers,
>
> Congratulations on the release!
> I did a quick upgrade and tried to run my test ETL jobs on Direct & Flink
> runners with great success.
> DataFlow however is throwing all sorts of errors. For example:
>
> * Handler for GET /v1.27/images/
> gcr.io/cloud-dataflow/v1beta3/beam-java-batch:beam-2.10.0/json returned
> error: No such image:
> gcr.io/cloud-dataflow/v1beta3/beam-java-batch:beam-2.10.0"
> * while reading 'google-dockercfg' metadata: http status code: 404 while
> fetching url
> http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
> "
> * Error syncing pod..."
>
> The job gets stuck after starting a worker and after an hour or so it
> gives up with a failure. 2.9.0 runs fine.
>
> I realize I've been quick to update this weekend but it would be great if
> I could get a feel of whether I'm messing up something in my Maven pom.xml
> or if integration with Beam 2.10.0 isn't done yet on DataFlow.
>
> Thanks in advance!
>
> Matt
>
>
>
> Op vr 15 feb. 2019 om 20:29 schreef Kenneth Knowles <ke...@apache.org>:
>
>> The Apache Beam team is pleased to announce the release of version 2.10.0!
>>
>> Apache Beam is an open source unified programming model to define and
>> execute data processing pipelines, including ETL, batch and stream
>> (continuous) processing. See https://beam.apache.org
>>
>> You can download the release here:
>>
>> https://beam.apache.org/get-started/downloads/
>>
>> This release includes bugfixes, features, and improvements detailed on
>> the Beam blog: https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html
>>
>> Thanks to everyone who contributed to this release, and we hope you enjoy
>> using Beam 2.10.0.
>>
>> -- Kenneth Knowles, on behalf of The Apache Beam team
>>
>
Re: [ANNOUNCE] Apache Beam 2.10.0 released!
Posted by Kenneth Knowles <kl...@google.com>.
Hi Matt,
I'm interested in the trouble you've encountered. I've filed
https://issues.apache.org/jira/browse/BEAM-6706 to follow up.
Kenn
On Mon, Feb 18, 2019 at 3:23 AM Matt Casters <ma...@gmail.com> wrote:
> Hi Beam-ers,
>
> Congratulations on the release!
> I did a quick upgrade and tried to run my test ETL jobs on Direct & Flink
> runners with great success.
> DataFlow however is throwing all sorts of errors. For example:
>
> * Handler for GET /v1.27/images/
> gcr.io/cloud-dataflow/v1beta3/beam-java-batch:beam-2.10.0/json returned
> error: No such image:
> gcr.io/cloud-dataflow/v1beta3/beam-java-batch:beam-2.10.0"
> * while reading 'google-dockercfg' metadata: http status code: 404 while
> fetching url
> http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
> "
> * Error syncing pod..."
>
> The job gets stuck after starting a worker and after an hour or so it
> gives up with a failure. 2.9.0 runs fine.
>
> I realize I've been quick to update this weekend but it would be great if
> I could get a feel of whether I'm messing up something in my Maven pom.xml
> or if integration with Beam 2.10.0 isn't done yet on DataFlow.
>
> Thanks in advance!
>
> Matt
>
>
>
> Op vr 15 feb. 2019 om 20:29 schreef Kenneth Knowles <ke...@apache.org>:
>
>> The Apache Beam team is pleased to announce the release of version 2.10.0!
>>
>> Apache Beam is an open source unified programming model to define and
>> execute data processing pipelines, including ETL, batch and stream
>> (continuous) processing. See https://beam.apache.org
>>
>> You can download the release here:
>>
>> https://beam.apache.org/get-started/downloads/
>>
>> This release includes bugfixes, features, and improvements detailed on
>> the Beam blog: https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html
>>
>> Thanks to everyone who contributed to this release, and we hope you enjoy
>> using Beam 2.10.0.
>>
>> -- Kenneth Knowles, on behalf of The Apache Beam team
>>
>
Re: [ANNOUNCE] Apache Beam 2.10.0 released!
Posted by Matt Casters <ma...@gmail.com>.
Hi Beam-ers,
Congratulations on the release!
I did a quick upgrade and tried to run my test ETL jobs on Direct & Flink
runners with great success.
DataFlow however is throwing all sorts of errors. For example:
* Handler for GET /v1.27/images/
gcr.io/cloud-dataflow/v1beta3/beam-java-batch:beam-2.10.0/json returned
error: No such image:
gcr.io/cloud-dataflow/v1beta3/beam-java-batch:beam-2.10.0"
* while reading 'google-dockercfg' metadata: http status code: 404 while
fetching url
http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
"
* Error syncing pod..."
The job gets stuck after starting a worker and after an hour or so it gives
up with a failure. 2.9.0 runs fine.
I realize I've been quick to update this weekend but it would be great if I
could get a feel of whether I'm messing up something in my Maven pom.xml or
if integration with Beam 2.10.0 isn't done yet on DataFlow.
Thanks in advance!
Matt
Op vr 15 feb. 2019 om 20:29 schreef Kenneth Knowles <ke...@apache.org>:
> The Apache Beam team is pleased to announce the release of version 2.10.0!
>
> Apache Beam is an open source unified programming model to define and
> execute data processing pipelines, including ETL, batch and stream
> (continuous) processing. See https://beam.apache.org
>
> You can download the release here:
>
> https://beam.apache.org/get-started/downloads/
>
> This release includes bugfixes, features, and improvements detailed on the
> Beam blog: https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html
>
> Thanks to everyone who contributed to this release, and we hope you enjoy
> using Beam 2.10.0.
>
> -- Kenneth Knowles, on behalf of The Apache Beam team
>