You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Neville Li <ne...@gmail.com> on 2018/01/04 20:41:34 UTC

Scio 0.4.7 released

Hi all,

We just release Scio 0.4.7. This release fixed a join performance
regression and introduced several improvements.

https://github.com/spotify/scio/releases/tag/v0.4.7

*"Hydrochoerus hydrochaeris"*
Features

   - Add support for TFRecordSpec #990
   <https://github.com/spotify/scio/pull/990>
   - Add convenience methods to randomSpit #987
   <https://github.com/spotify/scio/pull/987>
   - Add BigtableDoFn #922 <https://github.com/spotify/scio/issues/922> #931
   <https://github.com/spotify/scio/pull/931>
   - Add optional arguments validation #979
   <https://github.com/spotify/scio/pull/979>
   - Performance improvement in Avro and BigQuery macro converters #989
   <https://github.com/spotify/scio/issues/989>
   - Update new bigquery datetime and timestamp specification #982
   <https://github.com/spotify/scio/pull/982>

Bug fixes

   - Fix join performance regression introduced in 0.4.4 #976
   <https://github.com/spotify/scio/issues/976> #983
   <https://github.com/spotify/scio/pull/983>
   - Fine tune dynamic sink API and and integration tests #993
   <https://github.com/spotify/scio/issues/993> #995
   <https://github.com/spotify/scio/pull/995>

Re: Scio 0.4.7 released

Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.

Hi

I remember when working with Scio, there was some specific IOs in Scio not using 
the ones provided by Beam.

I didn't check recently, maybe it has changed.

Regards
JB

On 01/04/2018 09:51 PM, Neville Li wrote:
> Not sure what you mean. We use Beam file IOs whenever possible which should 
> already use the FileSystems API.
> 
> On Thu, Jan 4, 2018 at 3:48 PM Jean-Baptiste Onofré <jb@nanthrax.net 
> <ma...@nanthrax.net>> wrote:
> 
>     Hi Neville,
> 
>     Great !!!! Thanks for the update.
> 
>     By the way, any plan to leverage the Beam filesystems in Scio now ?
> 
>     Regards
>     JB
> 
>     On 01/04/2018 09:41 PM, Neville Li wrote:
>      > Hi all,
>      >
>      > We just release Scio 0.4.7. This release fixed a join performance
>     regression and
>      > introduced several improvements.
>      >
>      > https://github.com/spotify/scio/releases/tag/v0.4.7
>      >
>      > /"Hydrochoerus hydrochaeris"/
>      >
>      >
>      >       Features
>      >
>      >   * Add support for TFRecordSpec #990
>     <https://github.com/spotify/scio/pull/990>
>      >   * Add convenience methods to randomSpit #987
>      >     <https://github.com/spotify/scio/pull/987>
>      >   * Add BigtableDoFn #922 <https://github.com/spotify/scio/issues/922> #931
>      >     <https://github.com/spotify/scio/pull/931>
>      >   * Add optional arguments validation #979
>      >     <https://github.com/spotify/scio/pull/979>
>      >   * Performance improvement in Avro and BigQuery macro converters #989
>      >     <https://github.com/spotify/scio/issues/989>
>      >   * Update new bigquery datetime and timestamp specification #982
>      >     <https://github.com/spotify/scio/pull/982>
>      >
>      >
>      >       Bug fixes
>      >
>      >   * Fix join performance regression introduced in 0.4.4 #976
>      >     <https://github.com/spotify/scio/issues/976> #983
>      >     <https://github.com/spotify/scio/pull/983>
>      >   * Fine tune dynamic sink API and and integration tests #993
>      >     <https://github.com/spotify/scio/issues/993> #995
>      >     <https://github.com/spotify/scio/pull/995>
>      >
> 
>     --
>     Jean-Baptiste Onofré
>     jbonofre@apache.org <ma...@apache.org>
>     http://blog.nanthrax.net
>     Talend - http://www.talend.com
> 

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: Scio 0.4.7 released

Posted by Neville Li <ne...@gmail.com>.

Not sure what you mean. We use Beam file IOs whenever possible which should
already use the FileSystems API.

On Thu, Jan 4, 2018 at 3:48 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:

> Hi Neville,
>
> Great !!!! Thanks for the update.
>
> By the way, any plan to leverage the Beam filesystems in Scio now ?
>
> Regards
> JB
>
> On 01/04/2018 09:41 PM, Neville Li wrote:
> > Hi all,
> >
> > We just release Scio 0.4.7. This release fixed a join performance
> regression and
> > introduced several improvements.
> >
> > https://github.com/spotify/scio/releases/tag/v0.4.7
> >
> > /"Hydrochoerus hydrochaeris"/
> >
> >
> >       Features
> >
> >   * Add support for TFRecordSpec #990 <
> https://github.com/spotify/scio/pull/990>
> >   * Add convenience methods to randomSpit #987
> >     <https://github.com/spotify/scio/pull/987>
> >   * Add BigtableDoFn #922 <https://github.com/spotify/scio/issues/922>
> #931
> >     <https://github.com/spotify/scio/pull/931>
> >   * Add optional arguments validation #979
> >     <https://github.com/spotify/scio/pull/979>
> >   * Performance improvement in Avro and BigQuery macro converters #989
> >     <https://github.com/spotify/scio/issues/989>
> >   * Update new bigquery datetime and timestamp specification #982
> >     <https://github.com/spotify/scio/pull/982>
> >
> >
> >       Bug fixes
> >
> >   * Fix join performance regression introduced in 0.4.4 #976
> >     <https://github.com/spotify/scio/issues/976> #983
> >     <https://github.com/spotify/scio/pull/983>
> >   * Fine tune dynamic sink API and and integration tests #993
> >     <https://github.com/spotify/scio/issues/993> #995
> >     <https://github.com/spotify/scio/pull/995>
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: Scio 0.4.7 released

Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.

Hi Neville,

Great !!!! Thanks for the update.

By the way, any plan to leverage the Beam filesystems in Scio now ?

Regards
JB

On 01/04/2018 09:41 PM, Neville Li wrote:
> Hi all,
> 
> We just release Scio 0.4.7. This release fixed a join performance regression and 
> introduced several improvements.
> 
> https://github.com/spotify/scio/releases/tag/v0.4.7
> 
> /"Hydrochoerus hydrochaeris"/
> 
> 
>       Features
> 
>   * Add support for TFRecordSpec #990 <https://github.com/spotify/scio/pull/990>
>   * Add convenience methods to randomSpit #987
>     <https://github.com/spotify/scio/pull/987>
>   * Add BigtableDoFn #922 <https://github.com/spotify/scio/issues/922> #931
>     <https://github.com/spotify/scio/pull/931>
>   * Add optional arguments validation #979
>     <https://github.com/spotify/scio/pull/979>
>   * Performance improvement in Avro and BigQuery macro converters #989
>     <https://github.com/spotify/scio/issues/989>
>   * Update new bigquery datetime and timestamp specification #982
>     <https://github.com/spotify/scio/pull/982>
> 
> 
>       Bug fixes
> 
>   * Fix join performance regression introduced in 0.4.4 #976
>     <https://github.com/spotify/scio/issues/976> #983
>     <https://github.com/spotify/scio/pull/983>
>   * Fine tune dynamic sink API and and integration tests #993
>     <https://github.com/spotify/scio/issues/993> #995
>     <https://github.com/spotify/scio/pull/995>
> 

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com