You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Michel Davit <mi...@spotify.com> on 2022/05/02 15:50:53 UTC
Scio 0.11.6 released
Hi all,
We just released Scio 0.11.6
<https://github.com/spotify/scio/releases/tag/v0.11.6>. This bumps Beam SDK
to 2.38.0 <https://beam.apache.org/blog/beam-2.38.0/> and includes some
improvements & bug fixes.
Cheers,
*v0.11.6 "Colchis"*
[image: rocket] Enhancements
- Add scio-elasticsearch8 module (#4311
<https://github.com/spotify/scio/pull/4311>) @RustedBones
<https://github.com/RustedBones>
- Mock BQ wildcard table (#4314
<https://github.com/spotify/scio/pull/4314>) @RustedBones
<https://github.com/RustedBones>
- Include which user is authenticated in BQ 403 exceptions (#4191
<https://github.com/spotify/scio/pull/4191>) @clairemcginty
<https://github.com/clairemcginty>
- Move PopulateTestData script to GHA (#4294
<https://github.com/spotify/scio/pull/4294>) @clairemcginty
<https://github.com/clairemcginty>
[image: bug] Bug Fixes
- (fix #4216 <https://github.com/spotify/scio/issues/4216>) extend BQ
CoderInstances in main package (#4328
<https://github.com/spotify/scio/pull/4328>) @clairemcginty
<https://github.com/clairemcginty>
- Support build metadata to identify proper previous version (#4332
<https://github.com/spotify/scio/pull/4332>) @RustedBones
<https://github.com/RustedBones>
- Fix GHA publish due to non-deterministic implicit (#4323
<https://github.com/spotify/scio/pull/4323>) @RustedBones
<https://github.com/RustedBones>
- Fix projection for parquet.tensorflow JobTest + add example job (#4305
<https://github.com/spotify/scio/pull/4305>) @clairemcginty
<https://github.com/clairemcginty>
- (fix #4285 <https://github.com/spotify/scio/issues/4285>) Set
expiration time explicitly for temp BQ view tables (#4292
<https://github.com/spotify/scio/pull/4292>) @clairemcginty
<https://github.com/clairemcginty>
[image: green_book] Documentation
- Prepare documentation for release v0.11.6 (#4346
<https://github.com/spotify/scio/pull/4346>) @RustedBones
<https://github.com/RustedBones>
- Add spotify OSS maintainer metadata (#4326
<https://github.com/spotify/scio/pull/4326>) @kellen
<https://github.com/kellen>
[image: seedling] Dependency Updates
- Update beam to 2.38.0 (#4318
<https://github.com/spotify/scio/pull/4318>) @scala-steward
<https://github.com/scala-steward>
- Pin caffeine dependency (#4344
<https://github.com/spotify/scio/pull/4344>) @regadas
<https://github.com/regadas>
- Ignore beam dependent version updates (#4345
<https://github.com/spotify/scio/pull/4345>) @RustedBones
<https://github.com/RustedBones>
- Update pprint to 0.7.3 (#4298
<https://github.com/spotify/scio/pull/4298>) @scala-steward
<https://github.com/scala-steward>
- Update cassandra-driver-core to 3.11.2 (#4342
<https://github.com/spotify/scio/pull/4342>) @scala-steward
<https://github.com/scala-steward>
- Update scalatestplus scalacheck artifact (#4343
<https://github.com/spotify/scio/pull/4343>) @RustedBones
<https://github.com/RustedBones>
- Update elasticsearch, ... to 7.17.3 (#4337
<https://github.com/spotify/scio/pull/4337>) @scala-steward
<https://github.com/scala-steward>
- Update mysql-connector-java to 8.0.29 (#4329
<https://github.com/spotify/scio/pull/4329>) @scala-steward
<https://github.com/scala-steward>
- Bump JamesIves/github-pages-deploy-action from 4.3.2 to 4.3.3 (#4331
<https://github.com/spotify/scio/pull/4331>) @dependabot
<https://github.com/dependabot>
- Update scalatest to 3.2.12 (#4325
<https://github.com/spotify/scio/pull/4325>) @scala-steward
<https://github.com/scala-steward>
- Update tensorflow-core-api, ... to 0.4.1 (#4301
<https://github.com/spotify/scio/pull/4301>) @scala-steward
<https://github.com/scala-steward>
- Update scalacheck to 1.16.0 (#4309
<https://github.com/spotify/scio/pull/4309>) @scala-steward
<https://github.com/scala-steward>
- Update scalafmt-core to 3.5.2 (#4317
<https://github.com/spotify/scio/pull/4317>) @scala-steward
<https://github.com/scala-steward>
- Update sbt-header to 5.7.0 (#4319
<https://github.com/spotify/scio/pull/4319>) @scala-steward
<https://github.com/scala-steward>
- Update sbt-bloop to 1.5.0 (#4321
<https://github.com/spotify/scio/pull/4321>) @scala-steward
<https://github.com/scala-steward>
- Update zoltar-api, zoltar-core, ... to 0.6.0 (#4320
<https://github.com/spotify/scio/pull/4320>) @scala-steward
<https://github.com/scala-steward>
- Update magnolify-avro, magnolify-bigtable, ... to 0.4.8 (#4316
<https://github.com/spotify/scio/pull/4316>) @scala-steward
<https://github.com/scala-steward>
- Update sbt-scalafix to 0.10.0 (#4308
<https://github.com/spotify/scio/pull/4308>) @scala-steward
<https://github.com/scala-steward>
- Update sparkey to 3.2.2 (#4312
<https://github.com/spotify/scio/pull/4312>) @scala-steward
<https://github.com/scala-steward>
- Bump actions/setup-java from 2 to 3 (#4310
<https://github.com/spotify/scio/pull/4310>) @dependabot
<https://github.com/dependabot>
- Update sbt-mima-plugin to 1.1.0 (#4307
<https://github.com/spotify/scio/pull/4307>) @scala-steward
<https://github.com/scala-steward>
- Update jackson-module-scala to 2.13.2 (#4283
<https://github.com/spotify/scio/pull/4283>) @scala-steward
<https://github.com/scala-steward>
- Bump actions/checkout from 2.4.0 to 3 (#4275
<https://github.com/spotify/scio/pull/4275>) @dependabot
<https://github.com/dependabot>
- Update jna to 5.11.0 (#4296 <https://github.com/spotify/scio/pull/4296>
) @scala-steward <https://github.com/scala-steward>
- Update mysql-socket-factory to 1.5.0 (#4304
<https://github.com/spotify/scio/pull/4304>) @scala-steward
<https://github.com/scala-steward>
- Update shapeless to 2.3.9 (#4297
<https://github.com/spotify/scio/pull/4297>) @scala-steward
<https://github.com/scala-steward>
- Update sbt-api-mappings to 3.0.2 (#4290
<https://github.com/spotify/scio/pull/4290>) @scala-steward
<https://github.com/scala-steward>
- Update sbt-mdoc to 2.3.2 (#4299
<https://github.com/spotify/scio/pull/4299>) @scala-steward
<https://github.com/scala-steward>
- Update joda-time to 2.10.14 (#4291
<https://github.com/spotify/scio/pull/4291>) @scala-steward
<https://github.com/scala-steward>
- Update scala-collection-compat to 2.7.0 (#4293
<https://github.com/spotify/scio/pull/4293>) @scala-steward
<https://github.com/scala-steward>
--
Michel Davit
Data Engineer
Spotify France | 54 Rue de Londres | 75008 Paris, France
Re: Scio 0.11.6 released
Posted by Michel Davit <mi...@spotify.com>.
Hi again,
We've identified a regression on the BigQuery macros when annotations are
referencing a table with $LATEST
Release v0.11.7 <https://github.com/spotify/scio/releases/tag/v0.11.7>
addresses the issue
Sorry for the inconvenience
*v0.11.7 "Creta"*
[image: bug] Bug Fixes
- Fix BQ table pagination in tableReferences (#4372
<https://github.com/spotify/scio/pull/4372>) @RustedBones
<https://github.com/RustedBones>
- Fix BQ IT test (#4375 <https://github.com/spotify/scio/pull/4375>)
@RustedBones <https://github.com/RustedBones>
[image: green_book] Documentation
- Prepare documentation for release v0.11.7 (#4374
<https://github.com/spotify/scio/pull/4374>) @RustedBones
<https://github.com/RustedBones>
- Fix bullet points (#4373 <https://github.com/spotify/scio/pull/4373>)
@wscheep <https://github.com/wscheep>
[image: seedling] Dependency Updates
- Enable MiMa for scio-parquet and scio-elasticsearch8 (#4370
<https://github.com/spotify/scio/pull/4370>) @RustedBones
<https://github.com/RustedBones>
- Update scalactic to 3.2.12 (#4324
<https://github.com/spotify/scio/pull/4324>) @scala-steward
<https://github.com/scala-steward>
- Update elasticsearch-java to 8.1.3 (#4347
<https://github.com/spotify/scio/pull/4347>) @scala-steward
<https://github.com/scala-steward>
- Fix malformed scala-steward conf file (#4369
<https://github.com/spotify/scio/pull/4369>) @RustedBones
<https://github.com/RustedBones>
On Mon, May 2, 2022 at 5:50 PM Michel Davit <mi...@spotify.com> wrote:
> Hi all,
>
> We just released Scio 0.11.6
> <https://github.com/spotify/scio/releases/tag/v0.11.6>. This bumps Beam
> SDK to 2.38.0 <https://beam.apache.org/blog/beam-2.38.0/> and includes
> some improvements & bug fixes.
>
> Cheers,
>
> *v0.11.6 "Colchis"*
>
> [image: rocket] Enhancements
>
> - Add scio-elasticsearch8 module (#4311
> <https://github.com/spotify/scio/pull/4311>) @RustedBones
> <https://github.com/RustedBones>
> - Mock BQ wildcard table (#4314
> <https://github.com/spotify/scio/pull/4314>) @RustedBones
> <https://github.com/RustedBones>
> - Include which user is authenticated in BQ 403 exceptions (#4191
> <https://github.com/spotify/scio/pull/4191>) @clairemcginty
> <https://github.com/clairemcginty>
> - Move PopulateTestData script to GHA (#4294
> <https://github.com/spotify/scio/pull/4294>) @clairemcginty
> <https://github.com/clairemcginty>
>
> [image: bug] Bug Fixes
>
> - (fix #4216 <https://github.com/spotify/scio/issues/4216>) extend BQ
> CoderInstances in main package (#4328
> <https://github.com/spotify/scio/pull/4328>) @clairemcginty
> <https://github.com/clairemcginty>
> - Support build metadata to identify proper previous version (#4332
> <https://github.com/spotify/scio/pull/4332>) @RustedBones
> <https://github.com/RustedBones>
> - Fix GHA publish due to non-deterministic implicit (#4323
> <https://github.com/spotify/scio/pull/4323>) @RustedBones
> <https://github.com/RustedBones>
> - Fix projection for parquet.tensorflow JobTest + add example job (
> #4305 <https://github.com/spotify/scio/pull/4305>) @clairemcginty
> <https://github.com/clairemcginty>
> - (fix #4285 <https://github.com/spotify/scio/issues/4285>) Set
> expiration time explicitly for temp BQ view tables (#4292
> <https://github.com/spotify/scio/pull/4292>) @clairemcginty
> <https://github.com/clairemcginty>
>
> [image: green_book] Documentation
>
> - Prepare documentation for release v0.11.6 (#4346
> <https://github.com/spotify/scio/pull/4346>) @RustedBones
> <https://github.com/RustedBones>
> - Add spotify OSS maintainer metadata (#4326
> <https://github.com/spotify/scio/pull/4326>) @kellen
> <https://github.com/kellen>
>
> [image: seedling] Dependency Updates
>
> - Update beam to 2.38.0 (#4318
> <https://github.com/spotify/scio/pull/4318>) @scala-steward
> <https://github.com/scala-steward>
> - Pin caffeine dependency (#4344
> <https://github.com/spotify/scio/pull/4344>) @regadas
> <https://github.com/regadas>
> - Ignore beam dependent version updates (#4345
> <https://github.com/spotify/scio/pull/4345>) @RustedBones
> <https://github.com/RustedBones>
> - Update pprint to 0.7.3 (#4298
> <https://github.com/spotify/scio/pull/4298>) @scala-steward
> <https://github.com/scala-steward>
> - Update cassandra-driver-core to 3.11.2 (#4342
> <https://github.com/spotify/scio/pull/4342>) @scala-steward
> <https://github.com/scala-steward>
> - Update scalatestplus scalacheck artifact (#4343
> <https://github.com/spotify/scio/pull/4343>) @RustedBones
> <https://github.com/RustedBones>
> - Update elasticsearch, ... to 7.17.3 (#4337
> <https://github.com/spotify/scio/pull/4337>) @scala-steward
> <https://github.com/scala-steward>
> - Update mysql-connector-java to 8.0.29 (#4329
> <https://github.com/spotify/scio/pull/4329>) @scala-steward
> <https://github.com/scala-steward>
> - Bump JamesIves/github-pages-deploy-action from 4.3.2 to 4.3.3 (#4331
> <https://github.com/spotify/scio/pull/4331>) @dependabot
> <https://github.com/dependabot>
> - Update scalatest to 3.2.12 (#4325
> <https://github.com/spotify/scio/pull/4325>) @scala-steward
> <https://github.com/scala-steward>
> - Update tensorflow-core-api, ... to 0.4.1 (#4301
> <https://github.com/spotify/scio/pull/4301>) @scala-steward
> <https://github.com/scala-steward>
> - Update scalacheck to 1.16.0 (#4309
> <https://github.com/spotify/scio/pull/4309>) @scala-steward
> <https://github.com/scala-steward>
> - Update scalafmt-core to 3.5.2 (#4317
> <https://github.com/spotify/scio/pull/4317>) @scala-steward
> <https://github.com/scala-steward>
> - Update sbt-header to 5.7.0 (#4319
> <https://github.com/spotify/scio/pull/4319>) @scala-steward
> <https://github.com/scala-steward>
> - Update sbt-bloop to 1.5.0 (#4321
> <https://github.com/spotify/scio/pull/4321>) @scala-steward
> <https://github.com/scala-steward>
> - Update zoltar-api, zoltar-core, ... to 0.6.0 (#4320
> <https://github.com/spotify/scio/pull/4320>) @scala-steward
> <https://github.com/scala-steward>
> - Update magnolify-avro, magnolify-bigtable, ... to 0.4.8 (#4316
> <https://github.com/spotify/scio/pull/4316>) @scala-steward
> <https://github.com/scala-steward>
> - Update sbt-scalafix to 0.10.0 (#4308
> <https://github.com/spotify/scio/pull/4308>) @scala-steward
> <https://github.com/scala-steward>
> - Update sparkey to 3.2.2 (#4312
> <https://github.com/spotify/scio/pull/4312>) @scala-steward
> <https://github.com/scala-steward>
> - Bump actions/setup-java from 2 to 3 (#4310
> <https://github.com/spotify/scio/pull/4310>) @dependabot
> <https://github.com/dependabot>
> - Update sbt-mima-plugin to 1.1.0 (#4307
> <https://github.com/spotify/scio/pull/4307>) @scala-steward
> <https://github.com/scala-steward>
> - Update jackson-module-scala to 2.13.2 (#4283
> <https://github.com/spotify/scio/pull/4283>) @scala-steward
> <https://github.com/scala-steward>
> - Bump actions/checkout from 2.4.0 to 3 (#4275
> <https://github.com/spotify/scio/pull/4275>) @dependabot
> <https://github.com/dependabot>
> - Update jna to 5.11.0 (#4296
> <https://github.com/spotify/scio/pull/4296>) @scala-steward
> <https://github.com/scala-steward>
> - Update mysql-socket-factory to 1.5.0 (#4304
> <https://github.com/spotify/scio/pull/4304>) @scala-steward
> <https://github.com/scala-steward>
> - Update shapeless to 2.3.9 (#4297
> <https://github.com/spotify/scio/pull/4297>) @scala-steward
> <https://github.com/scala-steward>
> - Update sbt-api-mappings to 3.0.2 (#4290
> <https://github.com/spotify/scio/pull/4290>) @scala-steward
> <https://github.com/scala-steward>
> - Update sbt-mdoc to 2.3.2 (#4299
> <https://github.com/spotify/scio/pull/4299>) @scala-steward
> <https://github.com/scala-steward>
> - Update joda-time to 2.10.14 (#4291
> <https://github.com/spotify/scio/pull/4291>) @scala-steward
> <https://github.com/scala-steward>
> - Update scala-collection-compat to 2.7.0 (#4293
> <https://github.com/spotify/scio/pull/4293>) @scala-steward
> <https://github.com/scala-steward>
>
>
> --
>
>
>
> Michel Davit
> Data Engineer
> Spotify France | 54 Rue de Londres | 75008 Paris, France
>
>
--
Michel Davit
Data Engineer
Spotify France | 54 Rue de Londres | 75008 Paris, France