You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Neville Li <ne...@gmail.com> on 2020/09/10 20:10:08 UTC
Scio 0.9.4 released
Hi all,
We just released Scio 0.9.4.
Cheers,
Neville
https://github.com/spotify/scio/releases/tag/v0.9.4
*"Deletrius"*
There are no breaking changes in this release, but some were introduced
with v0.9.0:
See v0.9.0 Migration Guide
<https://spotify.github.io/scio/migrations/v0.9.0-Migration-Guide.html> for
detailed instructions.
Improvements
- Add SCollection#filterNot (#3291
<https://github.com/spotify/scio/pull/3291>)
- Improve filterValues doc (#3290
<https://github.com/spotify/scio/pull/3290>)
- Add support for JDBC sharding by UUID encoded as string (#3307
<https://github.com/spotify/scio/pull/3307>)
- Add optimised coder derivation for AnyVal (#3296
<https://github.com/spotify/scio/pull/3296>)
- Support BigQuery Avro Format (#3221
<https://github.com/spotify/scio/pull/3221>)
- Support sparkey compression, fix #3210
<https://github.com/spotify/scio/issues/3210> (#3295
<https://github.com/spotify/scio/pull/3295>)
- Warn if sparkey is bigger than memory, #3280
<https://github.com/spotify/scio/issues/3280>
- (fix #3278 <https://github.com/spotify/scio/issues/3278>) Warn on
chained .groupByKey.join (#3297
<https://github.com/spotify/scio/pull/3297>)
- [SMB] delete file early in NativeFileSorter (#3274
<https://github.com/spotify/scio/pull/3274>)
- Change default SMB codec to Deflate to match Scio (#3247
<https://github.com/spotify/scio/pull/3247>)
- Add java.time LocalDate, LocalDateTime, LocalTime, Period, Duration
coders(#3238 <https://github.com/spotify/scio/pull/3238>)
Bug Fixes
- Remove duplicate ShardedSparkeyReader
- Use andThen for future side effect ops (#3275
<https://github.com/spotify/scio/pull/3275>)
- Fix Bigquery IT test
- Add state to exception message when pipeline is cancelled (#3270
<https://github.com/spotify/scio/pull/3270>)
- Avoid scala.jdk.CollectionConverters implicit import in Avro macro (
#3250 <https://github.com/spotify/scio/pull/3250>)
- Avoid scala.jdk.CollectionConverters implicit import in Bigquery macro
(#3240 <https://github.com/spotify/scio/pull/3240>)
- fix(avro-bq): added EnumSymbol case for matching avro types to BQ
TableRow (#3226 <https://github.com/spotify/scio/issues/3226>) (#3232
<https://github.com/spotify/scio/pull/3232>)
- Remove uneeded LowPriority implicit (#3239
<https://github.com/spotify/scio/pull/3239>)
- Remove coders deprecation warns (#3242
<https://github.com/spotify/scio/pull/3242>)
- Use ## and support consistent Array hashCode (#3246
<https://github.com/spotify/scio/pull/3246>)
- Improve SMB error handling (#3253
<https://github.com/spotify/scio/pull/3253>)
- Workaround for sorter memory limit #3260
<https://github.com/spotify/scio/issues/3260> (#3269
<https://github.com/spotify/scio/pull/3269>)
Dependency Updates
- Bump sparkey to 3.2.0
- Remove unused imports (#3243
<https://github.com/spotify/scio/pull/3243>)
- Update case-app, case-app-annotations, ... to 2.0.4 (#3256
<https://github.com/spotify/scio/pull/3256>)
- Update cassandra-all to 3.11.8 (#3281
<https://github.com/spotify/scio/pull/3281>)
- Update cassandra-driver-core to 3.10.2 (#3276
<https://github.com/spotify/scio/pull/3276>)
- Update commons-io to 2.8.0 (#3310
<https://github.com/spotify/scio/pull/3310>)
- Update elasticsearch, ... to 7.9.1 (#3301
<https://github.com/spotify/scio/pull/3301>)
- Update elasticsearch, ... to 6.8.12 (#3264
<https://github.com/spotify/scio/pull/3264>)
- Update flink runner to 1.10.1 (#3249
<https://github.com/spotify/scio/pull/3249>)
- Update magnolia to 0.17.0 (#3262
<https://github.com/spotify/scio/pull/3262>)
- Update magnolify-avro, magnolify-bigtable, ... to 0.2.3 (#3263
<https://github.com/spotify/scio/pull/3263>)
- Update parquet-avro, parquet-column, ... to 1.11.1 (#3251
<https://github.com/spotify/scio/pull/3251>)
- Update protobuf-generic to 0.2.9 (#3227
<https://github.com/spotify/scio/pull/3227>)
- Update protobuf-java to 3.13.0 (#3257
<https://github.com/spotify/scio/pull/3257>)
- Update sbt-avro to 3.0.0 (#3252
<https://github.com/spotify/scio/pull/3252>)
- Update sbt-bloop to 1.4.4 (#3287
<https://github.com/spotify/scio/pull/3287>)
- Update sbt-buildinfo to 0.10.0 (#3245
<https://github.com/spotify/scio/pull/3245>)
- Update sbt-java-formatter to 0.6.0 (#3259
<https://github.com/spotify/scio/pull/3259>)
- Update sbt-jmh to 0.4.0 (#3288
<https://github.com/spotify/scio/pull/3288>)
- Update sbt-mdoc to 2.2.7 (#3311
<https://github.com/spotify/scio/pull/3311>)
- Update sbt-mima-plugin to 0.8.0 (#3305
<https://github.com/spotify/scio/pull/3305>)
- Update sbt-scalafix to 0.9.20 (#3298
<https://github.com/spotify/scio/pull/3298>)
- Update scalactic to 3.2.2 (#3271
<https://github.com/spotify/scio/pull/3271>)
- Update scalafmt-core to 2.7.0
- Update scalatest to 3.2.2 (#3272
<https://github.com/spotify/scio/pull/3272>)
- Update transport to 6.8.12 (#3265
<https://github.com/spotify/scio/pull/3265>)
- Use beam-runners-flink-1.10 when using BEAM_RUNNERS (#3303
<https://github.com/spotify/scio/pull/3303>)