You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by lo...@apache.org on 2022/12/27 18:40:45 UTC
[beam] branch beam24789convertifneeded created (now 93dbd7345a8)
This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a change to branch beam24789convertifneeded
in repository https://gitbox.apache.org/repos/asf/beam.git
at 93dbd7345a8 [#24789] Handle missing FullValue wrapper.
This branch includes the following new commits:
new 2184ee7293f Revert "Support SqlTypes Date and Timestamp (MicrosInstant) in AvroUtils (#23969)"
new 507ca4e4804 Add back javadoc
new 9ffa48ec5d1 Reduce version matrix of build wheel on pull request event (#24448)
new 4f152384bc2 [Spark dataset runner] Make sure PCollection views get only broadcasted once if reused (#24288)
new 486d4ec7710 Disable flaky async tests in samza runner
new affebb86db1 Fix for #22951 w/ GroupIntoBatchesOverride fix (#24463)
new ca574a99be4 Never mark issues as stale (#24494)
new 298cb59706d [BigQueryIO] Use final destination table schema and metadata when creating temp tables (#24471)
new e82672ab13c Minor visibility improvements to render runner. (#24480)
new cffcd14d546 [Go SDK]: Infer field names from struct tags (#24473)
new 715fa40c2ec Bump github.com/go-sql-driver/mysql from 1.6.0 to 1.7.0 in /sdks (#24487)
new dc2c0d58b4f [Website] add coauthor option to case-studies #24486 (#24475)
new 5765eac8af8 Add panel pointing to docs for ml benchmarks (#24503)
new a7759f424d3 Adding Beam Schemas capability to parse json-schemas. This is the de-… (#24271)
new f3785cc17af Bump tensorflow from 2.9.1 to 2.9.3 in /sdks/python/apache_beam/examples/ml-orchestration/tfx (#24300)
new cd52f027715 Fix broken typescript tests
new a9327ae42aa [SQL extension] Minor fixes to logging applying best practices (#24328)
new 06489d7b42f Apply task configuration avoidance (#24509)
new 5217d0202a7 Bump cloud.google.com/go/profiler from 0.3.0 to 0.3.1 in /sdks (#24498)
new 3d9b2dc6ce6 fix exception
new f0e49feaca9 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24502)
new 63b2ebad6f8 Adding support for PubSub Lite in Schema Transforms (#24275)
new 77b88305757 Fix mispositioned producer_type_hints and producer_batch_converter params. (#24511)
new e35e72d64a8 Update REVIEWERS.yml (#24507)
new f26e7f74c27 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#24501)
new 15da2bf3a00 [CdapIO] Add readme for CdapIO. Update readme for SparkReceiverIO. (#23959)
new c674bd83335 [Spark Dataset runner] Add @Experimental and reduce visibility where possible.
new 5fc6d6787c7 Fix grafana dashboard id (#24524)
new 457cbe660d3 [Spark runner] Support running (VR) tests with Java 17 (closes #24400) (#24401)
new 10e51421427 Replaced deprecated finalize with DoFn Teardown (#24516)
new abe87679cb2 Bump cloud.google.com/go/storage from 1.28.0 to 1.28.1 in /sdks (#24517)
new add52bdfc72 add clarifier to error message (#24449)
new 70f9267692b Batch rename requests in fileio.WriteToFiles (#24341)
new 8f12a25b692 Bump golang.org/x/text from 0.4.0 to 0.5.0 in /sdks (#24520)
new 0ece3fcf4c9 Update nbconvert requirement in /sdks/python
new 07cb97cccaa Support for JsonSchema in Kafka Read Schema Transform (#24272)
new 7bd43b08fa9 Run go fmt over full go directory with go 1.19 (#24525)
new b887d241eea Cloudbuild+manualsetup+playground (#24144)
new 299d2257d61 Bump golang.org/x/sys from 0.2.0 to 0.3.0 in /sdks (#24519)
new 6055869e1f7 Bump cloud.google.com/go/bigtable from 1.18.0 to 1.18.1 in /sdks (#24500)
new fa3747bf14e Update from interface{} -> any for core packages (#24505)
new decc8982bcc Implement FileWriteSchemaTransformConfiguration (#24479)
new 466a4422225 Bump cloud.google.com/go/pubsub from 1.27.0 to 1.27.1 in /sdks (#24518)
new 9741da14f47 [Playground] Healthcheck was added (#24227)
new 6dd0b747327 Update dataflow container version for Pandas upgrade (#24532)
new 300ab334506 [Website] center the main content #24521
new e525ca67f07 [Website] update copy icon positioning #24426
new e4fc93b01b1 [Spark Dataset runner] Broadcast pipeline options
new 2380285fa4a Update google-cloud-bigquery-storage requirement from <2.14,>=2.6.3 to >=2.6.3,<2.17 in /sdks/python (#24539)
new 4b41e9f15f8 Bump cloud.google.com/go/pubsub from 1.27.1 to 1.28.0 in /sdks (#24534)
new b63093beae4 Bump golang.org/x/net from 0.2.0 to 0.3.0 in /sdks (#24544)
new d5a8088c2e0 Precommit python version update (#24526)
new 7d1dac7a805 [CdapIO] Add CdapIO and SparkReceiverIO documentation in website (#23338)
new fcf134e8ede Adding support for Pubsub Lite Writes in SchemaTransforms (#24359)
new ecdc35dfaaa Disallow using the JRH with Python streaming pipelines (#24513)
new 350cc0213cb fix null pointer exception caused by clearing member variable
new 8177a3b3253 Add RunInference example for TensorFlow Hub pre-trained model (#24529)
new 4426bb2332c update(PULL Request template) remove Choose reviewer (#24540)
new 953e21b3168 Revert "Bump actions/setup-java from 3.6.0 to 3.7.0 (#24484)" (#24551)
new 94bf7a3b8f3 Interface{}->any for more subfolders (#24553)
new e3b19b688a4 Support multiple gradle tasks in one precommit job
new 4f7d31a7959 Split up some IOs from Java PreCommit
new de2d711c97c Migrate testing subpackages from interface{} to any (#24570)
new 5cff47b5a5e fix go lints (#24566)
new 00221455ea2 Samza runner support for non unique stateId across multiple ParDos (#24276)
new 7203baddb25 Suppress keyfor warnings
new 7fc42a8d500 Suppress checker warnings that are confusing and difficult
new 62e5350603a Add @Pure annotations to MongoDbIO autovalue fields
new 5d3e9599f96 Suppress checker in FnApiDoFnRunner due to crash
new db029cfcaec Suppress checker framework in Dataflow FetchAndFilterStreamingSideInputOperation due to crash
new 58c3542c3fb Fix some nullness errors in Spark runner
new e03d797f8ef Upgrade checker framework to 3.27.0
new 06a6dee77fb Bump to Hadoop 3.3.4 for performance tests (#24550)
new bfbb487f7f9 pubsub: fix typo in grpc client factory
new f31535132be regenerate python dependencies (#24582)
new c6b2fd6636c Return empty splits if unable to split, not errors (#24508)
new e02558586ff [Tour Of Beam] Playground Router GRPC API host (#24542)
new d7e7ed332d9 Bump golang.org/x/net from 0.3.0 to 0.4.0 in /sdks (#24587)
new 896dc0766cf Replaced finalize with DoFn Teardown in Neo4jIO (#24571)
new 3eb5652bfc2 Simplify bug report templates
new b893d3ff11d Fix bugs in issue template yml
new 28b14021d54 Fix issue templates (#24597)
new 0b019a4111b [#24024] Stop wrapping light weight functions with Contextful as they add a lot of overhead for functions that are meant to do almost no work. (#24025)
new e53d8e564fe Sample window size as well (#24388)
new e58d743a746 Implement Kafka Write Schema Transform (#24495)
new 2121c2d45f2 Exclude IOs already split from Java Precommit job
new 5879e7b3e93 Move expansion services into appropriate precommits
new f203b38d26e Split more IOs out of Java precommit
new 95078cd59ad Fix trigger paths for separated IOs
new aee9765c733 Eliminate null errors from JdbcIO
new e9a3871e825 docs(fix): Filter.whereFieldName(s?) -> Filter.create().whereFieldName(s?) (#24576)
new 78ca4484d9f ElasticsearchIO: Lower log level in flushBatch to avoid noisy log
new 307ae231e42 Turn rawtype checking back on for core Java SDK
new 080a322fc65 interface{} -> any for registration (#24600)
new 088f2e9c8a9 Restrict tox to be in 3.x version (#24601)
new 19165f6f8a0 [Playground] support for Kafka-enabled examples (#24459)
new cbb0dacd8a5 Fix some small notebook typos (#24616)
new bec453e4030 Fixing a breakage of multi-lang auto Runner v2 enabling
new 81f6f14f334 initialize and increment metrics properly (#24592)
new e5389138282 Add schema conversion support from Kafka Connect Record schemas to Beam schemas (#24605)
new 8f8f6079566 interface{} -> any for starcgen (#24618)
new 4d626f850e1 interface{} -> any for remaining references (#24625)
new c7f0532565e Updating issue-tagger Workflow (#171) (#23143)
new 7f468212fb6 [GitHub Actions] - Updates in Build Playground Backend to runs-on Self-hosted runners (#23134)
new 1d6486b77db [GitHub Actions] - Updates in Build Playground Frontend to runs-on Self-hosted runners (#23135)
new 48db410a32a [GitHub Actions] - Updates in Go Tests to runs-on Self-hosted runners (#23140)
new c56c16975ec [GitHub Actions] - Updates in Java Tests to runs-on Self-hosted runners (#23160)
new d0fb3a74316 Updated label_prs workflow (#173) (#23145)
new 90cf0f120a1 [CdapIO] CdapIO and SparkReceiverIO updates (#24436)
new d7f6e7ebbf6 Revert "[GitHub Actions] - Updates in Java Tests to runs-on Self-hosted runners (#23160)" (#24628)
new a2e03e60fe8 Disallow sliding windows with combiner fanout to prevent data loss (#24606)
new 47eab9d96c6 Composite Action to activate SA in GitHub Actions Runners (#24626)
new 5cf66bc6ff0 Remove 'examples' dependency from HadoopFormatIO
new c18e0bcbe69 Bump google.golang.org/api from 0.103.0 to 0.104.0 in /sdks (#24586)
new 71d270c29ed [Website] add table-wrapper--equal-p table-wrapper--pr classes, use them in md files with tables #24193 #24194
new 972d443bdea [Spark RDD runner] Remove obsolete unusable AggregatorsAccumulator / NamedAggregators from the runner (closes #24379)
new 6625b62fff1 [Spark Dataset runner] Remove obsolete unusable AggregatorsAccumulator / NamedAggregators from the runner (closes #24379)
new 32be0949764 Release automation - Publish GitHub release notes (#24439)
new e5c505781a8 Release Automation - Update Python Dependencies (#24358)
new 534b5b5b4f8 [Github Actions] - Choose RC Commit Workflow (#24075)
new 4fd8d4fb7aa Updating connector page for BigtableIO
new 72d0b31b99e Suppress checker framework memory constraint warning
new 2fd3396cecc Eliminate rawtypes from KafkaIO
new 67639fc9e00 Improve Integer conversion in BigQueryIO's Storage Write API method. (#24366)
new baea57da08a Remove 'Hidden' attribute from serviceAccount PipelineOption (#24642)
new 374394ed2f7 disable daemon
new e6d131de0bc Allow avro block size to be configurable in AvroIO.write (#24454)
new 589bb17928d Eliminate rawtypes from GCP IO
new 5bdd59d1aea [Website] fix join beam page mobile layout #24611
new 9bfd939dc94 [Playground] parse url_notebook example tag (#24402)
new 3ced6928d26 [CdapIO] Complete examples for CDAP ServiceNow plugins (#24590)
new 95f48b21d28 [19734] Delete WatchTest.testMultiplePollsWithManyResults test (#24644)
new 57615900b80 [Go SDK] RunInference wrapper supporting Sklearn Model Handler (#24497)
new 76a18500c07 Show error messages when Playground requests fail (#24033) (#24338)
new f93c6276d14 Gradle tasks to start playground locally (#23939) (#23941)
new 5b57c57cc62 [Playground] notebook url backend API (#24425)
new d6ef8e5d52f Make base_sdk URN versioned on default docker container. (#24668)
new f217d221ea4 Add hypothesis strategy to generate complex schemas, use it to test `schemas` module (#23361)
new 37cb5592109 Add Debezium IO to doc (#24608)
new 1b241710d1a Bump google.golang.org/api from 0.104.0 to 0.105.0 in /sdks (#24672)
new 363dc429e6b resume running flaky test (#24648)
new afa64d1b92a Ensure that we handle formatting errors during logging in portable Python jobs. (#24652)
new b1c583977df [CdapIO] Complete examples for CDAP Zendesk plugins (#24589)
new 472dd1f52d5 Bump github.com/aws/aws-sdk-go-v2 from 1.17.2 to 1.17.3 in /sdks (#24686)
new dc6442f0062 [Tour Of Beam] delete user progress (#24657)
new 1ec81facd2b Bump github.com/aws/aws-sdk-go-v2/config from 1.18.4 to 1.18.5 in /sdks (#24688)
new 054f2e8ab5d Add MultimapState API (#23491)
new d7c4ab276ca sickbay test (#24681)
new 9ef8a081c48 Add additional logging for temp file deletion (#24653)
new c31cb7ed857 [Spark Dataset runner] Reduce binary size of Java serialized task related for ParDo translation (#24543)
new ef4f09efd34 Fix BigQueryIO.Write.Method.STORAGE_WRITE_API missing project id (#24634)
new a9f8f40a191 [Github Actions] - Start Snapshot Build (#24230)
new 54c99bc52a5 [Playground] infrastructure ci/cd unify (#24696)
new 00df4eeeb59 Updated pr-bot-pr-updates workflow (#180) (#23148)
new 2f96e5c87af Updated pr-bot-prs-needing-attention Workflow (#181) (#23150)
new 3bc3c126108 Updated reportGenerator Workflow (#184) (#23153)
new e7a7048e04b Updated self-assign Workflow (#185) (#23154)
new 6b13b4dd65d Updated triaged-on-assign workflow (#177) (#23156)
new 04913d2ce8e [GitHub Actions] - Updates in Typescript Tests to runs-on Self-hosted runners (#23164)
new 649aeea963d Updated pr-bot-update-reviewers workflow (#182) (#23151)
new fbaa14e201e [GitHub Actions] - Updates in Stale to runs-on Self-hosted runners (#23155)
new 68588280c7b Updated pr-bot-new-prs Workflow(#179) (#23146)
new 56d744d9d62 [Playground]GKE autoscaling nodes (#24593)
new 1ff2579caef Bump cloud.google.com/go/spanner from 1.41.0 to 1.42.0 in /sdks (#24673)
new c2a6d0613de Ensure SDK suffix to avoid mis-labels of Go. (#24698)
new d62601b5d31 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#24687)
new 9346102d092 Implemented SchemaTransforms for SingleStoreIO (#24290)
new 48251a98b11 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks (#24716)
new a5d79bad000 Add Python Nexmark tests to Jenkins
new b25f3c4a059 Skip Broken Queries
new fe3eb147cfa Remove unused nexmark imports
new 02d2f4eccfd Fix export to influx on jenkins
new c4fa0732306 Ensure WRITE_TRUNCATE BigQuery loads to the same table are processed together. (#24536)
new 668b4049634 Bump qs from 6.5.2 to 6.5.3 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel (#24545)
new a6dde3b979d Bump github.com/aws/aws-sdk-go-v2/config from 1.18.5 to 1.18.6 in /sdks (#24714)
new e7a90cc5949 [24664] Fix ProcessBundleBenchmarkTest.testStateWithCaching (#24669)
new c36bbc2667c flaky -> flake (#24732)
new 6e5d0b7bbac Support writing to BQ via Storage Write API with SchemaTransforms (#23988)
new a576327b661 Add some alphabet companies that make heavy use of Beam. (#24719)
new 69f0d460d9e [BigQueryIO] Pick provided schema if existing table has null schema (#24699)
new 428194ce4f9 Fix link to videos-and-podcasts page. (#24733)
new 5c66a740510 DebeziumIO schema transform (#24629)
new 557a108a126 jamm on -javaagent instead of classpath
new aa6afcb0837 Bump actions/setup-java from 3.6.0 to 3.8.0 (#24559)
new bf42842f155 Fix some typos (#24736)
new a8f61937057 Adding Setup python to Dataflow job in Java Tests (#24712)
new 86b27e2967c Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#24740)
new 14c35840a35 Add a couple of well-known Google properties that use Beam extensively.
new 93dbd7345a8 [#24789] Handle missing FullValue wrapper.
The 189 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.