You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/03/06 19:23:36 UTC

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Spark #4350

See <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/4350/display/redirect?page=changes>

Changes:

[herohde] Initial sketches of a Go SDK

[herohde] Initial version of the direct style w/ direct runner. Incomplete.

[herohde] Add Data as UserFn context w/ immediate value.

[herohde] Added no-I/O wordcount for profiling.

[herohde] Fleshed out possible approach to generic transformations.

[herohde] Add “dag” example that use multiplexing and side input.

[herohde] Added a more complex DAG example.

[herohde] Add yatzy example with more complex construction-time setup

[herohde] Add proto for Fn API

[herohde] Add beam.Composite helper for the most common pattern to align with java

[herohde] Move pipeline-construction time errors into an accumulator

[herohde] Add Dataflow job and Fn API clients. Incomplete.

[herohde] Add binary cross-compile and upload to Dataflow runner. Incomplete.

[herohde] Add tentative runner indirection (default: local).

[herohde] Made data flow runner detect user main for cross-compilation.

[herohde] Remove error accumulation in favor of panic.

[herohde] Improve Dataflow translation of coders, side input and composite names.

[herohde] Fix name for AsView.

[herohde] Use 2 grpc endpoints in harness

[herohde] Add gRPC harness logging

[herohde] Flesh out harness and serialization further.

[herohde] Made the dataflow runner wait for job termination by default

[herohde] beam:

[herohde] beam:

[herohde] combinefn.go: fix compilation issues

[herohde] Improve dataflow serialization and execution. Incomplete.

[herohde] Sleep 30 sec in wordcap to allow logs to propagate to Cloud Logging.

[herohde] Move the 30s sleep for logging to the harness instead of in WordCap.

[herohde] Post-review updates.

[herohde] Doc updates.

[herohde] Flesh out coders. Incomplete.

[herohde] Added prototype implementation of more coders and the runner source.

[herohde] dofn: illustrates how dofns are written.

[herohde] beam: add viewfn and windowfn to side inputs match support Beam 1.0

[herohde] dofn: timers

[herohde] Complete revamp: coders, graph and execution use element-wise

[herohde] Fix coder encoding for Dataflow side input. Otherwise, the job is

[herohde] Added more godoc comments to graph types.

[herohde] Added more comments plus made local GBK use coder equality.

[herohde] Added Flatten support and “forest” example that uses it.

[herohde] Move bigqueryio to defunct

[herohde] Make forest example print less

[herohde] Add external struct registry and serialization.

[herohde] Updated comments in node.go.

[herohde] Replace real type with 'full type' since that's the current term.

[herohde] Refactor Fn API dependency.

[herohde] Added more comments to the runner/dataflow and runner/beamexec packages

[herohde] Fix most go vet issues

[herohde] Make core operations panic to cut down on the error propagation

[herohde] Add more comments to the graph package.

[herohde] Add DoFn wrapper to handle either function or (ptr to) struct

[herohde] Fix remaining go vet warnings.

[herohde] Code review for beam/graph/coder package.

[herohde] Code review of the runtime/graphx package.

[herohde] Remove Data options in favor of using a Fn struct

[herohde] Code review of the beam/graph/userfn package.

[herohde] Code review for beam/graph package.

[herohde] godoc for runtime/graphx

[herohde] Add support for []T and Combine functions

[herohde] Add adapted documentation from the Java SDK to the beam package

[herohde] Update snapshot of Fn API.

[herohde] Add experiments flag to the Dataflow runner

[herohde] Remove context arg from beamexec.Init

[herohde] Migration to Runner API.

[herohde] Add support for creating DOT graphs.

[herohde] Make pretty printing of types and coders more concise

[herohde] Add flexible Signature to aid type checking

[herohde] Adding unit testability to harness translation.

[herohde] Fix crash due to initialization order

[herohde] Add CreateValues and Impulse

[herohde] Add Runner API support for WindowingStrategy.

[herohde] Run goimports on baseline.

[herohde] Fix encoding of global window strategy.

[herohde] Ensure the windowed value is atomically encoded.

[herohde] Limit gRPC messages to max size.

[herohde] Developer conveniences for running jobs.

[herohde] Fix sends to not close the network channel.

[herohde] Add re-iterable side input

[herohde] Add per-key Combine

[herohde] Add Min

[herohde] Reorganize non-user-facing code into core

[herohde] Make type register reject unnamed or predeclared types

[herohde] Add type specialization tool

[herohde] Don't run grpc plugin in generate phase.

[herohde] Fix import reference path for runner API proto.

[herohde] Revamp runner registration as _ imports

[herohde] Add stats.Max and Mean

[herohde] Add global pipeline options

[herohde] Unify global and per-key combiners

[herohde] Add beam convenience wrapper for imports and runner selection

[herohde] Add session recording and CPU profiling to harness.

[herohde] Add ptest and passert for testing pipelines

[herohde] Add GCS and glob support to textio

[herohde] Add BigQuery IO and examples

[herohde] Adds a session runner for testing.

[herohde] Add Partition and dynamic functions

[herohde] Adding example that returns 10 words that contain provided search

[herohde] Remove duplicate LOG line

[herohde] Enable Combine Fns in Dataflow runner by modifying translation.

[herohde] Fixing type bug by dropping T and using underlying type of value in

[herohde] Adding graph validation at build time.

[herohde] Import the Fn API changes.

[herohde] Simple changes to support new Fn API coder changes.

[herohde] Update translator to work with new Fn API changes.

[herohde] Use appropriate equality tests.

[herohde] Fix test to not use path of package.

[herohde] Renaming directory to match package name.

[herohde] Fixing random nits in comments.

[herohde] Modify build command to avoid bash.

[herohde] Fixing selected golint issues.

[herohde] Addressing import review comments.

[herohde] Add coder specialization for bytes/strings.

[herohde] Adding unit tests to stats.

[herohde] Fixing typo.

[herohde] Add beam.External

[herohde] Fix grpc.Dial calls to block properly.

[herohde] Creates a symtab verifier by running Sym2Addr and Addr2Sym in a binary.

[herohde] Add spec field to help interpretation of payload.

[herohde] Use beam.T alias for typex.T etc with Go 1.9

[herohde] Move shared GCP options to a separate package

[herohde] Update portability protos

[herohde] Remove old source/sink from beam package

[herohde] Add context-aware logging for both pipeline-submission time and runtime

[herohde] Fix coder inference for strings.

[herohde] Improve tornadoes example

[herohde] Fix beam.External to map arguments correctly.

[herohde] Added comments to yatzy and forest

[herohde] Add comments to tornadoes from the java counterpart

[herohde] Rename Pipeline Composite to Scope

[herohde] Add 3 progressive wordcount examples

[herohde] Clarify comments in wordcount pipelines

[herohde] Add apache 2.0 license to files

[herohde] Updates to examples.

[herohde] Adding more godoc for the main beam package.

[herohde] Update to new proto structure

[herohde] Split Combine and fields in to global and per-key variants

[herohde] Refactor Flatten of a single []T into Explode

[herohde] Rename local runner to direct runner

[herohde] Fix argument index error in ParDo execution

[herohde] Add Apache copyright header to files that need it.

[herohde] Made debug.Head not just work per bundle

[herohde] Impose a total ordering on Fn parameters.

[herohde] Rename Dedup to Distinct for consistency with other SDKs

[herohde] Add coder to model coder translation

[herohde] Simplify harness coder translation

[herohde] Split Pipeline into Pipeline and Scope

[herohde] Relocate Go SDK code

[herohde] Fix Go SDK maven build

[herohde] Move Go SKD to latest version of bigquery

[herohde] Add Go SDK container image

[herohde] Add Go SDK README

[herohde] Update version for Go Dataflow pipelines

[herohde] Make Scope a value type

[herohde] Add Go graph/pipeline translation

[herohde] Stage Go model pipeline for Dataflow

[herohde] Use pipeline unmarhaller in runtime harness

[herohde] CR: [BEAM-3287] Use model pipelines in Go SDK

[herohde] CR: [BEAM-3287] Use model pipelines in Go SDK

[herohde] Fix name of syscallx ErrUnsupported

[herohde] Allow any named type to be registered and serialized as external

[herohde] Add more package comments for core packages

[herohde] Make Go SDK External a graph primitive

[herohde] Cache Go runtime symbol lookups

[wcn] Fix storagePath variable.

[wcn] BEAM-3368 fix translation for external

[robertwb] [BEAM-3356] Add Go SDK int and varint custom coders (#4276)

[lcwik] BEAM-3361 Increase Go gRPC message size

[herohde] Go SDK runtime revamp

[lcwik] Add a few function call overhead benchmarks

[lcwik] Add type-specialized emitters

[lcwik] BEAM-3324 improve symtab memory usage

[lcwik] BEAM-3324 improve symtab memory usage

[lcwik] BEAM-3324 improve symtab memory usage

[lcwik] Store objects in pool so they can be reused.

[lcwik] Add builtin varint coder

[herohde] Type-specialize custom decoders and encoders in Go SDK runtime

[herohde] Type-specialize iterators and side input in the Go SDK

[herohde] Add warnings if Go runtime registrations are overwritten

[herohde] Add reusable element coders in Go SDK runtime

[wcn] Updated translater to preserve payload and its URN.

[herohde] Initial version of type-specialized general reflective calls

[herohde] Add general-purpose untyped callers in Go SDK runtime

[herohde] Use fast caller for filter transform predicate

[herohde] CR: Clarified comment on caller template

[herohde] Fix value encoding for Create

[lcwik] BEAM-3473: Fix GroupByKey iterators to be initialized.

[lcwik] BEAM-3474 Include stacks in panic messages.

[lcwik] BEAM-3299: Add source reporting support.

[lcwik] Remove GetId() call from under lock.

[lcwik] Add additional comments about concurrency invariants.

[lcwik] Add initialization of active plans map.

[lcwik] Renamed Go runtime Caller to Func and added name

[lcwik] Use reflectx.Func as the fundamental function representation

[lcwik] CR: fix DynFn comments

[lcwik] CR: fix comments

[herohde] Avoid reflect.Value conversions in Go runtime

[wcn] Allow grpcx.Dial to support overrides.

[rober] Use a typeswitch instead of reflect.Convert when encoding strings or

[github] Update coder.go

[robert] Replace reflective convert to direct convert.

[rober] Fix beam.Combine to combine globally

[herohde] Add optional function registration to Go SDK runtime

[rober] fixup! Remove reflection from varint codecs

[herohde] Changed core GBK to CoGBK

[herohde] Add CoGBK support to direct runner and Join example

[herohde] [BEAM-3316] Translate bundle descriptors directly to execution plans in

[herohde] Translate CoGBK into GBK for Dataflow and model pipeline runners

[herohde] CR: [BEAM-3302] Support CoGBK in the Go SDK

[herohde] [BEAM-3579] Fix textio.Write

[herohde] CR: fix Go SDK textio.Write

[wcn] Fixing filename.

[ehudm] Integration test for Python HDFS implementation.

[wcn] Improve rendering of DOT diagrams.

[herohde] Update Go SDK coder constants

[tgroh] Make Impulse#create() visible

[tgroh] Add JavaReadViaImpulse to core-construction

[amyrvold] Beam runner inventory, run as a cron job, not on each CL

------------------------------------------
Started by GitHub push by tgroh
Started by GitHub push by tgroh
Started by GitHub push by tgroh
Started by GitHub push by tgroh
Started by GitHub push by tgroh
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/ws/>
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 45ff8732b89aa8000850ca9ce2a0fa4fea1d82ab (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 45ff8732b89aa8000850ca9ce2a0fa4fea1d82ab
Commit message: "Merge master into go-sdk"
 > git rev-list --no-walk 5b99fe2ea44e1a0d8a6f802ca757d1a1cfab91cc # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/ws/> at hudson.remoting.Channel@707da680:beam5: hudson.remoting.ProxyException: hudson.maven.MavenModuleSetBuild$MavenExecutionException: org.apache.maven.project.ProjectBuildingException: Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 21, column 11

	at hudson.FilePath.act(FilePath.java:993)
	at hudson.FilePath.act(FilePath.java:975)
	at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
	at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
	at hudson.model.Run.execute(Run.java:1724)
	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: hudson.maven.MavenModuleSetBuild$MavenExecutionException: org.apache.maven.project.ProjectBuildingException: Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 21, column 11

	at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
	at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
	at hudson.remoting.UserRequest.perform(UserRequest.java:207)
	at hudson.remoting.UserRequest.perform(UserRequest.java:53)
	at hudson.remoting.Request$2.run(Request.java:358)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to beam5
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
		at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
		at hudson.remoting.Channel.call(Channel.java:908)
		at hudson.FilePath.act(FilePath.java:986)
		at hudson.FilePath.act(FilePath.java:975)
		at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
		at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
		at hudson.model.Run.execute(Run.java:1724)
		at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: org.apache.maven.project.ProjectBuildingException: Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 21, column 11

	at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:364)
	at hudson.maven.MavenEmbedder.buildProjects(MavenEmbedder.java:361)
	at hudson.maven.MavenEmbedder.readProjects(MavenEmbedder.java:331)
	at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1328)
	... 10 more
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user amyrvold@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user wcn@google.com