You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@groovy.apache.org by pa...@apache.org on 2026/08/13 23:44:42 UTC
(groovy-website) branch asf-site updated: tweaks to GeaFlow blog post
This is an automated email from the ASF dual-hosted git repository.
paulk-asert pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/groovy-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 6f09268 tweaks to GeaFlow blog post
6f09268 is described below
commit 6f0926803f4cc69a0166e78d9881785e9ee94e29
Author: Paul King <pa...@asert.com.au>
AuthorDate: Fri Aug 14 09:44:31 2026 +1000
tweaks to GeaFlow blog post
---
site/src/site/blog/groovy-geaflow.adoc | 109 +++++++++----------------
site/src/site/blog/img/geaflow_influencers.png | Bin 0 -> 273809 bytes
2 files changed, 38 insertions(+), 71 deletions(-)
diff --git a/site/src/site/blog/groovy-geaflow.adoc b/site/src/site/blog/groovy-geaflow.adoc
index 8133c43..7da9262 100644
--- a/site/src/site/blog/groovy-geaflow.adoc
+++ b/site/src/site/blog/groovy-geaflow.adoc
@@ -60,7 +60,7 @@ are on Maven Central. For Gradle, the relevant part of the build file is:
[source,groovy]
----
dependencies {
- implementation "org.apache.groovy:groovy:5.0.8"
+ implementation "org.apache.groovy:groovy:6.0.0-beta-2"
implementation "org.apache.geaflow:geaflow-api:$geaflowVersion"
implementation "org.apache.geaflow:geaflow-pdata:$geaflowVersion"
implementation "org.apache.geaflow:geaflow-cluster:$geaflowVersion"
@@ -97,7 +97,7 @@ as captured in this map:
[source,groovy]
----
-var follows = [
+val follows = [
'Emily Seebohm' : ['Kaylee McKeown', 'Regan Smith'],
'Kylie Masse' : ['Kaylee McKeown', 'Regan Smith', 'Ingrid Wilm'],
'Regan Smith' : ['Kaylee McKeown', 'Katharine Berkoff'],
@@ -108,7 +108,9 @@ var follows = [
----
The keys of the map are the followers, and the values are the lists of
-swimmers they follow. Who are the influencers in this network?
+swimmers they follow. (If you haven't seen `val` before, it arrived in
+Groovy 6: like `var`, it declares a type-inferred local variable, but
+a final one.) Who are the influencers in this network?
The classic algorithm for answering that question is
https://en.wikipedia.org/wiki/PageRank[PageRank], originally used
to rank web pages, but equally applicable to any directed graph.
@@ -130,11 +132,11 @@ value which we won't use:
[source,groovy]
----
-List<IVertex<String, Double>> vertices = follows.keySet().collect {
- (IVertex<String, Double>) new ValueVertex<>(it, 1d)
+val vertices = follows.keySet().collect {
+ new ValueVertex<>(it, 1d)
}
-List<IEdge<String, Integer>> edges = follows.collectMany { follower, followed ->
- followed.collect { (IEdge<String, Integer>) new ValueEdge<>(follower, it, 1) }
+val edges = follows.collectMany { follower, followed ->
+ followed.collect { new ValueEdge<>(follower, it, 1) }
}
----
@@ -155,20 +157,19 @@ static class PRTask implements PipelineTask {
void execute(IPipelineTaskContext ctx) {
// ... follows, vertices, edges as above ...
- PWindowStream<IVertex<String, Double>> vertexSource =
+ val vertexSource =
ctx.buildSource(new CollectionSource<IVertex<String, Double>>(vertices),
- AllWindow.<IVertex<String, Double>> getInstance())
- PWindowStream<IEdge<String, Integer>> edgeSource =
+ AllWindow.getInstance())
+ val edgeSource =
ctx.buildSource(new CollectionSource<IEdge<String, Integer>>(edges),
- AllWindow.<IEdge<String, Integer>> getInstance())
+ AllWindow.getInstance())
- def graphViewDesc = GraphViewBuilder.createGraphView(GraphViewBuilder.DEFAULT_GRAPH)
+ val graphViewDesc = GraphViewBuilder.createGraphView(GraphViewBuilder.DEFAULT_GRAPH)
.withShardNum(1)
.withBackend(IViewDesc.BackendType.Memory)
.build()
- PGraphWindow<String, Double, Integer> graph =
- ctx.buildWindowStreamGraph(vertexSource, edgeSource, graphViewDesc)
+ val graph = ctx.buildWindowStreamGraph(vertexSource, edgeSource, graphViewDesc)
graph.compute(new PRAlgorithm(10, 0.85d))
.compute(1)
@@ -180,6 +181,10 @@ static class PRTask implements PipelineTask {
A few things to note:
+* We're writing idiomatic dynamic Groovy here — no explicit types are
+ needed for the sources or the graph. If you prefer compile-time type
+ checking, adding `@CompileStatic` works too, at the cost of a few
+ explicit generic types to help inference along.
* `AllWindow` tells GeaFlow to read all of the data as one batch,
giving _static_ graph computation. Size-based tumbling windows are
used instead when processing _streaming_ data incrementally.
@@ -223,18 +228,18 @@ static class PRComputeFunction implements VertexCentricComputeFunction<String, D
@Override
void compute(String vertexId, Iterator<Double> messages) {
- def vertex = context.vertex().get()
- def outEdges = context.edges().outEdges
+ val vertex = context.vertex().get()
+ val outEdges = context.edges().outEdges
if (context.iterationId == 1L) {
if (outEdges) {
context.sendMessageToNeighbors(vertex.value / outEdges.size() as double)
}
} else {
- double sum = messages.sum(0d) as double
+ double sum = messages.sum(0d)
double pr = sum * alpha + (1 - alpha)
context.setNewVertexValue(pr)
if (outEdges) {
- context.sendMessageToNeighbors(pr / outEdges.size() as double)
+ context.sendMessageToNeighbors(pr / outEdges.size())
}
}
}
@@ -270,10 +275,10 @@ and runs it:
[source,groovy]
----
static void main(String[] args) {
- def environment = EnvironmentFactory.onLocalEnvironment()
- def pipeline = PipelineFactory.buildPipeline(environment)
+ val environment = EnvironmentFactory.onLocalEnvironment()
+ val pipeline = PipelineFactory.buildPipeline(environment)
pipeline.submit(new PRTask())
- def result = pipeline.execute()
+ val result = pipeline.execute()
result.get()
environment.shutdown()
System.exit(0)
@@ -296,58 +301,20 @@ The two swimmers who traded the Olympic record back and forth,
Kaylee McKeown and Regan Smith, come out on top — as our hypothetical
follower graph intended. Phew!
-== A note on lambdas, closures and serialization
+image:img/geaflow_influencers.png["From follows map to influence ranking with Groovy and GeaFlow",width=800]
+
+== A note on serialization
GeaFlow is a distributed system. Tasks and functions are serialized
-(using https://github.com/EsotericSoftware/kryo[kryo]) and shipped
-across process boundaries — even in local mode, where the in-process
-components talk to each other over the same RPC protocol they'd use on
-a cluster. That has consequences for how we write our Groovy code.
-GeaFlow's functional interfaces like `PipelineTask` and `SinkFunction`
-all extend `Serializable`, and we found the following
-(with static compilation enabled via `@CompileStatic`):
-
-|===
-|Style |Example |Works?
-
-|Class implementing the interface
-|`pipeline.submit(new PRTask())`
-|✅
-
-|Native lambda expression
-|`pipeline.submit((IPipelineTaskContext ctx) -> runJob(ctx))`
-|✅
-
-|Method reference
-|`pipeline.submit(SubmitVariants::runJob)`
-|❌ (fixed in Groovy 6)
-
-|Closure
-|`pipeline.submit { IPipelineTaskContext ctx -> runJob(ctx) }`
-|❌
-|===
-
-Classes and lambda expressions work: Groovy compiles a lambda targeting
-a `Serializable` functional interface into a serializable lambda, just
-like Java does, and GeaFlow's kryo configuration knows how to transport
-those. A nice pattern is to hoist the body of your job into a static
-method and submit it with a one-line lambda.
-
-Method references surprised us: Groovy currently misses marking them
-as serializable, even when the target interface extends `Serializable`
-(where Java would). Investigating this while writing this post
-led us to https://issues.apache.org/jira/browse/GROOVY-11993[GROOVY-11993],
-which has already fixed this for the upcoming Groovy 6 —
-we verified that method references work fine with GeaFlow using a
-recent Groovy 6 pre-release.
-
-Closures (whether coerced explicitly with `as PipelineTask` or
-implicitly) are converted to JDK dynamic proxies, which kryo doesn't
-currently transport, so avoid them for functions that cross the RPC
-boundary — which includes tasks, sources, sinks, and compute functions.
-Interestingly, the closure proxies _are_ serializable with plain Java
-serialization, so teaching GeaFlow's kryo setup about them would be a
-nice little contribution for a Groovy-friendly volunteer.
+and shipped across process boundaries — even in local mode. So, write
+your tasks, sources, sinks, and compute functions as classes (as we did
+above), or, with static compilation, as lambda expressions or method
+references — all of these serialize cleanly. Avoid closures for these
+particular functions: they are converted to dynamic proxies which
+GeaFlow won't currently transport. More details, including a related
+method reference issue affecting earlier Groovy versions
+(https://issues.apache.org/jira/browse/GROOVY-11993[GROOVY-11993],
+fixed in Groovy 6), can be found in the accompanying repo.
== What else is in the box?
diff --git a/site/src/site/blog/img/geaflow_influencers.png b/site/src/site/blog/img/geaflow_influencers.png
new file mode 100644
index 0000000..bfe012f
Binary files /dev/null and b/site/src/site/blog/img/geaflow_influencers.png differ