You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Steiner Patrick <pa...@steiner-buchholz.de> on 2018/02/08 10:53:09 UTC

CoGroupByKey question

Hi all,

I'm trying to combine two PCollections<KV<String, Double>> via CoGroupByKey

When adapting the sample code

final TupleTag<V1> t1 = new TupleTag<>();
final TupleTag<V2> t2 = new TupleTag<>();

PCollection<KV<K, CoGbkResult>> coGbkResultCollection =
         KeyedPCollectionTuple.of(t1, pt1)
         .and(t2, pt2)
         .apply(CoGroupByKey.<K>create());

I do get

final TupleTag<Double> tempTuple = new TupleTag<>();
final TupleTag<Double> humTuple = new TupleTag<>();

PCollection<KV<String, CoGbkResult>> coGbkResultCollection =
         KeyedPCollectionTuple.of(tempTuple, tempPCollection)
         .and(humTuple, humPCollection)
         .apply(CoGroupByKey.<String>create());

which results in

Type mismatch: cannot convert from POutput to 
PCollection<KV<String,CoGbkResult>>

I guess it's a simple, stupid thing I'm either missing or not 
understanding, when combining my two PCollections to one.

Any advice is appreciated

Patrick

Re: CoGroupByKey question

Posted by Steiner Patrick <pa...@steiner-buchholz.de>.

Hi Eugene,

that seemed to be the problem! Thanks a lot!

Patrick

Eugene Kirpichov wrote:
> You typically get this compile error when you have a raw type (e.g. 
> PCollection instead of PCollection<Something>) sneaking somewhere. 
> Perhaps tempPCollection or humPCollection is declared as a PCollection 
> without a type?
>
> On Thu, Feb 8, 2018 at 2:53 AM Steiner Patrick 
> <patrick@steiner-buchholz.de <ma...@steiner-buchholz.de>> wrote:
>
>     Hi all,
>
>     I'm trying to combine two PCollections<KV<String, Double>> via
>     CoGroupByKey
>
>     When adapting the sample code
>
>     final TupleTag<V1> t1 = new TupleTag<>();
>     final TupleTag<V2> t2 = new TupleTag<>();
>
>     PCollection<KV<K, CoGbkResult>> coGbkResultCollection =
>             KeyedPCollectionTuple.of(t1, pt1)
>             .and(t2, pt2)
>             .apply(CoGroupByKey.<K>create());
>
>     I do get
>
>     final TupleTag<Double> tempTuple = new TupleTag<>();
>     final TupleTag<Double> humTuple = new TupleTag<>();
>
>     PCollection<KV<String, CoGbkResult>> coGbkResultCollection =
>             KeyedPCollectionTuple.of(tempTuple, tempPCollection)
>             .and(humTuple, humPCollection)
>             .apply(CoGroupByKey.<String>create());
>
>     which results in
>
>     Type mismatch: cannot convert from POutput to
>     PCollection<KV<String,CoGbkResult>>
>
>     I guess it's a simple, stupid thing I'm either missing or not
>     understanding, when combining my two PCollections to one.
>
>     Any advice is appreciated
>
>
>     Patrick
>

Re: CoGroupByKey question

Posted by Eugene Kirpichov <ki...@google.com>.

You typically get this compile error when you have a raw type (e.g.
PCollection instead of PCollection<Something>) sneaking somewhere. Perhaps
tempPCollection or humPCollection is declared as a PCollection without a
type?

On Thu, Feb 8, 2018 at 2:53 AM Steiner Patrick <pa...@steiner-buchholz.de>
wrote:

> Hi all,
>
> I'm trying to combine two PCollections<KV<String, Double>> via CoGroupByKey
>
> When adapting the sample code
>
> final TupleTag<V1> t1 = new TupleTag<>();
> final TupleTag<V2> t2 = new TupleTag<>();
>
> PCollection<KV<K, CoGbkResult>> coGbkResultCollection =
>         KeyedPCollectionTuple.of(t1, pt1)
>         .and(t2, pt2)
>         .apply(CoGroupByKey.<K>create());
>
> I do get
>
> final TupleTag<Double> tempTuple = new TupleTag<>();
> final TupleTag<Double> humTuple = new TupleTag<>();
>
> PCollection<KV<String, CoGbkResult>> coGbkResultCollection =
>         KeyedPCollectionTuple.of(tempTuple, tempPCollection)
>         .and(humTuple, humPCollection)
>         .apply(CoGroupByKey.<String>create());
>
> which results in
>
> Type mismatch: cannot convert from POutput to
> PCollection<KV<String,CoGbkResult>>
>
> I guess it's a simple, stupid thing I'm either missing or not
> understanding, when combining my two PCollections to one.
>
> Any advice is appreciated
>
>
> Patrick
>