You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Kenneth Knowles (JIRA)" <ji...@apache.org> on 2017/11/17 23:30:03 UTC
[jira] [Comment Edited] (BEAM-3223) PTransform spec should not
reuse FunctionSpec
[ https://issues.apache.org/jira/browse/BEAM-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16257746#comment-16257746 ]
Kenneth Knowles edited comment on BEAM-3223 at 11/17/17 11:29 PM:
------------------------------------------------------------------
Suppose we have {{PTransformSpec}} and {{UdfSpec}} and {{CoderSpec}} that are all a pair of URN and payload. This is much more type-friendly and is a standard way of avoiding stupid mistakes.
Now they each occur in one place - {{PTransform}}, {{SdkFunctionSpec}}, and {{Coder}}, respectively. So we could just inline the fields. That seems much better to me.
{code}
message Udf {
string urn;
bytes payload;
string environment_id;
}
messsage Coder {
string id;
string urn;
bytes payload;
repeated string component_coder_ids;
}
message PTransform {
.. all the existing stuff ...
string urn;
bytes payload;
string unique_name = 5;
repeated string subtransforms = 2;
map<string, string> inputs = 3;
map<string, string> outputs = 4;
DisplayData display_data = 6;
}
{code}
This makes the differences between the three very obvious.
I am only hesitant because the current protos represent a compromise that made all parties satisfied, but none happy. So bikeshedding is a danger.
was (Author: kenn):
Suppose we have {{PTransformSpec}} and {{UdfSpec}} and {{CoderSpec}} that are all a pair of URN and payload. This is much more type-friendly and is a standard way of avoiding stupid mistakes.
Now they each occur in one place - {{PTransform}}, {{SdkFunctionSpec}}, and {{Coder}}, respectively. So we could just inline the fields. That seems much better to me.
{code}
message Udf {
string urn;
bytes payload;
string environment_id;
}
messsage Coder {
string id;
string urn;
bytes payload;
repeated string component_coder_ids;
}
message PTransform {
.. all the existing stuff ...
string urn;
bytes payload;
}
{code}
This makes the differences between the three very obvious.
I am only hesitant because the current protos represent a compromise that made all parties satisfied, but none happy. So bikeshedding is a danger.
> PTransform spec should not reuse FunctionSpec
> ---------------------------------------------
>
> Key: BEAM-3223
> URL: https://issues.apache.org/jira/browse/BEAM-3223
> Project: Beam
> Issue Type: Sub-task
> Components: beam-model
> Reporter: Henning Rohde
> Assignee: Henning Rohde
> Labels: portability
>
> We should add a new type instead, TransformSpec, say, or just inline a URN and payload. It's confusing otherwise.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)