You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/07/25 05:57:56 UTC
[GitHub] [incubator-seatunnel] EricJoy2048 opened a new issue, #2261: [ST-Engine][LogicalPlan] Generate LogicalPlan from JobConfig file
EricJoy2048 opened a new issue, #2261:
URL: https://github.com/apache/incubator-seatunnel/issues/2261
### Search before asking
- [X] I had searched in the [feature](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
### Description
We define job pipeline by job config file, So the first thing the SeaTunnelClient need to do parse the job config file and generate the action list. `action` is similar to the `operator` in Flink, It is an encapsulation of the `SeaTunnel API`. An action contains an instance of `SeaTunnelSource` or `SeaTunnelTransform` or `SeaTunnelSink`. Every action need to know the upstream of it self.
```
public interface Action extends Serializable {
@NonNull
String name();
void setName(@NonNull String name);
@NonNull
List<Action> upstream();
void addUpstream(@NonNull Action action);
}
```
We support `SourceAction` 、`SinkAction` and `TransformAction` yet.
### Usage Scenario
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] 2013650523 commented on issue #2261: [ST-Engine][LogicalPlan] Generate LogicalPlan from JobConfig file
Posted by GitBox <gi...@apache.org>.
2013650523 commented on issue #2261:
URL: https://github.com/apache/incubator-seatunnel/issues/2261#issuecomment-1196773065
hi ,please Why does the parallelism of the parallel transformation change from 4 to 2?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on issue #2261: [ST-Engine][LogicalPlan] Generate LogicalPlan from JobConfig file
Posted by GitBox <gi...@apache.org>.
EricJoy2048 commented on issue #2261:
URL: https://github.com/apache/incubator-seatunnel/issues/2261#issuecomment-1197560844
>  hi ,please Why does the parallelism of the parallel transformation change from 4 to 2?
In our design, Only `Source` and `PartitionTransform` Connector can set parallelism. The `PartitionTransform` Connector is used to repartition data and `parallelism` is the target partition number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ashulin commented on issue #2261: [ST-Engine][LogicalPlan] Generate LogicalPlan from JobConfig file
Posted by GitBox <gi...@apache.org>.
ashulin commented on issue #2261:
URL: https://github.com/apache/incubator-seatunnel/issues/2261#issuecomment-1197811483
<img width="814" alt="image" src="https://user-images.githubusercontent.com/36807946/181454461-68803243-9674-4868-801e-9e9afd6ede29.png">
I can't understand what `PartitionTransform` does in the diagram.
If there is a mechanism for `Transform4` to distribute data to `PartitionTransform`, then why don't we distribute to Sink directly?
Wouldn't it be simpler to let the user specify the parallelism of the sink in this case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [ST-Engine][LogicalPlan] Generate LogicalPlan from JobConfig file [seatunnel]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X closed issue #2261: [ST-Engine][LogicalPlan] Generate LogicalPlan from JobConfig file
URL: https://github.com/apache/seatunnel/issues/2261
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org