You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "zhilinli123 (via GitHub)" <gi...@apache.org> on 2023/01/30 10:18:51 UTC
[GitHub] [incubator-seatunnel] zhilinli123 opened a new pull request, #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
zhilinli123 opened a new pull request, #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004
<!--
Thank you for contributing to SeaTunnel! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/incubator-seatunnel/issues).
- Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
-->
## Purpose of this pull request
<!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
## Check list
* [ ] Code changed are covered with tests, or it does not need tests for reason:
* [ ] If any new Jar binary package adding in your PR, please add License Notice according
[New License Guide](https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/contribution/new-license.md)
* [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
* [ ] If you are contributing the connector code, please check that the following files are updated:
1. Update change log that in connector document. For more details you can refer to [connector-v2](https://github.com/apache/incubator-seatunnel/tree/dev/docs/en/connector-v2)
2. Update [plugin-mapping.properties](https://github.com/apache/incubator-seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it
3. Update the pom file of [seatunnel-dist](https://github.com/apache/incubator-seatunnel/blob/dev/seatunnel-dist/pom.xml)
* [ ] Update the [`release-note`](https://github.com/apache/incubator-seatunnel/blob/dev/release-note.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "CalvinKirs (via GitHub)" <gi...@apache.org>.
CalvinKirs commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1418581523
@liugddx In the case that the contributor is still following up, we'd better help him complete this contribution instead of creating a new PR. This is not a good practice.
CC @Hisoka-X @ashulin
@zhilinli123 I'm sorry for this. and looking forward to your next contribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1411415266
> > > > > 我在本地重新调试了@TyrantLucifer
> > > >
> > > >
> > > > 等待CICD。
> > >
> > >
> > > 我的 CICD 似乎失败了。我不知道是什么原因造成的
> >
> >
> > https://github.com/apache/incubator-seatunnel/actions/runs/4045655710/jobs/6977367847
>
> Can you give me some tips to solve the problem? I don't see the underlying problem
Case module
Case details
Case error reason
When fake connector defined row, data generation based on this configuration item takes precedence
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X closed pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X closed pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
URL: https://github.com/apache/incubator-seatunnel/pull/4004
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "liugddx (via GitHub)" <gi...@apache.org>.
liugddx commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1416919356
#4061
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1408351199
issues: https://github.com/apache/incubator-seatunnel/issues/4002
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1411341755
> > > I re-debugged it locally @TyrantLucifer
> >
> >
> > Waiting CICD.
>
> my CICD seems to have failed. I don't know what caused it
https://github.com/apache/incubator-seatunnel/actions/runs/4045655710/jobs/6977367847
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 commented on code in PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#discussion_r1092962791
##########
seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSourceReader.java:
##########
@@ -70,11 +70,8 @@ public void pollNext(Collector<SeaTunnelRow> output) throws InterruptedException
synchronized (output.getCheckpointLock()) {
FakeSourceSplit split = splits.poll();
if (null != split) {
- // Generate a random number of rows to emit.
- List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(split.getRowNum());
Review Comment:
remove unused method `generateFakedRows`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on a diff in pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#discussion_r1095249438
##########
seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSourceReader.java:
##########
@@ -70,11 +70,8 @@ public void pollNext(Collector<SeaTunnelRow> output) throws InterruptedException
synchronized (output.getCheckpointLock()) {
FakeSourceSplit split = splits.poll();
if (null != split) {
- // Generate a random number of rows to emit.
- List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(split.getRowNum());
Review Comment:
@hailin0 Can you help me locate the reason why my CICD failed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 commented on code in PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#discussion_r1094126676
##########
seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSourceReader.java:
##########
@@ -70,11 +70,8 @@ public void pollNext(Collector<SeaTunnelRow> output) throws InterruptedException
synchronized (output.getCheckpointLock()) {
FakeSourceSplit split = splits.poll();
if (null != split) {
- // Generate a random number of rows to emit.
- List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(split.getRowNum());
Review Comment:
update test cases
org.apache.seatunnel.connectors.seatunnel.fake.source.FakeDataGeneratorTest#testRowDataParse
```java
//List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(fakeConfig.getRowNum());
List<SeaTunnelRow> seaTunnelRows = new ArrayList<>();
fakeDataGenerator.collectFakedRows(fakeConfig.getRowNum(), new Collector<SeaTunnelRow>() {
@Override
public void collect(SeaTunnelRow record) {
seaTunnelRows.add(record);
}
@Override
public Object getCheckpointLock() {
throw new UnsupportedOperationException();
}
});
Assertions.assertIterableEquals(expected, seaTunnelRows);
```
##########
seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSourceReader.java:
##########
@@ -70,11 +70,8 @@ public void pollNext(Collector<SeaTunnelRow> output) throws InterruptedException
synchronized (output.getCheckpointLock()) {
FakeSourceSplit split = splits.poll();
if (null != split) {
- // Generate a random number of rows to emit.
- List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(split.getRowNum());
Review Comment:
@zhilinli123
update test cases
org.apache.seatunnel.connectors.seatunnel.fake.source.FakeDataGeneratorTest#testRowDataParse
```java
//List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(fakeConfig.getRowNum());
List<SeaTunnelRow> seaTunnelRows = new ArrayList<>();
fakeDataGenerator.collectFakedRows(fakeConfig.getRowNum(), new Collector<SeaTunnelRow>() {
@Override
public void collect(SeaTunnelRow record) {
seaTunnelRows.add(record);
}
@Override
public Object getCheckpointLock() {
throw new UnsupportedOperationException();
}
});
Assertions.assertIterableEquals(expected, seaTunnelRows);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 commented on code in PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#discussion_r1092956556
##########
seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeDataGenerator.java:
##########
@@ -92,6 +93,21 @@ public List<SeaTunnelRow> generateFakedRows(int rowNum) {
return seaTunnelRows;
}
+ /**
+ * @param rowNum The number of pieces of data to be generated by the current task
+ * @param output Data collection and distribution
+ **/
+ public void collectFakedRows(int rowNum, Collector<SeaTunnelRow> output) {
+ if (fakeConfig.getFakeRows() != null) {
+ for (FakeConfig.RowData rowData : fakeConfig.getFakeRows()) {
+ output.collect(convertRow(rowData));
+ }
+ }
+ for (int i = 0; i < rowNum; i++) {
+ output.collect(randomRow());
+ }
Review Comment:
```suggestion
if (fakeConfig.getFakeRows() != null) {
for (FakeConfig.RowData rowData : fakeConfig.getFakeRows()) {
output.collect(convertRow(rowData));
}
} else {
for (int i = 0; i < rowNum; i++) {
output.collect(randomRow());
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "liugddx (via GitHub)" <gi...@apache.org>.
liugddx commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1418584773
> @liugddx In the case that the contributor is still following up, we'd better help him complete this contribution instead of creating a new PR. This is not a good practice. CC @Hisoka-X @ashulin
>
> @zhilinli123 I'm sorry for this. and looking forward to your next contribution.
Excuse me. I originally wanted to add content to this pr, but since I don't have permission, I opened another pr to try. cc @zhilinli123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1408390662
Unfortunately, your code style has some problems. Give you some tips:
1. Download `checkstyle` plugin in idea and enabled it in the whole project. `checkstyle.xml` you can find it at `tools/checkstyle`
2. Before submit pull request, use the command `mvn checkstyle:check` to check your code style is or not right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1408349082
@liugddx Help review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1408386571
Thank you for your contribution. Overall looks good to me. Let's waiting CICD
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1411409838
> > > > 我在本地重新调试了@TyrantLucifer
> > >
> > >
> > > 等待CICD。
> >
> >
> > 我的 CICD 似乎失败了。我不知道是什么原因造成的
>
> https://github.com/apache/incubator-seatunnel/actions/runs/4045655710/jobs/6977367847
Can you give me some tips to solve the problem? I don't see the underlying problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 commented on code in PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#discussion_r1094126676
##########
seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSourceReader.java:
##########
@@ -70,11 +70,8 @@ public void pollNext(Collector<SeaTunnelRow> output) throws InterruptedException
synchronized (output.getCheckpointLock()) {
FakeSourceSplit split = splits.poll();
if (null != split) {
- // Generate a random number of rows to emit.
- List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(split.getRowNum());
Review Comment:
@zhilinli123
update test cases and remove unused method `generateFakedRows`
org.apache.seatunnel.connectors.seatunnel.fake.source.FakeDataGeneratorTest#testRowDataParse
```java
//List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(fakeConfig.getRowNum());
List<SeaTunnelRow> seaTunnelRows = new ArrayList<>();
fakeDataGenerator.collectFakedRows(fakeConfig.getRowNum(), new Collector<SeaTunnelRow>() {
@Override
public void collect(SeaTunnelRow record) {
seaTunnelRows.add(record);
}
@Override
public Object getCheckpointLock() {
throw new UnsupportedOperationException();
}
});
Assertions.assertIterableEquals(expected, seaTunnelRows);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1411321505
> > I re-debugged it locally @TyrantLucifer
>
> Waiting CICD.
my CICD seems to have failed. I don't know what caused it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 commented on code in PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#discussion_r1093130963
##########
seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSourceReader.java:
##########
@@ -70,11 +70,8 @@ public void pollNext(Collector<SeaTunnelRow> output) throws InterruptedException
synchronized (output.getCheckpointLock()) {
FakeSourceSplit split = splits.poll();
if (null != split) {
- // Generate a random number of rows to emit.
- List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(split.getRowNum());
Review Comment:
@zhilinli123 here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on a diff in pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#discussion_r1094150573
##########
seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSourceReader.java:
##########
@@ -70,11 +70,8 @@ public void pollNext(Collector<SeaTunnelRow> output) throws InterruptedException
synchronized (output.getCheckpointLock()) {
FakeSourceSplit split = splits.poll();
if (null != split) {
- // Generate a random number of rows to emit.
- List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(split.getRowNum());
Review Comment:
Thank you for your help. I will submit it again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on a diff in pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer commented on code in PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#discussion_r1094165537
##########
seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSourceReader.java:
##########
@@ -70,11 +70,8 @@ public void pollNext(Collector<SeaTunnelRow> output) throws InterruptedException
synchronized (output.getCheckpointLock()) {
FakeSourceSplit split = splits.poll();
if (null != split) {
- // Generate a random number of rows to emit.
- List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(split.getRowNum());
Review Comment:
CICD had been starting. Let's waiting CICD.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on a diff in pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on code in PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#discussion_r1093906466
##########
seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSourceReader.java:
##########
@@ -70,11 +70,8 @@ public void pollNext(Collector<SeaTunnelRow> output) throws InterruptedException
synchronized (output.getCheckpointLock()) {
FakeSourceSplit split = splits.poll();
if (null != split) {
- // Generate a random number of rows to emit.
- List<SeaTunnelRow> seaTunnelRows = fakeDataGenerator.generateFakedRows(split.getRowNum());
Review Comment:
> remove unused method `generateFakedRows`
The generateFakedRows method is used for other test cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1408880888
I re-debugged it locally
@TyrantLucifer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1418623780
> @liugddx在贡献者还在跟进的情况下，我们最好帮助他完成这次贡献，而不是创建一个新的 PR。这不是一个好的做法。 CC@Hisoka-X @ashulin
>
> @zhilinli123 我为此感到抱歉。并期待您的下一次贡献。
> @liugddx在捐献者还在跟进的情况下，我们最好的帮助他完成了这次捐献，而不是创造一个新的公关。这不是一个好的做法。 CC@Hisoka-X @ashulin
>
> @zhilinli123 我为此感到拥抱。并期待您的下一次奉献。
Thank you for your help and I'd love to contribute
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1410438327
> I re-debugged it locally @TyrantLucifer
Waiting CICD.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zhilinli123 commented on pull request #4004: [Improve][Connector-fake] Optimizing Data Generation Strategies #4002
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on PR #4004:
URL: https://github.com/apache/incubator-seatunnel/pull/4004#issuecomment-1411701999
> > > > > > 我在本地重新测试了@TyrantLucifer
> > > > >
> > > > >
> > > > > 等候 CICD。
> > > >
> > > >
> > > > 我的CICD好像失败了。我不知道是什么原因造成的
> > >
> > >
> > > https://github.com/apache/incubator-seatunnel/actions/runs/4045655710/jobs/6977367847
> >
> >
> > 你能给我一些解决问题的建议吗？我没有看到潜在的问题
>
> 案例模块 
>
> 案例详情 
>
> 案例错误原因 
>
> fake connector定义行时，优先根据该配置项生成数据
<img width="1410" alt="image" src="https://user-images.githubusercontent.com/76689593/215998936-d4108b43-6642-4c50-90ce-00b468c27344.png">
The code in the current project is not clear from where it was loaded
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org