You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/08/02 03:59:23 UTC
[GitHub] [hudi] Mathieu1124 opened a new pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Mathieu1124 opened a new pull request #1901:
URL: https://github.com/apache/hudi/pull/1901
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a pull request.*
## What is the purpose of the pull request
*Add java doc for hudi test suite test classes*
## Brief change log
*Add java doc for hudi test suite test classes*
## Verify this pull request
This pull request is a trivial rework / code cleanup without any test coverage.
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pratyakshsharma commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
pratyakshsharma commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r466832582
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/dag/HiveSyncDagGenerator.java
##########
@@ -31,6 +31,9 @@
import org.apache.hudi.integ.testsuite.dag.nodes.HiveSyncNode;
import org.apache.hudi.integ.testsuite.dag.nodes.InsertNode;
+/**
+ * An implementation of {@link WorkflowDagGenerator}, that generate hive sync workflowDag.
Review comment:
ditto.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Mathieu1124 commented on pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
Mathieu1124 commented on pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#issuecomment-670506233
@cheshta2904 @pratyakshsharma I have addressed all your concerns, thanks for your detailed review :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Mathieu1124 commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
Mathieu1124 commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r467025252
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/ITTestBase.java
##########
@@ -48,6 +48,9 @@
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
+/**
+ * Base test class for ITTest help to run cmd and generate data.
Review comment:
done, thanks for your review
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cheshta2904 commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
cheshta2904 commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r466831488
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/ITTestBase.java
##########
@@ -48,6 +48,9 @@
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
+/**
+ * Base test class for ITTest help to run cmd and generate data.
Review comment:
"ITTest help" -> ITTest. Helps
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Mathieu1124 commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
Mathieu1124 commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r467025753
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/dag/ComplexDagGenerator.java
##########
@@ -33,6 +33,9 @@
import org.apache.hudi.integ.testsuite.dag.nodes.ValidateNode;
import org.apache.spark.api.java.JavaRDD;
+/**
+ * An implementation of {@link WorkflowDagGenerator}, that generate complex workflowDag.
Review comment:
> generate -> generates.
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Mathieu1124 commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
Mathieu1124 commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r467025989
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/utils/TestUtils.java
##########
@@ -28,6 +28,9 @@
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.sql.SparkSession;
+/**
+ * A utility class for test purpose.
Review comment:
done
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/utils/TestUtils.java
##########
@@ -45,6 +48,15 @@
return dataGenerator.generateGenericRecords(numRecords);
}
+ /**
+ * Method help to create avro files and save it to file.
+ *
+ * @param jsc {@link JavaSparkContext}.
+ * @param sparkSession {@link SparkSession}.
+ * @param basePath The basePath where files written to.
Review comment:
done
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/TestDFSHoodieTestSuiteWriterAdapter.java
##########
@@ -52,6 +52,9 @@
import org.junit.jupiter.api.Test;
import org.mockito.Mockito;
+/**
+ * An adapter of {@link HoodieTestSuiteWriter} help to test write DFS file.
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wangxianghu commented on pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
wangxianghu commented on pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#issuecomment-679072269
@yanghua thanks for your review. I have addressed all your concerns, please take another when free
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wangxianghu commented on pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
wangxianghu commented on pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#issuecomment-671970849
@yanghua this pr is ready for review now :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pratyakshsharma commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
pratyakshsharma commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r466832642
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/dag/HiveSyncDagGeneratorMOR.java
##########
@@ -31,6 +31,9 @@
import org.apache.hudi.integ.testsuite.dag.nodes.HiveSyncNode;
import org.apache.hudi.integ.testsuite.dag.nodes.InsertNode;
+/**
+ * An implementation of {@link WorkflowDagGenerator}, that generate hive sync workflowDag for MOR table.
Review comment:
ditto.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pratyakshsharma commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
pratyakshsharma commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r466832837
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/utils/TestUtils.java
##########
@@ -28,6 +28,9 @@
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.sql.SparkSession;
+/**
+ * A utility class for test purpose.
Review comment:
test -> testing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pratyakshsharma commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
pratyakshsharma commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r466832530
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/dag/ComplexDagGenerator.java
##########
@@ -33,6 +33,9 @@
import org.apache.hudi.integ.testsuite.dag.nodes.ValidateNode;
import org.apache.spark.api.java.JavaRDD;
+/**
+ * An implementation of {@link WorkflowDagGenerator}, that generate complex workflowDag.
Review comment:
generate -> generates.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pratyakshsharma commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
pratyakshsharma commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r466833643
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/TestDFSHoodieTestSuiteWriterAdapter.java
##########
@@ -52,6 +52,9 @@
import org.junit.jupiter.api.Test;
import org.mockito.Mockito;
+/**
+ * An adapter of {@link HoodieTestSuiteWriter} help to test write DFS file.
Review comment:
{@link HoodieTestSuiteWriter}. Helps to test writing a DFS file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yanghua commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
yanghua commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r475291250
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/ITTestBase.java
##########
@@ -50,6 +50,9 @@
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
+/**
+ * Base test class for IT Test. helps to run cmd and generate data.
Review comment:
`Base test class for IT Test helps to run command and generate data.`?
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/TestDFSHoodieTestSuiteWriterAdapter.java
##########
@@ -52,6 +52,9 @@
import org.junit.jupiter.api.Test;
import org.mockito.Mockito;
+/**
+ * {@link HoodieTestSuiteWriter}. Helps to test writing a DFS file.
Review comment:
`Helps`? This usage may not be correct?
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/utils/TestUtils.java
##########
@@ -45,6 +48,15 @@
return dataGenerator.generateGenericRecords(numRecords);
}
+ /**
+ * Method help to create avro files and save it to file.
Review comment:
`Methods`?
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/configuration/TestWorkflowBuilder.java
##########
@@ -30,41 +30,58 @@
import org.apache.hudi.integ.testsuite.dag.WorkflowDag;
import org.junit.jupiter.api.Test;
+/**
+ * Unit test for the build process of {@link DagNode} and {@link WorkflowDag}.
+ */
public class TestWorkflowBuilder {
@Test
public void testWorkloadOperationSequenceBuilder() {
Review comment:
please remove all the comments of this method
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/job/TestHoodieTestSuiteJob.java
##########
@@ -49,6 +49,9 @@
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
+/**
+ * Unit tests against {@link HoodieTestSuiteJob}.
Review comment:
`Unit test`?
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/converter/TestUpdateConverter.java
##########
@@ -49,11 +53,16 @@ public void teardown() {
jsc.stop();
}
+ /**
+ * Test {@link UpdateConverter} by generates random updates from existing records.
+ */
@Test
public void testGenerateUpdateRecordsFromInputRecords() throws Exception {
+ // 1. prepare input record
Review comment:
`record` -> `records`
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/utils/TestUtils.java
##########
@@ -45,6 +48,15 @@
return dataGenerator.generateGenericRecords(numRecords);
}
+ /**
+ * Method help to create avro files and save it to file.
+ *
+ * @param jsc {@link JavaSparkContext}.
Review comment:
We should not only use `{@link }` in the comment, add more description.
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/dag/ComplexDagGenerator.java
##########
@@ -46,6 +51,7 @@ public WorkflowDag build() {
.withNumInsertPartitions(1)
.withRecordSize(10000).build());
+ // function to build ValidateNode with
Review comment:
with what?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pratyakshsharma commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
pratyakshsharma commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r466833116
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/utils/TestUtils.java
##########
@@ -45,6 +48,15 @@
return dataGenerator.generateGenericRecords(numRecords);
}
+ /**
+ * Method help to create avro files and save it to file.
+ *
+ * @param jsc {@link JavaSparkContext}.
+ * @param sparkSession {@link SparkSession}.
+ * @param basePath The basePath where files written to.
Review comment:
basePath where files are written.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yanghua merged pull request #1901: [HUDI-532] Add java doc for the test classes of hudi test suite
Posted by GitBox <gi...@apache.org>.
yanghua merged pull request #1901:
URL: https://github.com/apache/hudi/pull/1901
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Mathieu1124 commented on a change in pull request #1901: [HUDI-532]Add java doc for hudi test suite test classes
Posted by GitBox <gi...@apache.org>.
Mathieu1124 commented on a change in pull request #1901:
URL: https://github.com/apache/hudi/pull/1901#discussion_r467025816
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/dag/HiveSyncDagGenerator.java
##########
@@ -31,6 +31,9 @@
import org.apache.hudi.integ.testsuite.dag.nodes.HiveSyncNode;
import org.apache.hudi.integ.testsuite.dag.nodes.InsertNode;
+/**
+ * An implementation of {@link WorkflowDagGenerator}, that generate hive sync workflowDag.
Review comment:
done
##########
File path: hudi-integ-test/src/test/java/org/apache/hudi/integ/testsuite/dag/HiveSyncDagGeneratorMOR.java
##########
@@ -31,6 +31,9 @@
import org.apache.hudi.integ.testsuite.dag.nodes.HiveSyncNode;
import org.apache.hudi.integ.testsuite.dag.nodes.InsertNode;
+/**
+ * An implementation of {@link WorkflowDagGenerator}, that generate hive sync workflowDag for MOR table.
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org