You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "chinmay-bhat (via GitHub)" <gi...@apache.org> on 2023/12/19 13:54:21 UTC

[PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

chinmay-bhat opened a new pull request, #9342:
URL: https://github.com/apache/iceberg/pull/9342

   This PR contains the Junit5 migration for Spark 3.5v for the `spark/source` directory. 
   Related to PR : #9341 
   Only non-parameter files have been migrated.
   
   Issue: #9086 
   
   TODO:
   
   - [x] self-review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1864517617

   while i have your attention, can you review #9341 too? :)
   I opened it around the same time as this PR, and it has most of the changes you suggested here.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1864514872

   changes made!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432169104


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestAvroScan.java:
##########
@@ -41,25 +43,24 @@
 import org.apache.spark.sql.Dataset;
 import org.apache.spark.sql.Row;
 import org.apache.spark.sql.SparkSession;
-import org.junit.AfterClass;
-import org.junit.Assert;
-import org.junit.BeforeClass;
-import org.junit.Rule;
-import org.junit.rules.TemporaryFolder;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.io.TempDir;
 
 public class TestAvroScan extends AvroDataTest {
   private static final Configuration CONF = new Configuration();
 
-  @Rule public TemporaryFolder temp = new TemporaryFolder();
+  @TempDir
+  public Path temp;

Review Comment:
   updated



##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestBaseReader.java:
##########
@@ -47,14 +49,13 @@
 import org.apache.iceberg.relocated.com.google.common.collect.Maps;
 import org.apache.iceberg.spark.data.RandomData;
 import org.apache.iceberg.types.Types;
-import org.junit.Assert;
-import org.junit.Rule;
-import org.junit.Test;
-import org.junit.rules.TemporaryFolder;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
 
 public class TestBaseReader {
 
-  @Rule public TemporaryFolder temp = new TemporaryFolder();
+  @TempDir
+  public Path temp;

Review Comment:
   updated



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1863976054

   Thanks for the review. Yes, I assumed the convention was the other way. Will update all the files.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1864274603

   I don't think that would work. See [line 106](https://github.com/apache/iceberg/blob/a83bfe72ccc18ec6f22621630b6dbabaf4648064/data/src/test/java/org/apache/iceberg/RecordWrapperTest.java#L106). 
   We use `Assert:assertEquals` through the `AssertMethod` interface. If we switch the imports, we would have to change the interface & its downstream functions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark 3:5 Migrate tests to JUnit5 in source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1867476496

   done!
   
   Also, I added the parameterized annotations to `CatalogTestBase`, since `SparkCatalogTestBase` was parameterized but `CatalogTestBase` was surprisingly not.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432347207


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReadProjection.java:
##########
@@ -110,7 +110,7 @@ public static void stopSpark() {
   @Override
   protected Record writeAndRead(String desc, Schema writeSchema, Schema readSchema, Record record)
       throws IOException {
-    File parent = temp.newFolder(desc);
+    File parent = temp.resolve(desc).toFile();

Review Comment:
   I wanted to create a PR with only non-paramterized changes. This file was using the tempDir from its base class, which was now a JUnit5 implementation. So I only made this small change. The changes with parameterizing annotations would be in another PR. 
   Or is it ok to do it all in 1 PR? That might make the PR hard to review



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432340649


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkStagedScan.java:
##########
@@ -111,7 +111,9 @@ public void testTaskSetPlanning() throws NoSuchTableException, IOException {
               .option(SparkReadOptions.SCAN_TASK_SET_ID, setID)
               .option(SparkReadOptions.SPLIT_SIZE, tasks.get(0).file().fileSizeInBytes())
               .load(tableName);
-      Assert.assertEquals("Num partitions should match", 2, scanDF.javaRDD().getNumPartitions());
+      assertThat(2)

Review Comment:
   should be the other way around



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432417641


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestBaseReader.java:
##########
@@ -47,14 +49,13 @@
 import org.apache.iceberg.relocated.com.google.common.collect.Maps;
 import org.apache.iceberg.spark.data.RandomData;
 import org.apache.iceberg.types.Types;
-import org.junit.Assert;
-import org.junit.Rule;
-import org.junit.Test;
-import org.junit.rules.TemporaryFolder;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
 
 public class TestBaseReader {
 
-  @Rule public TemporaryFolder temp = new TemporaryFolder();
+  @TempDir
+  public Path temp;

Review Comment:
   if it's used in a subclass then `protected` makes sense, otherwise it should be `private`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432971413


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -64,11 +66,14 @@ public void testFullProjection() throws Exception {
 
     Record projected = writeAndRead("full_projection", schema, schema, record);
 
-    Assert.assertEquals(
-        "Should contain the correct id value", 34L, (long) projected.getField("id"));
+    assertThat((long) projected.getField("id"))
+        .as("Should contain the correct id value")
+        .isEqualTo(34L);
 
     int cmp =
         Comparators.charSequences().compare("test", (CharSequence) projected.getField("data"));
+
+    // TODO: update this Assert with AssertJ

Review Comment:
   sorry for the mis-communication, that's my bad. I didn't mean to mention `org.junit.Assert`. In order for the test to be a JUnit4 test it just needs `org.junit.Rule` / `org.junit.Test` / `org.junit.rules.TemporaryFolder`. The assertions being used can be AssertJ (which you already converted)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark 3:5 Migrate tests to JUnit5 in source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1434921722


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestWriteMetricsConfig.java:
##########
@@ -271,28 +271,30 @@ public void testCustomMetricCollectionForNestedParquet() throws IOException {
       DataFile file = task.file();
 
       Map<Integer, Long> nullValueCounts = file.nullValueCounts();
-      Assert.assertEquals(3, nullValueCounts.size());
-      Assert.assertTrue(nullValueCounts.containsKey(longCol.fieldId()));
-      Assert.assertTrue(nullValueCounts.containsKey(recordId.fieldId()));
-      Assert.assertTrue(nullValueCounts.containsKey(recordData.fieldId()));
+      assertThat(nullValueCounts).hasSize(3);
+      assertThat(nullValueCounts).containsKey(longCol.fieldId());
+      assertThat(nullValueCounts).containsKey(recordId.fieldId());
+      assertThat(nullValueCounts).containsKey(recordData.fieldId());
 
       Map<Integer, Long> valueCounts = file.valueCounts();
-      Assert.assertEquals(3, valueCounts.size());
-      Assert.assertTrue(valueCounts.containsKey(longCol.fieldId()));
-      Assert.assertTrue(valueCounts.containsKey(recordId.fieldId()));
-      Assert.assertTrue(valueCounts.containsKey(recordData.fieldId()));
+      assertThat(valueCounts).hasSize(3);

Review Comment:
   these call all be combined to a more fluent call: `assertThat(..).hasSize(3).containsKey(..).containsKey(..).containsKey(...)`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark 3:5 Migrate tests to JUnit5 in source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1867367623

   hmm it seems like even after we merged #9341 into main, Github doesn't recognise and ignore the changes in this PR that now exist in main. Do you recommend that I rebase onto main and force push?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432340253


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReadProjection.java:
##########
@@ -110,7 +110,7 @@ public static void stopSpark() {
   @Override
   protected Record writeAndRead(String desc, Schema writeSchema, Schema readSchema, Record record)
       throws IOException {
-    File parent = temp.newFolder(desc);
+    File parent = temp.resolve(desc).toFile();

Review Comment:
   given that you're adjusting this file, shouldn't this also be updated to use JUnit5? There is `@BeforeClass` / `@AfterClass` usage. For parameterizing the test you can use the annotations introduced by https://github.com/apache/iceberg/pull/9161



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432738655


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java:
##########
@@ -1964,19 +1989,21 @@ public void testRemoveOrphanFilesActionSupport() throws InterruptedException {
             .location(table.location() + "/metadata")
             .olderThan(System.currentTimeMillis())
             .execute();
-    Assert.assertTrue(
-        "Should not delete any metadata files", Iterables.isEmpty(result1.orphanFileLocations()));
+
+    assertThat(Iterables.isEmpty(result1.orphanFileLocations()))

Review Comment:
   ```suggestion
       assertThat(result1.orphanFileLocations()).isEmpty()
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432731435


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSpark.java:
##########
@@ -59,83 +60,78 @@ public static void stopSpark() {
   public void testRegisterIntegerBucketUDF() {
     IcebergSpark.registerBucketUDF(spark, "iceberg_bucket_int_16", DataTypes.IntegerType, 16);
     List<Row> results = spark.sql("SELECT iceberg_bucket_int_16(1)").collectAsList();
-    Assert.assertEquals(1, results.size());
-    Assert.assertEquals(
-        (int) Transforms.bucket(16).bind(Types.IntegerType.get()).apply(1),
-        results.get(0).getInt(0));
+
+    assertThat(results).hasSize(1);
+    assertThat(results.get(0).getInt(0))
+        .isEqualTo((int) Transforms.bucket(16).bind(Types.IntegerType.get()).apply(1));

Review Comment:
   I think all of these casts to `(int)` can now be removed. This was rather used because `assertEquals()` was complaining about it



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432954411


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -64,11 +66,14 @@ public void testFullProjection() throws Exception {
 
     Record projected = writeAndRead("full_projection", schema, schema, record);
 
-    Assert.assertEquals(
-        "Should contain the correct id value", 34L, (long) projected.getField("id"));
+    assertThat((long) projected.getField("id"))
+        .as("Should contain the correct id value")
+        .isEqualTo(34L);
 
     int cmp =
         Comparators.charSequences().compare("test", (CharSequence) projected.getField("data"));
+
+    // TODO: update this Assert with AssertJ

Review Comment:
   > because you're changing this file it seems you'd also have to update `TestSparkReadProjection`, otherwise CI won't pass.
   > 
   > Alternatively you could reintroduce
   > 
   > ```
   > import org.junit.Assert;
   > import org.junit.Rule;
   > import org.junit.Test;
   > import org.junit.rules.TemporaryFolder;
   > ```
   > 
   > but leave the AssertJ conversions in this file
   
   Based on this comment, I thought we need to keep all those imports for clearing CI. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432341279


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestStreamingOffset.java:
##########
@@ -35,10 +36,10 @@ public void testJsonConversion() {
           new StreamingOffset(System.currentTimeMillis(), 3L, false),
           new StreamingOffset(System.currentTimeMillis(), 4L, true)
         };
-    Assert.assertArrayEquals(
-        "StreamingOffsets should match",
-        expected,
-        Arrays.stream(expected).map(elem -> StreamingOffset.fromJson(elem.json())).toArray());
+    assertThat(expected)

Review Comment:
   same here



##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestStreamingOffset.java:
##########
@@ -51,6 +52,6 @@ public void testToJson() throws Exception {
     actual.put("scan_all_files", false);
     String expectedJson = expected.json();
     String actualJson = JsonUtil.mapper().writeValueAsString(actual);
-    Assert.assertEquals("Json should match", expectedJson, actualJson);
+    assertThat(expectedJson).isEqualTo(actualJson);

Review Comment:
   same as above



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432413077


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestBaseReader.java:
##########
@@ -47,14 +49,13 @@
 import org.apache.iceberg.relocated.com.google.common.collect.Maps;
 import org.apache.iceberg.spark.data.RandomData;
 import org.apache.iceberg.types.Types;
-import org.junit.Assert;
-import org.junit.Rule;
-import org.junit.Test;
-import org.junit.rules.TemporaryFolder;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
 
 public class TestBaseReader {
 
-  @Rule public TemporaryFolder temp = new TemporaryFolder();
+  @TempDir
+  public Path temp;

Review Comment:
   I'm now getting build errors on private access.
   Example:
   ```
   error: temp has private access in TestIcebergSourceTablesBase
       this.tableDir = temp.toFile();
   ```
   
   In [this PR](https://github.com/apache/iceberg/pull/9129#discussion_r1403012159), you mentioned using `protected` instead of `public`. Can I do that instead?
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432834544


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -49,7 +50,7 @@ public abstract class TestReadProjection {
   protected abstract Record writeAndRead(
       String desc, Schema writeSchema, Schema readSchema, Record record) throws IOException;
 
-  @Rule public TemporaryFolder temp = new TemporaryFolder();
+  @TempDir protected Path temp;

Review Comment:
   because you're changing this file it seems you'd also have to update `TestSparkReadProjection`, otherwise CI won't pass.
   
   Alternatively you could reintroduce
   ```
   import org.junit.Assert;
   import org.junit.Rule;
   import org.junit.Test;
   import org.junit.rules.TemporaryFolder;
   ```
   
   but leave the AssertJ conversions in this file



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1431752607


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestBaseReader.java:
##########
@@ -47,14 +49,13 @@
 import org.apache.iceberg.relocated.com.google.common.collect.Maps;
 import org.apache.iceberg.spark.data.RandomData;
 import org.apache.iceberg.types.Types;
-import org.junit.Assert;
-import org.junit.Rule;
-import org.junit.Test;
-import org.junit.rules.TemporaryFolder;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
 
 public class TestBaseReader {
 
-  @Rule public TemporaryFolder temp = new TemporaryFolder();
+  @TempDir
+  public Path temp;

Review Comment:
   with JUnit5 these can all be `private` now. Please also adjust accordingly in other files



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432357336


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkReadProjection.java:
##########
@@ -110,7 +110,7 @@ public static void stopSpark() {
   @Override
   protected Record writeAndRead(String desc, Schema writeSchema, Schema readSchema, Record record)
       throws IOException {
-    File parent = temp.newFolder(desc);
+    File parent = temp.resolve(desc).toFile();

Review Comment:
   I just want to make sure that we don't partially update files and then forgot about updating them fully. I'm ok doing it in an immediate follow-up PR



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432326205


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestDataSourceOptions.java:
##########
@@ -219,12 +219,14 @@ public void testSplitOptionsOverridesTableProperties() throws IOException {
             .option(SparkReadOptions.SPLIT_SIZE, String.valueOf(splitSize))
             .load(tableLocation);
 
-    Assert.assertEquals("Spark partitions should match", 2, resultDf.javaRDD().getNumPartitions());
+    assertThat(2)

Review Comment:
   this should be the other way around: `assertThat(resultDf.javaRDD().getNumPartitions())...isEqualTo(2)`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432334842


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSpark.java:
##########
@@ -59,83 +60,78 @@ public static void stopSpark() {
   public void testRegisterIntegerBucketUDF() {
     IcebergSpark.registerBucketUDF(spark, "iceberg_bucket_int_16", DataTypes.IntegerType, 16);
     List<Row> results = spark.sql("SELECT iceberg_bucket_int_16(1)").collectAsList();
-    Assert.assertEquals(1, results.size());
-    Assert.assertEquals(
-        (int) Transforms.bucket(16).bind(Types.IntegerType.get()).apply(1),
-        results.get(0).getInt(0));
+
+    assertThat(results).hasSize(1);
+    assertThat((int) Transforms.bucket(16).bind(Types.IntegerType.get()).apply(1))

Review Comment:
   it looks like in this file the expected/actual is the other way around



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432336655


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -64,12 +65,13 @@ public void testFullProjection() throws Exception {
 
     Record projected = writeAndRead("full_projection", schema, schema, record);
 
-    Assert.assertEquals(
-        "Should contain the correct id value", 34L, (long) projected.getField("id"));
+    assertThat(34L)

Review Comment:
   actual/expected is the wrong way in this file as well



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432337147


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSparkAggregates.java:
##########
@@ -50,27 +51,34 @@ public void testAggregates() {
           Max max = new Max(namedReference);
           Expression expectedMax = Expressions.max(unquoted);
           Expression actualMax = SparkAggregates.convert(max);
-          Assert.assertEquals("Max must match", expectedMax.toString(), actualMax.toString());
+          assertThat(expectedMax.toString())

Review Comment:
   actual/expected is the wrong way in this file as well



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432728000


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestDataSourceOptions.java:
##########
@@ -407,13 +409,13 @@ public void testExtraSnapshotMetadata() throws IOException {
 
     Table table = tables.load(tableLocation);
 
-    Assert.assertTrue(table.currentSnapshot().summary().get("extra-key").equals("someValue"));
-    Assert.assertTrue(table.currentSnapshot().summary().get("another-key").equals("anotherValue"));
+    assertThat(table.currentSnapshot().summary().get("extra-key")).isEqualTo("someValue");

Review Comment:
   in such cases it's usually better to do something like
   ```
   assertThat(table.currentSnapshot().summary().containsEntry("extra-key", "someValue).containsEntry("another-key", "anotherValue")
   ```
   because if the assertion ever fails, the full map content will be printed, which makes debugging much easier



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1433000616


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -64,11 +66,14 @@ public void testFullProjection() throws Exception {
 
     Record projected = writeAndRead("full_projection", schema, schema, record);
 
-    Assert.assertEquals(
-        "Should contain the correct id value", 34L, (long) projected.getField("id"));
+    assertThat((long) projected.getField("id"))
+        .as("Should contain the correct id value")
+        .isEqualTo(34L);
 
     int cmp =
         Comparators.charSequences().compare("test", (CharSequence) projected.getField("data"));
+
+    // TODO: update this Assert with AssertJ

Review Comment:
   ok, I'll add these tomorrow!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1431751123


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestAvroScan.java:
##########
@@ -41,25 +43,24 @@
 import org.apache.spark.sql.Dataset;
 import org.apache.spark.sql.Row;
 import org.apache.spark.sql.SparkSession;
-import org.junit.AfterClass;
-import org.junit.Assert;
-import org.junit.BeforeClass;
-import org.junit.Rule;
-import org.junit.rules.TemporaryFolder;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.io.TempDir;
 
 public class TestAvroScan extends AvroDataTest {
   private static final Configuration CONF = new Configuration();
 
-  @Rule public TemporaryFolder temp = new TemporaryFolder();
+  @TempDir
+  public Path temp;

Review Comment:
   ```suggestion
     private Path temp;
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432336282


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestPathIdentifier.java:
##########
@@ -75,12 +74,11 @@ public void testPathIdentifier() throws TableAlreadyExistsException, NoSuchTable
             sparkCatalog.createTable(
                 identifier, SparkSchemaUtil.convert(SCHEMA), new Transform[0], ImmutableMap.of());
 
-    Assert.assertEquals(table.table().location(), tableLocation.getAbsolutePath());
-    Assertions.assertThat(table.table()).isInstanceOf(BaseTable.class);
-    Assertions.assertThat(((BaseTable) table.table()).operations())
-        .isInstanceOf(HadoopTableOperations.class);
+    assertThat(table.table().location()).isEqualTo(tableLocation.getAbsolutePath());

Review Comment:
   actual/expected is the wrong way here and in `assertThat(sparkCatalog.loadTable(identifier)).isEqualTo(table);`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432338113


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestDataSourceOptions.java:
##########
@@ -297,7 +299,7 @@ public void testIncrementalScanOptions() throws IOException {
             .orderBy("id")
             .as(Encoders.bean(SimpleRecord.class))
             .collectAsList();
-    Assert.assertEquals("Records should match", expectedRecords.subList(1, 4), result);
+    assertThat(expectedRecords.subList(1, 4)).as("Records should match").isEqualTo(result);

Review Comment:
   my bad, I thought the convention was (expected, actual). There's a lot of places that I've followed it. Will change it to (actual, expected)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432856893


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -49,7 +50,7 @@ public abstract class TestReadProjection {
   protected abstract Record writeAndRead(
       String desc, Schema writeSchema, Schema readSchema, Record record) throws IOException;
 
-  @Rule public TemporaryFolder temp = new TemporaryFolder();
+  @TempDir protected Path temp;

Review Comment:
   I chose to reintroduce the imports.
   For that, I reversed the TempDir change, and changed 1 AssertJ back to Assert (added a TODO so I can change it in the future).
   Otherwise, the imports were getting auto-removed in the IDE.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1864327646

   > I don't think that would work. See [line 106](https://github.com/apache/iceberg/blob/a83bfe72ccc18ec6f22621630b6dbabaf4648064/data/src/test/java/org/apache/iceberg/RecordWrapperTest.java#L106). We use `Assert:assertEquals` through the `AssertMethod` interface. If we switch the imports, we would have to change the interface & its downstream functions.
   
   The imports I meant is the imports for `@Test`, not for the `Assert` class


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark 3:5 Migrate tests to JUnit5 in source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra merged PR #9342:
URL: https://github.com/apache/iceberg/pull/9342


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark 3:5 Migrate tests to JUnit5 in source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1867460904

   yes you need a rebase + force push


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1866286222

   After testing, I realized that this PR depends a lot on the changes I introduced in #9341. So we need to prioritise merging #9341 first.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1864140400

   Changes made:
   1. fix `assertThat()` to consistently use (actual, expected) convention
   2. update tempDir access to protected only when `temp` is used in subclasses.
   
   Notes:
   1. partially updated `TestSparkReadProjection.java` to be fully updated with parameterized annotations in next PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1864173459

   Also, file [TestInternalRowWrapper](https://github.com/apache/iceberg/blob/main/spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestInternalRowWrapper.java)  extends [RecordWrapperTest](https://github.com/apache/iceberg/blob/main/data/src/test/java/org/apache/iceberg/RecordWrapperTest.java). 
   
   RecordWrapperTest is also used by `iceberg-flink` and uses JUnit4. 
   Do we need to make a copy of RecordWrapperTest in JUnit5 to migrate TestInternalRowWrapper? The copy and original RecordWrapperTest will be available at the same time. Maybe in another PR?
   
   I haven't added my changes to TestInternalRowWrapper to this PR for that reason. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1864184926

   We should be able to just switch the imports in `RecordWrapperTest` to JUnit5. That should make the subclasses run with JUnit in Spark + Flink. Converting the Flink subclasses to AssertJ doesn't have to be done within this PR though


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1864366627

   gotcha, I've made all the changes. Do let me know if anymore changes are to be made.
   
   Once this PR is merged, I'll continue work on the remaining files (having parameterize and non-parameterize)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432919611


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -64,11 +66,14 @@ public void testFullProjection() throws Exception {
 
     Record projected = writeAndRead("full_projection", schema, schema, record);
 
-    Assert.assertEquals(
-        "Should contain the correct id value", 34L, (long) projected.getField("id"));
+    assertThat((long) projected.getField("id"))
+        .as("Should contain the correct id value")
+        .isEqualTo(34L);
 
     int cmp =
         Comparators.charSequences().compare("test", (CharSequence) projected.getField("data"));
+
+    // TODO: update this Assert with AssertJ

Review Comment:
   I don't understand why this can't use AssertJ assertion



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432954411


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -64,11 +66,14 @@ public void testFullProjection() throws Exception {
 
     Record projected = writeAndRead("full_projection", schema, schema, record);
 
-    Assert.assertEquals(
-        "Should contain the correct id value", 34L, (long) projected.getField("id"));
+    assertThat((long) projected.getField("id"))
+        .as("Should contain the correct id value")
+        .isEqualTo(34L);
 
     int cmp =
         Comparators.charSequences().compare("test", (CharSequence) projected.getField("data"));
+
+    // TODO: update this Assert with AssertJ

Review Comment:
   > because you're changing this file it seems you'd also have to update `TestSparkReadProjection`, otherwise CI won't pass.
   > 
   > Alternatively you could reintroduce
   > 
   > ```
   > import org.junit.Assert;
   > import org.junit.Rule;
   > import org.junit.Test;
   > import org.junit.rules.TemporaryFolder;
   > ```
   > 
   > but leave the AssertJ conversions in this file
   
   Based on this comment, I thought we need to keep all those imports for clearing CI, including the Assert import. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark 3:5 Migrate tests to JUnit5 in source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1867295814

   Since this PR depends on #9341, I added a patch of files changed in #9341. Commit [4f88900](https://github.com/apache/iceberg/pull/9342/commits/4f8890089b77a17987811e14e8659a43fac4651a).
   
   As long as we merge #9341 first, this PR should be good to go :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark 3:5 Migrate tests to JUnit5 in source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1434929879


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestWriteMetricsConfig.java:
##########
@@ -271,28 +271,30 @@ public void testCustomMetricCollectionForNestedParquet() throws IOException {
       DataFile file = task.file();
 
       Map<Integer, Long> nullValueCounts = file.nullValueCounts();
-      Assert.assertEquals(3, nullValueCounts.size());
-      Assert.assertTrue(nullValueCounts.containsKey(longCol.fieldId()));
-      Assert.assertTrue(nullValueCounts.containsKey(recordId.fieldId()));
-      Assert.assertTrue(nullValueCounts.containsKey(recordData.fieldId()));
+      assertThat(nullValueCounts).hasSize(3);
+      assertThat(nullValueCounts).containsKey(longCol.fieldId());
+      assertThat(nullValueCounts).containsKey(recordId.fieldId());
+      assertThat(nullValueCounts).containsKey(recordData.fieldId());
 
       Map<Integer, Long> valueCounts = file.valueCounts();
-      Assert.assertEquals(3, valueCounts.size());
-      Assert.assertTrue(valueCounts.containsKey(longCol.fieldId()));
-      Assert.assertTrue(valueCounts.containsKey(recordId.fieldId()));
-      Assert.assertTrue(valueCounts.containsKey(recordData.fieldId()));
+      assertThat(valueCounts).hasSize(3);

Review Comment:
   updated



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#issuecomment-1863235505

   can you please run `./gradlew spotlessApply` to fix any formatting?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432327788


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestDataSourceOptions.java:
##########
@@ -297,7 +299,7 @@ public void testIncrementalScanOptions() throws IOException {
             .orderBy("id")
             .as(Encoders.bean(SimpleRecord.class))
             .collectAsList();
-    Assert.assertEquals("Records should match", expectedRecords.subList(1, 4), result);
+    assertThat(expectedRecords.subList(1, 4)).as("Records should match").isEqualTo(result);

Review Comment:
   this should be the other way around: `expectedRecords.subList(1, 4)` should go into `isEqualTo()` as that's what's being expected.
   
   It looks like the assertions are the other way around in a bunch of places. Could you please fix those?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432856893


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -49,7 +50,7 @@ public abstract class TestReadProjection {
   protected abstract Record writeAndRead(
       String desc, Schema writeSchema, Schema readSchema, Record record) throws IOException;
 
-  @Rule public TemporaryFolder temp = new TemporaryFolder();
+  @TempDir protected Path temp;

Review Comment:
   I chose to reintroduce the imports.
   For that, I reversed the TempDir change, and changed 1 AssertJ back to Assert (added a TODO so I can change it in the future).
   Otherwise, the imports were getting auto-removed from the IDE.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432947294


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -64,11 +66,14 @@ public void testFullProjection() throws Exception {
 
     Record projected = writeAndRead("full_projection", schema, schema, record);
 
-    Assert.assertEquals(
-        "Should contain the correct id value", 34L, (long) projected.getField("id"));
+    assertThat((long) projected.getField("id"))
+        .as("Should contain the correct id value")
+        .isEqualTo(34L);
 
     int cmp =
         Comparators.charSequences().compare("test", (CharSequence) projected.getField("data"));
+
+    // TODO: update this Assert with AssertJ

Review Comment:
   why do you need to keep `import org.junit.Assert`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark Migration to JUnit5 AssertJ: non-parameterized, spark/source directory [iceberg]

Posted by "chinmay-bhat (via GitHub)" <gi...@apache.org>.
chinmay-bhat commented on code in PR #9342:
URL: https://github.com/apache/iceberg/pull/9342#discussion_r1432931800


##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestReadProjection.java:
##########
@@ -64,11 +66,14 @@ public void testFullProjection() throws Exception {
 
     Record projected = writeAndRead("full_projection", schema, schema, record);
 
-    Assert.assertEquals(
-        "Should contain the correct id value", 34L, (long) projected.getField("id"));
+    assertThat((long) projected.getField("id"))
+        .as("Should contain the correct id value")
+        .isEqualTo(34L);
 
     int cmp =
         Comparators.charSequences().compare("test", (CharSequence) projected.getField("data"));
+
+    // TODO: update this Assert with AssertJ

Review Comment:
   Because, everytime I try to add an import that is unused in the code, the IDE automatically removes the import. So if I need to keep the `import org.junit.Assert;` import statement, it needs to be used somewhere. 
   
   So I just chose the first simple AssertJ statement and reverted it to use Assert.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org