You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/11/12 06:45:39 UTC
[GitHub] [incubator-pinot] sjeetSingh opened a new pull request #6261: Add test case for FileUtils class
sjeetSingh opened a new pull request #6261:
URL: https://github.com/apache/incubator-pinot/pull/6261
## Description
Added test cases for `/common/utils/FileUtilsTest` class
## Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)
* [ ] Yes (Please label as **<code>backward-incompat</code>**, and complete the section below on Release Notes)
Does this PR fix a zero-downtime upgrade introduced earlier?
* [ ] Yes (Please label this as **<code>backward-incompat</code>**, and complete the section below on Release Notes)
Does this PR otherwise need attention when creating release notes? Things to consider:
- New configuration options
- Deprecation of configurations
- Signature changes to public methods/interfaces
- New plugins added or old plugins removed
* [ ] Yes (Please label this PR as **<code>release-notes</code>** and complete the section on Release Notes)
## Release Notes
If you have tagged this as either backward-incompat or release-notes,
you MUST add text here that you would like to see appear in release notes of the
next release.
If you have a series of commits adding or enabling a feature, then
add this section only in final commit that marks the feature completed.
Refer to earlier release notes to see examples of text
## Documentation
If you have introduced a new feature or configuration, please add it to the documentation as well.
See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #6261: Add test case for FileUtils class
Posted by GitBox <gi...@apache.org>.
mayankshriv commented on a change in pull request #6261:
URL: https://github.com/apache/incubator-pinot/pull/6261#discussion_r522278254
##########
File path: pinot-common/src/test/java/org/apache/pinot/common/utils/FileUtilsTest.java
##########
@@ -0,0 +1,125 @@
+package org.apache.pinot.common.utils;
+
+import com.google.common.collect.Iterables;
+import junit.framework.TestCase;
+import org.junit.jupiter.api.Test;
+
+import java.io.Closeable;
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.channels.FileChannel;
+import java.util.HashMap;
+import java.util.Map;
+
+public class FileUtilsTest extends TestCase {
Review comment:
Add javadoc to point to the class being tested, and perhaps add details on what is being tested.
##########
File path: pinot-common/src/test/java/org/apache/pinot/common/utils/FileUtilsTest.java
##########
@@ -0,0 +1,125 @@
+package org.apache.pinot.common.utils;
+
+import com.google.common.collect.Iterables;
+import junit.framework.TestCase;
+import org.junit.jupiter.api.Test;
+
+import java.io.Closeable;
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.channels.FileChannel;
+import java.util.HashMap;
+import java.util.Map;
+
+public class FileUtilsTest extends TestCase {
+ @Test
+ public void testMoveFileWithOverwrite_destFileNotPresent() {
+ // Create source directory
+ File sourceDir = new File("/tmp/sourceDir");
Review comment:
Use `System.getProperty("java.io.tmpdir") + File.separator`, or `FileUtils.getTempDirectory()` (from apache.common.io), instead.
##########
File path: pinot-common/src/test/java/org/apache/pinot/common/utils/FileUtilsTest.java
##########
@@ -0,0 +1,125 @@
+package org.apache.pinot.common.utils;
+
+import com.google.common.collect.Iterables;
+import junit.framework.TestCase;
+import org.junit.jupiter.api.Test;
+
+import java.io.Closeable;
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.channels.FileChannel;
+import java.util.HashMap;
+import java.util.Map;
+
+public class FileUtilsTest extends TestCase {
+ @Test
+ public void testMoveFileWithOverwrite_destFileNotPresent() {
+ // Create source directory
+ File sourceDir = new File("/tmp/sourceDir");
+ sourceDir.mkdir();
+ // Create destination directory
+ File destDir = new File("/tmp/destDir");
Review comment:
Same here, and all other places.
##########
File path: pinot-common/src/test/java/org/apache/pinot/common/utils/FileUtilsTest.java
##########
@@ -0,0 +1,125 @@
+package org.apache.pinot.common.utils;
+
+import com.google.common.collect.Iterables;
+import junit.framework.TestCase;
+import org.junit.jupiter.api.Test;
+
+import java.io.Closeable;
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.channels.FileChannel;
+import java.util.HashMap;
+import java.util.Map;
+
+public class FileUtilsTest extends TestCase {
+ @Test
+ public void testMoveFileWithOverwrite_destFileNotPresent() {
+ // Create source directory
+ File sourceDir = new File("/tmp/sourceDir");
+ sourceDir.mkdir();
+ // Create destination directory
+ File destDir = new File("/tmp/destDir");
+ destDir.mkdir();
+
+ // Define source and dest file
+ File sourceFile = new File(sourceDir, "sourceFile.txt");
+ try {
+ // Create empty source file
+ sourceFile.createNewFile();
+ FileUtils.moveFileWithOverwrite(sourceFile, new File(destDir, "destFile.txt"));
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ String[] fileList = destDir.list();
+ for(String file : fileList) {
Review comment:
Is a linear search warranted? Can we not just try to open the file in the destDir, and assert we can open it? It would be good to also ensure something like file size is the same (if not content).
##########
File path: pinot-common/src/test/java/org/apache/pinot/common/utils/FileUtilsTest.java
##########
@@ -0,0 +1,125 @@
+package org.apache.pinot.common.utils;
+
+import com.google.common.collect.Iterables;
+import junit.framework.TestCase;
+import org.junit.jupiter.api.Test;
+
+import java.io.Closeable;
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.channels.FileChannel;
+import java.util.HashMap;
+import java.util.Map;
+
+public class FileUtilsTest extends TestCase {
+ @Test
+ public void testMoveFileWithOverwrite_destFileNotPresent() {
+ // Create source directory
+ File sourceDir = new File("/tmp/sourceDir");
+ sourceDir.mkdir();
+ // Create destination directory
+ File destDir = new File("/tmp/destDir");
+ destDir.mkdir();
+
+ // Define source and dest file
+ File sourceFile = new File(sourceDir, "sourceFile.txt");
+ try {
+ // Create empty source file
+ sourceFile.createNewFile();
+ FileUtils.moveFileWithOverwrite(sourceFile, new File(destDir, "destFile.txt"));
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ String[] fileList = destDir.list();
+ for(String file : fileList) {
+ if (file.equals("destFile")) {
+ assertTrue(true);
+ break;
+ }
+ }
+ }
+
+ @Test
+ public void testMoveFileWithOverwrite_destFilePresent() {
+ // Create source directory
+ File sourceDir = new File("/tmp/sourceDir");
+ sourceDir.mkdir();
+ // Create destination directory
+ File destDir = new File("/tmp/destDir");
+ destDir.mkdir();
+
+ // Define source and dest file
+ File sourceFile = new File(sourceDir, "sourceFile.txt");
+ try {
+ // Create empty source file
+ sourceFile.createNewFile();
+ File destFile = new File(destDir, "destFile_old.txt");
+ // Create empty dest file
+ destFile.createNewFile();
+ FileUtils.moveFileWithOverwrite(sourceFile, new File(destDir, "destFile.txt"));
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ assertTrue(destDir.exists());
Review comment:
How is this testing the `move`?
##########
File path: pinot-common/src/test/java/org/apache/pinot/common/utils/FileUtilsTest.java
##########
@@ -0,0 +1,125 @@
+package org.apache.pinot.common.utils;
+
+import com.google.common.collect.Iterables;
+import junit.framework.TestCase;
+import org.junit.jupiter.api.Test;
+
+import java.io.Closeable;
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.channels.FileChannel;
+import java.util.HashMap;
+import java.util.Map;
+
+public class FileUtilsTest extends TestCase {
+ @Test
+ public void testMoveFileWithOverwrite_destFileNotPresent() {
+ // Create source directory
+ File sourceDir = new File("/tmp/sourceDir");
+ sourceDir.mkdir();
+ // Create destination directory
+ File destDir = new File("/tmp/destDir");
+ destDir.mkdir();
+
+ // Define source and dest file
+ File sourceFile = new File(sourceDir, "sourceFile.txt");
+ try {
+ // Create empty source file
+ sourceFile.createNewFile();
+ FileUtils.moveFileWithOverwrite(sourceFile, new File(destDir, "destFile.txt"));
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ String[] fileList = destDir.list();
+ for(String file : fileList) {
+ if (file.equals("destFile")) {
+ assertTrue(true);
+ break;
+ }
+ }
+ }
+
+ @Test
+ public void testMoveFileWithOverwrite_destFilePresent() {
+ // Create source directory
+ File sourceDir = new File("/tmp/sourceDir");
+ sourceDir.mkdir();
+ // Create destination directory
+ File destDir = new File("/tmp/destDir");
+ destDir.mkdir();
+
+ // Define source and dest file
+ File sourceFile = new File(sourceDir, "sourceFile.txt");
+ try {
+ // Create empty source file
+ sourceFile.createNewFile();
+ File destFile = new File(destDir, "destFile_old.txt");
+ // Create empty dest file
+ destFile.createNewFile();
+ FileUtils.moveFileWithOverwrite(sourceFile, new File(destDir, "destFile.txt"));
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ assertTrue(destDir.exists());
+ }
+
+ @Test
+ public void transferBytes_SuccessFullTest() {
+ File sourceFile = new File("/tmp/source_file.txt");
+ File destFile = new File("/tmp/dest_file.txt");
+ try {
+ // Source file operations
+ org.apache.commons.io.FileUtils.writeStringToFile(sourceFile, "This is dummy content. Please don't read any further....");
+ FileChannel sourceFileChannel = new RandomAccessFile(sourceFile, "r").getChannel();
+ long sourceFileChannelSize = sourceFileChannel.size();
+ // Dest file operations
+ FileChannel destFileChannel = new RandomAccessFile(destFile, "rw").getChannel();
+
+ FileUtils.transferBytes(sourceFileChannel, 1, sourceFileChannelSize-20, destFileChannel);
+ assertEquals(destFileChannel.size(), sourceFileChannel.size()-20);
+
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+
+ @Test
+ public void close_WithIterables() {
+ Map<String, DummyCloseableClass> map = new HashMap<>();
+ File dummyFile = new File("dummyFile.txt");
+ map.put("value", new DummyCloseableClass(dummyFile));
+
+ try {
+ FileUtils.close(Iterables.concat(map.values()));
+ // FileUtils.close(new DummyCloseableClass(new File("somefile")));
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+
+ }
+
+ @Test
+ public void close_WithIndividualCloseable() {
+
+ try {
+ FileUtils.close(new DummyCloseableClass(new File("somefile")));
Review comment:
Perhaps try to read file after closing and assert on the right exception?
##########
File path: pinot-common/src/test/java/org/apache/pinot/common/utils/FileUtilsTest.java
##########
@@ -0,0 +1,125 @@
+package org.apache.pinot.common.utils;
+
+import com.google.common.collect.Iterables;
+import junit.framework.TestCase;
+import org.junit.jupiter.api.Test;
+
+import java.io.Closeable;
+import java.io.File;
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.channels.FileChannel;
+import java.util.HashMap;
+import java.util.Map;
+
+public class FileUtilsTest extends TestCase {
+ @Test
+ public void testMoveFileWithOverwrite_destFileNotPresent() {
+ // Create source directory
+ File sourceDir = new File("/tmp/sourceDir");
+ sourceDir.mkdir();
+ // Create destination directory
+ File destDir = new File("/tmp/destDir");
+ destDir.mkdir();
+
+ // Define source and dest file
+ File sourceFile = new File(sourceDir, "sourceFile.txt");
+ try {
+ // Create empty source file
+ sourceFile.createNewFile();
+ FileUtils.moveFileWithOverwrite(sourceFile, new File(destDir, "destFile.txt"));
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ String[] fileList = destDir.list();
+ for(String file : fileList) {
+ if (file.equals("destFile")) {
+ assertTrue(true);
+ break;
+ }
+ }
+ }
+
+ @Test
+ public void testMoveFileWithOverwrite_destFilePresent() {
+ // Create source directory
+ File sourceDir = new File("/tmp/sourceDir");
+ sourceDir.mkdir();
+ // Create destination directory
+ File destDir = new File("/tmp/destDir");
+ destDir.mkdir();
+
+ // Define source and dest file
+ File sourceFile = new File(sourceDir, "sourceFile.txt");
+ try {
+ // Create empty source file
+ sourceFile.createNewFile();
+ File destFile = new File(destDir, "destFile_old.txt");
+ // Create empty dest file
+ destFile.createNewFile();
+ FileUtils.moveFileWithOverwrite(sourceFile, new File(destDir, "destFile.txt"));
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ assertTrue(destDir.exists());
+ }
+
+ @Test
+ public void transferBytes_SuccessFullTest() {
+ File sourceFile = new File("/tmp/source_file.txt");
+ File destFile = new File("/tmp/dest_file.txt");
+ try {
+ // Source file operations
+ org.apache.commons.io.FileUtils.writeStringToFile(sourceFile, "This is dummy content. Please don't read any further....");
+ FileChannel sourceFileChannel = new RandomAccessFile(sourceFile, "r").getChannel();
+ long sourceFileChannelSize = sourceFileChannel.size();
+ // Dest file operations
+ FileChannel destFileChannel = new RandomAccessFile(destFile, "rw").getChannel();
+
+ FileUtils.transferBytes(sourceFileChannel, 1, sourceFileChannelSize-20, destFileChannel);
+ assertEquals(destFileChannel.size(), sourceFileChannel.size()-20);
+
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+
+ @Test
+ public void close_WithIterables() {
+ Map<String, DummyCloseableClass> map = new HashMap<>();
+ File dummyFile = new File("dummyFile.txt");
+ map.put("value", new DummyCloseableClass(dummyFile));
+
+ try {
+ FileUtils.close(Iterables.concat(map.values()));
+ // FileUtils.close(new DummyCloseableClass(new File("somefile")));
Review comment:
Remove commented out code?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org