You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Konstantin Boudnik (JIRA)" <ji...@apache.org> on 2009/09/15 18:49:57 UTC
[jira] Created: (HDFS-620) HDFS code is extensively using system
property 'test.build.data' which is likely to fail under Eclipse
environment
HDFS code is extensively using system property 'test.build.data' which is likely to fail under Eclipse environment
------------------------------------------------------------------------------------------------------------------
Key: HDFS-620
URL: https://issues.apache.org/jira/browse/HDFS-620
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Konstantin Boudnik
Because some of the system properties aren't available in Eclipse environment and there's a sufficient number usages of 'test.data.dir' system property the chances are that some of the tests won't work in Eclipse. There were two occasions of such behavior which were fixed already (see HDFS-601, HDFS-614). Although many more are left intact for post append time frame
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-620) HDFS code is extensively using system
property 'test.build.data' which is likely to fail under Eclipse
environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated HDFS-620:
------------------------------------
Description: Because some of the system properties aren't available in Eclipse environment and there's a sufficient number usages of 'test.build.data' system property the chances are that some of the tests won't work in Eclipse. There were two occasions of such behavior which were fixed already (see HDFS-601, HDFS-614). Although many more are left intact for post append time frame (was: Because some of the system properties aren't available in Eclipse environment and there's a sufficient number usages of 'test.data.dir' system property the chances are that some of the tests won't work in Eclipse. There were two occasions of such behavior which were fixed already (see HDFS-601, HDFS-614). Although many more are left intact for post append time frame)
> HDFS code is extensively using system property 'test.build.data' which is likely to fail under Eclipse environment
> ------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-620
> URL: https://issues.apache.org/jira/browse/HDFS-620
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Konstantin Boudnik
>
> Because some of the system properties aren't available in Eclipse environment and there's a sufficient number usages of 'test.build.data' system property the chances are that some of the tests won't work in Eclipse. There were two occasions of such behavior which were fixed already (see HDFS-601, HDFS-614). Although many more are left intact for post append time frame
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-620) HDFS code is extensively using system
property 'test.build.data' which is likely to fail under Eclipse
environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755623#action_12755623 ]
Konstantin Boudnik commented on HDFS-620:
-----------------------------------------
In the above comment I've used a wrong name for the system property. It always should read 'test.build.data'. Thanks for indirectly pointing this out, Philip!
> HDFS code is extensively using system property 'test.build.data' which is likely to fail under Eclipse environment
> ------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-620
> URL: https://issues.apache.org/jira/browse/HDFS-620
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Konstantin Boudnik
>
> Because some of the system properties aren't available in Eclipse environment and there's a sufficient number usages of 'test.build.data' system property the chances are that some of the tests won't work in Eclipse. There were two occasions of such behavior which were fixed already (see HDFS-601, HDFS-614). Although many more are left intact for post append time frame
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-620) HDFS code is extensively using system
property 'test.build.data' which is likely to fail under Eclipse
environment
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755574#action_12755574 ]
Philip Zeyliger commented on HDFS-620:
--------------------------------------
FWIW, the Eclipse templates in https://issues.apache.org/jira/browse/HDFS-598 specify test.build.data, and a couple other things. That said, I'm very much +100 for standardizing on how to get at this sort of thing from code, as opposed to environment variables.
> HDFS code is extensively using system property 'test.build.data' which is likely to fail under Eclipse environment
> ------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-620
> URL: https://issues.apache.org/jira/browse/HDFS-620
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Konstantin Boudnik
>
> Because some of the system properties aren't available in Eclipse environment and there's a sufficient number usages of 'test.data.dir' system property the chances are that some of the tests won't work in Eclipse. There were two occasions of such behavior which were fixed already (see HDFS-601, HDFS-614). Although many more are left intact for post append time frame
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-620) HDFS code is extensively using system
property 'test.build.data' which is likely to fail under Eclipse
environment
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756245#action_12756245 ]
Jakob Homan commented on HDFS-620:
----------------------------------
My last comment wasn't particularly insightful without context: it was part of me linking this issue to HADOOP-5916. This may be considered a duplicate or extension of that issue. It would be great if we could be tidy with our files and stick to build and keep Eclipse happy.
> HDFS code is extensively using system property 'test.build.data' which is likely to fail under Eclipse environment
> ------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-620
> URL: https://issues.apache.org/jira/browse/HDFS-620
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Konstantin Boudnik
>
> Because some of the system properties aren't available in Eclipse environment and there's a sufficient number usages of 'test.build.data' system property the chances are that some of the tests won't work in Eclipse. There were two occasions of such behavior which were fixed already (see HDFS-601, HDFS-614). Although many more are left intact for post append time frame
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-620) HDFS code is extensively using system
property 'test.build.data' which is likely to fail under Eclipse
environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755580#action_12755580 ]
Konstantin Boudnik commented on HDFS-620:
-----------------------------------------
Ideally, wherever {{MiniDFSCluster}} is used aforementioned system property should be replaced with {{getDataDirectory()}} invocation from {{MiniDFSCluster}}'s instance. However, one should be advised that these two locations aren't equivalent:
- 'test.data.dir' points to {{${build}/test/data}}
- {{getDataDirectory()}} refers to {{${build}/test/data/dfs/data}}
Here's the list of the potentially offensive uses of 'test.data.dir'. Be advised that in some cases such a use might be well justified.
{noformat}
./src/contrib/hdfsproxy/src/test/org/apache/hadoop/hdfsproxy/TestHdfsProxy.java: "test.build.data", "/tmp")).toString().replace(' ', '+');
./src/test/hdfs/org/apache/hadoop/fs/loadGenerator/TestLoadGenerator.java: System.getProperty("test.build.data","build/test/data");
./src/test/hdfs/org/apache/hadoop/fs/loadGenerator/TestLoadGenerator.java: final String SCRIPT_TEST_DIR = new File(System.getProperty("test.build.data",
./src/test/hdfs/org/apache/hadoop/hdfs/BenchmarkThroughput.java: System.setProperty("test.build.data", localDir);
./src/test/hdfs/org/apache/hadoop/hdfs/DataNodeCluster.java: System.setProperty("test.build.data", dataNodeDirs);
./src/test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java: return System.getProperty("test.build.data", "build/test/data") + "/dfs/";
./src/test/hdfs/org/apache/hadoop/hdfs/server/common/TestDistributedUpgrade.java: if (System.getProperty("test.build.data") == null) { // to test to be run outside of ant
./src/test/hdfs/org/apache/hadoop/hdfs/server/common/TestDistributedUpgrade.java: System.setProperty("test.build.data", "build/test/data");
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestFsck.java: File baseDir = new File(System.getProperty("test.build.data",
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestFsck.java: File baseDir = new File(System.getProperty("test.build.data",
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestNameEditsConfigs.java: System.getProperty("test.build.data", "build/test/data"), "dfs/");
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestOverReplicatedBlocks.java: File scanLog = new File(System.getProperty("test.build.data"),
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStartup.java: String baseDir = System.getProperty("test.build.data", "/tmp");
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStorageRestore.java: String baseDir = System.getProperty("test.build.data", "build/test/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestCrcCorruption.java: File data_dir = new File(System.getProperty("test.build.data"),
./src/test/hdfs/org/apache/hadoop/hdfs/TestCrcCorruption.java: data_dir = new File(System.getProperty("test.build.data"),
./src/test/hdfs/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java: File baseDir = new File(System.getProperty("test.build.data"), "dfs/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java: File baseDir = new File(System.getProperty("test.build.data"), "dfs/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java: File baseDir = new File(System.getProperty("test.build.data"), "dfs/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSShell.java: new Path(System.getProperty("test.build.data","/tmp"))
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSShell.java: String bak = System.getProperty("test.build.data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSShell.java: System.setProperty("test.build.data", nameDir.toString());
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSShell.java: System.setProperty("test.build.data", bak);
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSUpgradeFromImage.java: String dataDir = System.getProperty("test.build.data", "build/test/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSUpgradeFromImage.java: if (System.getProperty("test.build.data") == null) { // to allow test to be run outside of Ant
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSUpgradeFromImage.java: System.setProperty("test.build.data", "build/test/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestFileCorruption.java: File data_dir = new File(System.getProperty("test.build.data"),
./src/test/hdfs/org/apache/hadoop/hdfs/TestFileCorruption.java: Path file = new Path(System.getProperty("test.build.data"), "corruptFile");
./src/test/hdfs/org/apache/hadoop/hdfs/TestFSInputChecker.java: String dir = System.getProperty("test.build.data", ".");
./src/test/hdfs/org/apache/hadoop/hdfs/TestHDFSServerPorts.java: return System.getProperty("test.build.data", "build/test/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestReplication.java: File baseDir = new File(System.getProperty("test.build.data"),
./src/test/hdfs/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestDelimitedImageVisitor.java: private static String ROOT = System.getProperty("test.build.data","/tmp");
./src/test/hdfs/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java: private static String ROOT = System.getProperty("test.build.data",
./src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java: System.getProperty("test.build.data","/tmp").replace(' ', '+'));
./src/test/hdfs-with-mr/org/apache/hadoop/fs/DFSCIOTest.java: private static String TEST_ROOT_DIR = System.getProperty("test.build.data","/benchmarks/DFSCIOTest");
./src/test/hdfs-with-mr/org/apache/hadoop/fs/DistributedFSCheck.java: private static Path TEST_ROOT_DIR = new Path(System.getProperty("test.build.data","/benchmarks/DistributedFSCheck"));
./src/test/hdfs-with-mr/org/apache/hadoop/fs/JHLogAnalyzer.java: System.getProperty("test.build.data", "stats/JHLA");
./src/test/hdfs-with-mr/org/apache/hadoop/fs/TestDFSIO.java: private static System.getProperty("test.build.data","/benchmarks/TestDFSIO");
./src/test/hdfs-with-mr/org/apache/hadoop/fs/TestFileSystem.java: private static String ROOT = System.getProperty("test.build.data","fs_test");
./src/test/hdfs-with-mr/org/apache/hadoop/fs/TestJHLA.java: private String historyLog = System.getProperty("test.build.data",
./src/test/hdfs-with-mr/org/apache/hadoop/io/TestSequenceFileMergeProgress.java: Path dir = new Path(System.getProperty("test.build.data",".") + "/mapred");
{noformat}
> HDFS code is extensively using system property 'test.build.data' which is likely to fail under Eclipse environment
> ------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-620
> URL: https://issues.apache.org/jira/browse/HDFS-620
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Konstantin Boudnik
>
> Because some of the system properties aren't available in Eclipse environment and there's a sufficient number usages of 'test.data.dir' system property the chances are that some of the tests won't work in Eclipse. There were two occasions of such behavior which were fixed already (see HDFS-601, HDFS-614). Although many more are left intact for post append time frame
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-620) HDFS code is extensively using system
property 'test.build.data' which is likely to fail under Eclipse
environment
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755582#action_12755582 ]
Konstantin Boudnik commented on HDFS-620:
-----------------------------------------
Ideally, wherever {{MiniDFSCluster}} is used aforementioned system property should be replaced with {{getDataDirectory()}} invocation from {{MiniDFSCluster}}'s instance. However, one should be advised that these two locations aren't equivalent:
- 'test.data.dir' points to {{${build}/test/data}}
- {{getDataDirectory()}} refers to {{${build}/test/data/dfs/data}}
Here's the list of the potentially offensive uses of 'test.data.dir'. Be advised that in some cases such a use might be well justified.
{noformat}
./src/contrib/hdfsproxy/src/test/org/apache/hadoop/hdfsproxy/TestHdfsProxy.java: "test.build.data", "/tmp")).toString().replace(' ', '+');
./src/test/hdfs/org/apache/hadoop/fs/loadGenerator/TestLoadGenerator.java: System.getProperty("test.build.data","build/test/data");
./src/test/hdfs/org/apache/hadoop/fs/loadGenerator/TestLoadGenerator.java: final String SCRIPT_TEST_DIR = new File(System.getProperty("test.build.data",
./src/test/hdfs/org/apache/hadoop/hdfs/BenchmarkThroughput.java: System.setProperty("test.build.data", localDir);
./src/test/hdfs/org/apache/hadoop/hdfs/DataNodeCluster.java: System.setProperty("test.build.data", dataNodeDirs);
./src/test/hdfs/org/apache/hadoop/hdfs/MiniDFSCluster.java: return System.getProperty("test.build.data", "build/test/data") + "/dfs/";
./src/test/hdfs/org/apache/hadoop/hdfs/server/common/TestDistributedUpgrade.java: if (System.getProperty("test.build.data") == null) { // to test to be run outside of ant
./src/test/hdfs/org/apache/hadoop/hdfs/server/common/TestDistributedUpgrade.java: System.setProperty("test.build.data", "build/test/data");
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestFsck.java: File baseDir = new File(System.getProperty("test.build.data",
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestFsck.java: File baseDir = new File(System.getProperty("test.build.data",
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestNameEditsConfigs.java: System.getProperty("test.build.data", "build/test/data"), "dfs/");
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestOverReplicatedBlocks.java: File scanLog = new File(System.getProperty("test.build.data"),
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStartup.java: String baseDir = System.getProperty("test.build.data", "/tmp");
./src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStorageRestore.java: String baseDir = System.getProperty("test.build.data", "build/test/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestCrcCorruption.java: File data_dir = new File(System.getProperty("test.build.data"),
./src/test/hdfs/org/apache/hadoop/hdfs/TestCrcCorruption.java: data_dir = new File(System.getProperty("test.build.data"),
./src/test/hdfs/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java: File baseDir = new File(System.getProperty("test.build.data"), "dfs/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java: File baseDir = new File(System.getProperty("test.build.data"), "dfs/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java: File baseDir = new File(System.getProperty("test.build.data"), "dfs/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSShell.java: new Path(System.getProperty("test.build.data","/tmp"))
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSShell.java: String bak = System.getProperty("test.build.data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSShell.java: System.setProperty("test.build.data", nameDir.toString());
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSShell.java: System.setProperty("test.build.data", bak);
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSUpgradeFromImage.java: String dataDir = System.getProperty("test.build.data", "build/test/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSUpgradeFromImage.java: if (System.getProperty("test.build.data") == null) { // to allow test to be run outside of Ant
./src/test/hdfs/org/apache/hadoop/hdfs/TestDFSUpgradeFromImage.java: System.setProperty("test.build.data", "build/test/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestFileCorruption.java: File data_dir = new File(System.getProperty("test.build.data"),
./src/test/hdfs/org/apache/hadoop/hdfs/TestFileCorruption.java: Path file = new Path(System.getProperty("test.build.data"), "corruptFile");
./src/test/hdfs/org/apache/hadoop/hdfs/TestFSInputChecker.java: String dir = System.getProperty("test.build.data", ".");
./src/test/hdfs/org/apache/hadoop/hdfs/TestHDFSServerPorts.java: return System.getProperty("test.build.data", "build/test/data");
./src/test/hdfs/org/apache/hadoop/hdfs/TestReplication.java: File baseDir = new File(System.getProperty("test.build.data"),
./src/test/hdfs/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestDelimitedImageVisitor.java: private static String ROOT = System.getProperty("test.build.data","/tmp");
./src/test/hdfs/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java: private static String ROOT = System.getProperty("test.build.data",
./src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java: System.getProperty("test.build.data","/tmp").replace(' ', '+'));
./src/test/hdfs-with-mr/org/apache/hadoop/fs/DFSCIOTest.java: private static String TEST_ROOT_DIR = System.getProperty("test.build.data","/benchmarks/DFSCIOTest");
./src/test/hdfs-with-mr/org/apache/hadoop/fs/DistributedFSCheck.java: private static Path TEST_ROOT_DIR = new Path(System.getProperty("test.build.data","/benchmarks/DistributedFSCheck"));
./src/test/hdfs-with-mr/org/apache/hadoop/fs/JHLogAnalyzer.java: System.getProperty("test.build.data", "stats/JHLA");
./src/test/hdfs-with-mr/org/apache/hadoop/fs/TestDFSIO.java: private static System.getProperty("test.build.data","/benchmarks/TestDFSIO");
./src/test/hdfs-with-mr/org/apache/hadoop/fs/TestFileSystem.java: private static String ROOT = System.getProperty("test.build.data","fs_test");
./src/test/hdfs-with-mr/org/apache/hadoop/fs/TestJHLA.java: private String historyLog = System.getProperty("test.build.data",
./src/test/hdfs-with-mr/org/apache/hadoop/io/TestSequenceFileMergeProgress.java: Path dir = new Path(System.getProperty("test.build.data",".") + "/mapred");
{noformat}
> HDFS code is extensively using system property 'test.build.data' which is likely to fail under Eclipse environment
> ------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-620
> URL: https://issues.apache.org/jira/browse/HDFS-620
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Konstantin Boudnik
>
> Because some of the system properties aren't available in Eclipse environment and there's a sufficient number usages of 'test.data.dir' system property the chances are that some of the tests won't work in Eclipse. There were two occasions of such behavior which were fixed already (see HDFS-601, HDFS-614). Although many more are left intact for post append time frame
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.