You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/01/27 14:44:45 UTC
[GitHub] [flink] pnowojski commented on a change in pull request #10952:
[FLINK-15777][filesystem]Determine Hadoop version with fs classpath.
pnowojski commented on a change in pull request #10952: [FLINK-15777][filesystem]Determine Hadoop version with fs classpath.
URL: https://github.com/apache/flink/pull/10952#discussion_r371270839
##########
File path: flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableFsDataOutputStream.java
##########
@@ -175,44 +172,39 @@ private static void safelyTruncateFile(
}
}
- private static void ensureTruncateInitialized() throws FlinkRuntimeException {
- if (HadoopUtils.isMinHadoopVersion(2, 7) && truncateHandle == null) {
- Method truncateMethod;
+ private static Method findTrunacteMethod(FileSystem fs) throws FlinkRuntimeException {
+ Method truncateMethod = null;
+ if (HadoopUtils.isMinHadoopVersion(fs.getClass().getClassLoader(), 2, 7)) {
try {
- truncateMethod = FileSystem.class.getMethod("truncate", Path.class, long.class);
- }
- catch (NoSuchMethodException e) {
+ truncateMethod = fs.getClass().getMethod("truncate", Path.class, long.class);
+ } catch (NoSuchMethodException e) {
throw new FlinkRuntimeException("Could not find a public truncate method on the Hadoop File System.");
}
-
- if (!Modifier.isPublic(truncateMethod.getModifiers())) {
- throw new FlinkRuntimeException("Could not find a public truncate method on the Hadoop File System.");
- }
-
- truncateHandle = truncateMethod;
}
+
+ return truncateMethod;
}
- private static boolean truncate(final FileSystem hadoopFs, final Path file, final long length) throws IOException {
- if (!HadoopUtils.isMinHadoopVersion(2, 7)) {
- throw new IllegalStateException("Truncation is not available in hadoop version < 2.7 , You are on Hadoop " + VersionInfo.getVersion());
+ private static boolean truncate(
Review comment:
change to non static method and use `truncateHandle` from the instance?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services