You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Íñigo Goiri (Jira)" <ji...@apache.org> on 2021/04/02 16:39:00 UTC
[jira] [Commented] (HDFS-15944) Prevent truncation by snprintf
[ https://issues.apache.org/jira/browse/HDFS-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313961#comment-17313961 ]
Íñigo Goiri commented on HDFS-15944:
------------------------------------
Thanks [~gautham] for the fix.
Merged PR 2853.
> Prevent truncation by snprintf
> ------------------------------
>
> Key: HDFS-15944
> URL: https://issues.apache.org/jira/browse/HDFS-15944
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fuse-dfs, libhdfs
> Affects Versions: 3.4.0
> Reporter: Gautham Banasandra
> Assignee: Gautham Banasandra
> Priority: Critical
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> There are some areas of code in libhdfs and fuse-dfs components where the destination buffer is smaller than the source that's trying to write. This would cause truncation. Thus we need to ensure that the source that's being written doesn't exceed the destination buffer size.
> The following warnings are reported for this issue -
> {code}
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c: In function ‘doTestHdfsOperations.isra.0’:
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:226:39: warning: ‘/many_files_’ directive output may be truncated writing 12 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
> 226 | snprintf(filename, PATH_MAX, "%s/many_files_%d", listDirTest, nFile);
> | ^~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:226:36: note: directive argument in the range [0, 9999]
> 226 | snprintf(filename, PATH_MAX, "%s/many_files_%d", listDirTest, nFile);
> | ^~~~~~~~~~~~~~~~~~
> In file included from /usr/include/stdio.h:867,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:19:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 14 and 4112 bytes into a destination of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> {code}
> {code}
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:255:33: warning: ‘/a’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
> 255 | snprintf(tmp, sizeof(tmp), "%s/a", base);
> | ^~
> In file included from /usr/include/stdio.h:867,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:263:33: warning: ‘/b’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
> 263 | snprintf(tmp, sizeof(tmp), "%s/b", base);
> | ^~
> In file included from /usr/include/stdio.h:867,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:271:33: warning: ‘/a’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
> 271 | snprintf(src, sizeof(src), "%s/a", base);
> | ^~
> In file included from /usr/include/stdio.h:867,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:272:33: warning: ‘/c’ directive output may be truncated writing 2 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
> 272 | snprintf(dst, sizeof(dst), "%s/c", base);
> | ^~
> In file included from /usr/include/stdio.h:867,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:297:35: warning: ‘/b/’ directive output may be truncated writing 3 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
> 297 | snprintf(tmp, sizeof(tmp), "%s/b/%d", base, i);
> | ^~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:297:32: note: directive argument in the range [0, 2]
> 297 | snprintf(tmp, sizeof(tmp), "%s/b/%d", base, i);
> | ^~~~~~~~~
> In file included from /usr/include/stdio.h:867,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 5 and 4100 bytes into a destination of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:187:35: warning: ‘/trunc.txt’ directive output may be truncated writing 10 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
> 187 | snprintf(path, sizeof(path), "%s/trunc.txt", base);
> | ^~~~~~~~~~
> In file included from /usr/include/stdio.h:867,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 11 and 4106 bytes into a destination of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org