You are viewing the static version of this archive.
For the interactive version, please visit: This page
hello everyone, I use 'black duck' to scan hadoop and found a vulnerability below: BDSA-2018-1828Apache Hadoop is vulnerable to an arbitrary file write vulnerability via a directory traversal. An attacker could exploit this vulnerability by supplying the component with a malic...
Hi! I use systemd to control hdfs service, when reboot, i find below issue: 11月 13 14:47:58 test-storage.novalocal start-dfs.sh[1915]: Starting namenodes on [test-storage.novalocal] 11月 13 14:48:03 test-storage.novalocal start-dfs.sh[1915]: test-storage.novalocal: starting namenode, logging...
Hi all, I'm trying to delete files from HDFS using this command: hdfs dfs -ls /projects/BX77004/tmp | grep -e REM_AV1876B_ADL_12_20180720143210 | awk '{print $8}' | xargs hdfs dfs -rm I want to delete all files that begin with REM_AV1876B_ADL_12_20180720143210, but response is: rm: `/projects/BX...
Hi, Someone can enable JMX server for Namenode and Datanode by adding the journal node options to variables HADOOP_NAMENODE_OPTS and HADOOP_DATANODE_OPTS, respectively. So I'm looking for the corresponding variable for Journalnodes. Is there such a variable?
Hi Hadoop community. We are having this issue of Hadoop jobs stay in pending state after job completes: Here is the screenshot of it: [cid:image001.png@01D480B7.937E85A0] Mapper task finished within 2min, this is short running job, but Hadoop holds the job in pending for ~15 min then complete...
Hi all, I am pleased to announce that the Apache Hadoop 2.9.2 has been released. Apache Hadoop 2.9.2 is the next release of Apache Hadoop 2.9 line, which includes 204 fixes since previous Hadoop 2.9.1 release. - For major changes included in Hadoop 2.9 line, please refer Hadoop 2.9.2 main page [1...
CVE-2018-8009: Apache Hadoop distributed cache archive vulnerability Severity: Severe Vendor: The Apache Software Foundation Versions Affected: Hadoop 0.23.0 to 0.23.11 Hadoop 2.0.0-alpha to 2.7.6 Hadoop 2.8.0 to 2.8.4 Hadoop 2.9.0 to 2.9.1 Hadoop 3.0.0-alpha to 3.0.2 Hadoop 3.1.0 Us...
It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop Ozone 0.3.0-alpha (Arches). Apache Hadoop Ozone is an object store for Hadoop built using Hadoop Distributed Data Store. This release contains a new S3 compatible interface and additional st...
On HDP3, I cannot get the full log of a failing spark job by using yarn api: curl -k -u guest:"" -X GET https:// myhost.com/gateway/ui/resourcemanager/v1/cluster/apps/ <https://confluence.oci.oraclecorp.com/display/BDW/dw-knox-prod1.us-phoenix-1.oracleiaas.com/gateway/ui/resourcemanager/v1/cluster/...
CVE-2018-11766: Apache Hadoop privilege escalation vulnerability Severity: Critical Vendor: The Apache Software Foundation Versions Affected: Apache Hadoop versions from 2.7.4 to 2.7.6 Description: In Apache Hadoop 2.7.4 to 2.7.6, the security fix for CVE-2016-6811 is incomplete. A user who can ...
Hi: On the apache Hadoop doc,that says Lasy_Persist policy is useful only for single replica blocks. Lasy_Persist is for set for ram disk My question is (1)when memory is full,would blocks write to the disks? (2)when cluster unfortunately shot down, it means my data gone? 從 Windows 10 的...
hi all, I am on hadoop hdfs version of Hadoop 2.6.0-cdh5.8.0. I discovered that the DirectoryScanner is keep causing java GC and slow down the hadoop nodes. Digging into the log file I discovered this: 2018-11-29 13:34:37,995 INFO org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: BlockPool ...
Hi, I��m running Flink applications on YARN 2.6.0-cdh5.6.0 and get a situation. After running for a while (could be longer than 7 days) the application might need to rescale up or recover from a node failure but it is not able to allocate new containers. All the incoming containers would fail t...
When a machine dies in a Hadoop cluster, the data in mapreduce.job.local.dir is presumably lost. This data contains mapper output files as well as reducer input files. In addition, data stored in the Shuffle Fetcher memory is lost. How do we ensure that this lost data is re-processed when a machine...
hi I am seeing that there is a url mistake on the main page of https://hadoop.apache.org/, having a Ambari link pointing to https://apache.ambari.org instead of https://ambari.apache.org. Can someone tell me any ways that I can download the codes and do a PR to fix this? Thanks! -- Hiu Yen Onn ...
Please unsubscribe me Thanks Al --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org For additional commands, e-mail: user-help@hadoop.apache.org
Hi, AFAIK, YARN RM rolls master keys used to generate AMRMToken periodically (default to 1 day). Before the new master key takes effect, there's a window that the AMs can update its AMRMToken, during which RM would generate a new AMRMToken and send it back to the AM when it detects an AMRMToken usi...
--------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org For additional commands, e-mail: user-help@hadoop.apache.org
hello, everyone: Why there is no Java API for command "hadoop key create" in DistributedFileSystem? 胡晓东 huxiaodong 网管及服务系统部 Network Management & Service System Dept M...