You are viewing the static version of this archive.
For the interactive version, please visit: This page
The last 2.8 release (2.8.4) was made in the last May, more than a year ago. https://hadoop.apache.org/old/releases.html How do folks feel about the fate of branch-2.8? During the last community meetup in June, it sounds like most users are still on 2.8 or even 2.7, so I don't think we want to aban...
Dear Hadoop community, Is there a way to setup use quotas for cpu, memory and storage per user/group/project in yarn/hadoop? Thank you very much NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain...
Hi,all: My env. : hive-2.3.0+tez-0.8.5+hadoop-2.7.1, and I want to use hive llap in my project. while it is failed in loading Application Dtails/DAGs/Configurations when redirecting url to tez ui by click "Tracking URL ApplicationMaster". And the whole error in tez-ui(https://xxx.xxx.xxx....
I try "yarn logs -applicationId application_1564655863362_19006" but get "/tmp/logs/chenzl/logs/application_1564655863362_19006 does not have any log files". Does spark only get output or logs in kill app running in cluster mode?
Hello, I have a cluster with one machine holding the name nodes (primary and secondary) a yarn node (resource manager) and four data nodes. I am running hadoop 2.7.0. When I submit a job to the cluster I can see it in the scheduler webpage. If I go to the container page and check the logs, in the ...
Hello! For this week's community online sync (English, Wednesday 9am US Pacific Time), we will have CR Hota from Uber to talk about the latest update in Router Based Federation. He will touch upon the following topics: 1. Security (Development and zookeeper scale testing learnings) 2. Isolation fo...
Very happy to have CR from Uber leading today's discussion. Here's today's sync meeting notes. https://docs.google.com/document/d/1jXM5Ujvf-zhcyw_5kiQVx6g-HeKe-YGnFS_1-qFXomI/edit 8/5/2019 CR Hota (Uber) gave an update on Router Based Federation Attendee: Cloudera (Weichiu, Adam), Uber (CR Hota) a...
Getting confused when trying to run a YARN process and getting errors. Looking in ambari UI YARN section, seeing... [image: enter image description here] <https://i.stack.imgur.com/0Fohu.png>[image: enter image description here] <https://i.stack.imgur.com/nHpX3.png>(note it says 60GB available). Yet...
The Hadoop community welcome your patch contribution, and like increasingly patches are submitted via GitHub Pull Requests. That is great, as it reduces the friction to review code & commit code. However, please make sure to file a jira for your PR, as described in the How to Contribute <https://c...
Hello, I have the following task : An application that stores files, enables a user to add and delete files. When such an event occurs I append to a file in a hdfs the following record when there was a file added : userid image-uuid size_in_bytes and the following when a file was removed -useri...
Using HDFS NFS gateway with HDP 3.1 and seeing odd behavior where directories accessed via NFS mount have permissions different that those of the local host user that created them (eg. via "hadoop fs -mkdir ..." command). Eg. [etl@HW04 data]$ hadoop fs -ls /user Found 13 items drwx------ - myuser...
Having a problem with HDFS NFS, addressed on another site <https://community.pivotal.io/s/article/Write-data-to-HDFS-via-NFS-gateway-failed-with-Input-output-error> where it is recommended to set hdfs-site.xml like... <property> <name>dfs.namenode.accesstime.precision</name> <value>3600000</value>...
Unsubscribe Sent from my iPhone --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org For additional commands, e-mail: user-help@hadoop.apache.org
Hi, I would like to ask whether the Fault Injection Framework ( https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/FaultInjectFramework.html) is still supported in the Hadoop HDFS? I think that this documentation was created around *v0.23* and was not updated since then. Additio...
Hi all, I installed a basic 3 nodes Hadoop 2.9.1 cluster and playing with YARN settings. The 3 nodes has following configuration: 1 cpu / 1 core / 512MB RAM I wonder I was able to configure yarn-site.xml with following settings (higher than node physical limits) and successfully run a mapreduce '...
Hi all, As far as I know, HDFS is designed to target local file systems like ext4 or xfs. Is it a bad approach to use SAN technology as storage for HDFS? Thank you, Daegyu ᐧ
Hi all, I'm currently studying HDFS, and I want to analyze HDFS io latency. I know that C / C ++ programs can use perf and ftrace under Linux to analyze user level and kernel level latency measurements and overhead. I would like to analyze the read io latency in HDFS to user level (HDFS) and syst...
I received some feedbacks that the bi-weekly storage online sync that happens Wednesday 9AM US pacific time (GMT-8) is too early for west coast folks, and the fact is that the majority of Hadoop developers are in the US. Would it make sense to move it to a later time to allow more US west coast par...
For this week, We will have Konstantin and the LinkedIn folks to discuss a recent project that's been baking for quite a while. This is an exciting project as it has the potential to improve NameNode's throughput by 40%. HDFS-14703 <https://issues.apache.org/jira/browse/HDFS-14703> NameNode Fine-Gr...
Hi all, When we configure kerberos to submit tasks, only the primary account can be submitted successfully, and other accounts cannot be submitted. Report Could not find or load main class org.apache.hadoop.yarn.server.nodemanager.containermanager.loca lizer.ContainerLocalizer My question is the ...