You are viewing the static version of this archive.
For the interactive version, please visit: This page
I am using HDP3.0 which uses HADOOP3.1.0 and Spark 2.3.1. My spark streaming jobs running fine in HDP2.6.4 (HADOOP2.7.3, spark 2.2.0) fails in HDP3: java.lang.IllegalAccessError: class org.apache.hadoop.hdfs.web.HftpFileSystem cannot access its superinterface org.apache.hadoop.hdfs.web.TokenAspect$...
unsubscribe
Hi, I am trying to use larger block size like 1GB, 2GB. In our case the files are 5GB to 12GB size and we process whole file per mapper, is there any side effects to using larger block size like 1GB, 2GB? Like HDFS stability when doing replication? Thanks Sudhir ------------------------------------...
Hi, I am trying to use larger block size like 1GB, 2GB. In our case the files are 5GB to 12GB size and we process whole file per mapper, is there any side effects to using larger block size like 1GB, 2GB? Like HDFS stability when doing replication? Thanks Sudhir ------------------------------------...
We need your help to make the Apache Washington DC Roadshow on Dec 4th a success. What do we need most? Speakers! We're bringing a unique DC flavor to this event by mixing Open Source Software with talks about Apache projects as well as OSS CyberSecurity, OSS in Government and and OSS Career ad...
Hi everyone! I'm running a simple sql(select xx,xx... from viewXXX where xxxxx) using hive0.13.1 on hadoop2.6.0(the framework is MRv2, not tez). After submitting it, I find that it's a MR job which has only 17000+ map tasks a...
When querying different HDFS clusters I get different separators (don't really know if they're actually separators). From zkCli.sh on different clusters, running the following I get: cmd: get /hadoop-ha/clusterX/ActiveBreadCrumb Cluster1 (comma): cluster1active-nn1,active-nn1.example.com �>(�...
Hello, We started are using CGroups with LinuxContainerExecutor recently, running Apache Hadoop 3.0.0. Occasionally (once out of many millions of tasks) a yarn container will fail with a message like the following: WARN privileged.PrivilegedOperationExecutor: Shell execution returned exit code: 35....
Hi, How do i access MR Job Info (like total num of Map/Reduce, Avg Map/Reduce Time, Shuffle time etc) from Job History Server with JAVA Program ? I was trying to use HistoryViewer.AnalyzedJob and HistoryViewer.SummarizedJob classes but HisotryViewer is marked as InterfaceAudience.Private. Is ther...
We are launching YARN jobs in which the tasks access Hive. While it is fairly simple to get the delegation token for Metastore and pass it along to the YARN context, we cannot make the same thing work for HiveServer2. Is there an example someone can point us to? And yes, we know that Oozie and Ma...
Hi everyone, After the release of the new Hadoop website the latest releases of v2.7 and v2.8 are not listet on the download page. Are there reasons, e.g. problems in that releases to list older releases only? Best regards Bob
Hey, I am a researcher, and my focus is on Hadoop's performance mainly about the allocation of reduce Tasks. The idea is about interfering with the current allocation of tasks (Reduce tasks) in Hadoop (which is made by AM and RM of YARN), and it can be divided into the following questions (regarding...
If you are trying to use the Job History Server then I would suggest you follow the next two links on how to activate and enter the UI. https://stackoverflow.com/questions/42884289/job-history-server-in-hadoop-2-7-1-is-not-working https://stackoverflow.com/questions/50200130/hadoop-cant-access-track...