You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by as...@apache.org on 2016/03/08 20:06:19 UTC
[18/50] [abbrv] hadoop git commit: HADOOP-11792. Remove all of the
CHANGES.txt files.
http://git-wip-us.apache.org/repos/asf/hadoop/blob/1bc45461/hadoop-yarn-project/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt
deleted file mode 100644
index 4934e89..0000000
--- a/hadoop-yarn-project/CHANGES.txt
+++ /dev/null
@@ -1,6905 +0,0 @@
-Hadoop YARN Change Log
-
-Trunk - Unreleased
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- YARN-2882. Add an OPPORTUNISTIC ExecutionType.
- (Konstantinos Karanasos and Inigo Goiri via kasha)
-
- IMPROVEMENTS
-
- YARN-2438. yarn-env.sh cleanup (aw)
-
- YARN-2472. yarn-daemons.sh should jsut call yarn directly (Masatake Iwasaki
- via aw)
-
- YARN-2437. start-yarn.sh/stop-yarn should give info (Varun Saxena via aw)
-
- YARN-2796. deprecate sbin/yarn-daemon.sh (aw)
-
- YARN-2980. Move health check script related functionality to hadoop-common
- (Varun Saxena via aw)
-
- YARN-3168. Convert site documentation from apt to markdown (Gururaj Shetty
- via aw)
-
- YARN-3199. Fair Scheduler documentation improvements (Rohit Agarwal via
- aw)
-
- YARN-2280. Resource manager web service fields are not accessible
- (Krisztian Horvath via aw)
-
- YARN-3261. rewrite resourcemanager restart doc to remove roadmap bits (Gururaj Shetty via aw)
-
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via gtcarrera9)
-
- YARN-524 TestYarnVersionInfo failing if generated properties doesn't
- include an SVN URL. (stevel)
-
- YARN-1471. The SLS simulator is not running the preemption policy
- for CapacityScheduler (Carlo Curino via cdouglas)
-
- YARN-2436. [post-HADOOP-9902] yarn application help doesn't work (aw)
-
- YARN-2525. yarn logs command gives error on trunk (Akira AJISAKA via aw)
-
- YARN-3002. YARN documentation needs updating post-shell rewrite (aw)
-
- YARN-2428. LCE default banned user list should have yarn (Varun
- Saxena via aw)
-
- YARN-2355. MAX_APP_ATTEMPTS_ENV may no longer be a useful env var
- for a container (Darrell Taylor via aw)
-
- YARN-3915. scmadmin help message correction (Bibin A Chundatt via aw)
-
-Release 2.9.0 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- YARN-1856. Added cgroups based memory monitoring for containers as another
- alternative to custom memory-monitoring. (Varun Vasudev via vinodkv)
-
- YARN-4462. FairScheduler: Disallow preemption from a queue. (Tao Jie via kasha)
-
- IMPROVEMENTS
-
- YARN-4072. ApplicationHistoryServer, WebAppProxyServer, NodeManager and
- ResourceManager to support JvmPauseMonitor as a service.
- (Sunil G via Stevel)
-
- YARN-4341. add doc about timeline performance tool usage (Chang Li via
- sjlee)
-
- YARN-4417. Make RM and Timeline-server REST APIs more consistent.
- (wtan via jianhe)
-
- YARN-3480. Remove attempts that are beyond max-attempt limit from state
- store. (Jun Gong via jianhe)
-
- YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda)
-
- YARN-4438. Implement RM leader election with curator. (Jian He via xgong)
-
- YARN-4571. Make app id/name available to the yarn authorizer provider for
- better auditing. (Jian He via wangda)
-
- YARN-4551. Address the duplication between StatusUpdateWhenHealthy and
- StatusUpdateWhenUnhealthy transitions. (Sunil G via kasha)
-
- YARN-4553. Add cgroups support for docker containers.
- (Sidharta Seethana via vvasudev)
-
- YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it.
- (kasha)
-
- YARN-4559. Make leader elector and zk store share the same curator client.
- (Jian He via xgong)
-
- YARN-4603. FairScheduler should mention user requested queuename in error
- message when failed in queue ACL check. (Tao Jie via kasha)
-
- YARN-4496. Improve HA ResourceManager Failover detection on the client.
- (Jian He via xgong)
-
- YARN-3542. Refactored existing CPU cgroups support to use the newer and
- integrated ResourceHandler mechanism, and also deprecated the old
- LCEResourceHandler inteface hierarchy. (Varun Vasudev via vinodkv)
-
- YARN-4647. Make RegisterNodeManagerRequestPBImpl thread-safe. (kasha)
-
- YARN-4649. Add additional logging to some NM state store operations.
- (Sidharta Seethana via vvasudev)
-
- YARN-4625. Make ApplicationSubmissionContext and
- ApplicationSubmissionContextInfo more consistent.
- (Xuan Gong via vvasudev)
-
- YARN-4307. Display blacklisted nodes for AM container in the RM web UI.
- (Naganarasimha G R via vvasudev)
-
- YARN-4628. Display application priority in yarn top.
- (Bibin A Chundatt via vvasudev)
-
- YARN-4655. Log uncaught exceptions/errors in various thread pools in YARN.
- (Sidharta Seethana via vvasudev)
-
- YARN-4689. FairScheduler: Cleanup preemptContainer to be more readable.
- (Kai Sasaki via kasha)
-
- YARN-4708. Missing default mapper type in TimelineServer performance test tool
- usage. (Kai Sasaki via ozawa)
-
- YARN-4648. Move preemption related tests from TestFairScheduler to
- TestFairSchedulerPreemption. (Kai Sasaki via ozawa)
-
- YARN-4697. NM aggregation thread pool is not bound by
- limits (haibochen via rkanter)
-
- YARN-4718. Rename variables in SchedulerNode to reduce ambiguity post
- YARN-1011. (Inigo Goiri via kasha)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- YARN-4109. Exception on RM scheduler page loading with labels.
- (Mohammad Shahid Khan via rohithsharmaks)
-
- YARN-4156. TestAMRestart#testAMBlacklistPreventsRestartOnSameNode
- assumes CapacityScheduler. (Anubhav Dhoot via kasha)
-
- YARN-2934. Improve handling of container's stderr.
- (Naganarasimha G R via gera)
-
- YARN-4530. LocalizedResource trigger a NPE Cause the NodeManager exit
- (tangshangwen via rohithsharmaks)
-
- YARN-4535. Fix checkstyle error in CapacityScheduler.java
- (Naganarasimha G R via rohithsharmaks)
-
- YARN-4393. Fix intermittent test failure for TestResourceLocalizationService
- #testFailedDirsResourceRelease (Varun Saxana via rohithsharmaks)
-
- YARN-4550. Some tests in TestContainerLanch fails on non-english locale
- environment. (Takashi Ohnishi via stevel)
-
- YARN-4567. javadoc failing on java 8. (Steve Loughran via aajisaka)
-
- YARN-3446. FairScheduler headroom calculation should exclude nodes in the
- blacklist. (Zhihai Xu via kasha)
-
- YARN-4611. Fix scheduler load simulator to support multi-layer network
- location. (Ming Ma via xgong)
-
- YARN-4584. RM startup failure when AM attempts greater than max-attempts.
- (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4578. Directories that are mounted in docker containers need to be more
- restrictive/container-specific. (Sidharta Seethana via vvasudev)
-
- YARN-4497. RM might fail to restart when recovering apps whose attempts are missing.
- (Jun Gong via rohithsharmaks)
-
- YARN-4574. Fix random failure in TestAMRMClientOnRMRestart.
- (Takashi Ohnishi via rohithsharmaks)
-
- YARN-4613. Fix test failure in TestClientRMService#testGetClusterNodes.
- (Takashi Ohnishi via rohithsharmaks)
-
- YARN-4612. Fix rumen and scheduler load simulator handle killed tasks properly.
- (Ming Ma via xgong)
-
- YARN-4573. Fix test failure in TestRMAppTransitions#testAppRunningKill and
- testAppKilledKilled. (Takashi Ohnishi via rohithsharmaks)
-
- YARN-4633. Fix random test failure in TestRMRestart#testRMRestartAfterPreemption
- (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant
- (Akihiro Suda via rohithsharmaks)
-
- YARN-4615. Fix random test failure in TestAbstractYarnScheduler#testResource
- RequestRecoveryToTheRightAppAttempt. (Sunil G via rohithsharmaks)
-
- YARN-4594. container-executor fails to remove directory tree when chmod
- required (Colin Patrick McCabe via jlowe)
-
- YARN-4669. Fix logging statements in resource manager's Application class.
- (Seethana Sidharta via vvasudev)
-
- YARN-4629. Distributed shell breaks under strong security.
- (Daniel Templeton via stevel)
-
- YARN-4684. TestYarnCLI#testGetContainers failing in CN locale.
- (Bibin A Chundatt via vvasudev)
-
- YARN-4569. Remove incorrect part of maxResources in FairScheduler
- documentation. (Ray Chiang via kasha)
-
- YARN-4651. Document movetoqueue option in 'YARN Commands'
- (Takashi Ohnishi via rohithsharmaks)
-
- YARN-4729. SchedulerApplicationAttempt#getTotalRequiredResources can throw
- an NPE. (kasha)
-
- YARN-4701. When task logs are not available, port 8041 is referenced
- instead of port 8042 (haibochen via rkanter)
-
- YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization.
- (Takashi Ohnishi via rohithsharmaks)
-
- YARN-4731. container-executor should not follow symlinks in
- recursive_unlink_children (Colin Patrick McCabe via jlowe)
-
- YARN-4704. TestResourceManager#testResourceAllocation() fails when using
- FairScheduler. (Yufei Gu via kasha)
-
-Release 2.8.0 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- YARN-2336. Fair scheduler's REST API returns a missing '[' bracket JSON for
- deep queue tree. (Kenji Kikushima and Akira Ajisaka via ozawa)
-
- NEW FEATURES
-
- YARN-3360. Add JMX metrics to TimelineDataManager (Jason Lowe via jeagles)
-
- YARN-3345. Add non-exclusive node label API. (Wangda Tan via jianhe)
-
- YARN-3365. Enhanced NodeManager to support using the 'tc' tool via
- container-executor for outbound network traffic control. (Sidharta Seethana
- via vinodkv)
-
- YARN-1376. NM need to notify the log aggregation status to RM through
- heartbeat. (Xuan Gong via junping_du)
-
- YARN-3348. Add a 'yarn top' tool to help understand cluster usage. (Varun
- Vasudev via jianhe)
-
- YARN-3347. Improve YARN log command to get AMContainer logs as well as
- running containers logs. (Xuan Gong via junping_du)
-
- YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support
- for new resource types on the NM. (Sidharta Seethana via junping_du)
-
- YARN-3361. CapacityScheduler side changes to support non-exclusive node
- labels. (Wangda Tan via jianhe)
-
- YARN-3318. Create Initial OrderingPolicy Framework and FifoOrderingPolicy.
- (Craig Welch via wangda)
-
- YARN-3326. Support RESTful API for getLabelsToNodes. (Naganarasimha G R
- via ozawa)
-
- YARN-3354. Add node label expression in ContainerTokenIdentifier to support
- RM recovery. (Wangda Tan via jianhe)
-
- YARN-1402. Update related Web UI and CLI with exposing client API to check
- log aggregation status. (Xuan Gong via junping_du)
-
- YARN-3463. Integrate OrderingPolicy Framework with CapacityScheduler.
- (Craig Welch via wangda)
-
- YARN-3410. YARN admin should be able to remove individual application
- records from RMStateStore. (Rohith Sharmaks via wangda)
-
- YARN-3225. New parameter of CLI for decommissioning node gracefully in
- RMAdmin CLI. (Devaraj K via junping_du)
-
- YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing
- network bandwidth traffic originating from YARN containers (Sidharta Seethana
- via vinodkv)
-
- YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda)
-
- YARN-2498. Respect labels in preemption policy of capacity scheduler for
- inter-queue preemption. (Wangda Tan via jianhe)
-
- YARN-2619. Added NodeManager support for disk io isolation through cgroups.
- (Varun Vasudev and Wei Yan via vinodkv)
-
- YARN-3448. Added a rolling time-to-live LevelDB timeline store implementation.
- (Jonathan Eagles via zjshen)
-
- YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in
- RMApps. (Xuan Gong via junping_du)
-
- YARN-3541. Add version info on timeline service / generic history web UI
- and REST API. (Zhijie Shen via xgong)
-
- YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from
- underlying OS when configured to do so. (Varun Vasudev via vinodkv)
-
- YARN-41. The RM should handle the graceful shutdown of the NM. (Devaraj K via
- junping_du)
-
- YARN-1012. Report NM aggregated container resource utilization in heartbeat.
- (Inigo Goiri via kasha)
-
- YARN-3116. RM notifies NM whether a container is an AM container or normal
- task container. (Giovanni Matteo Fumarola via zjshen)
-
- YARN-2003. Support for Application priority : Changes in RM and Capacity
- Scheduler. (Sunil G via wangda)
-
- YARN-3656. LowCost: A Cost-Based Placement Agent for YARN Reservations.
- (Jonathan Yaniv and Ishai Menache via curino)
-
- YARN-3852. Add docker container support to container-executor
- (Abin Shahab via vvasudev)
-
- YARN-3853. Add docker container runtime support to LinuxContainterExecutor.
- (Sidharta Seethana via vvasudev)
-
- YARN-3736. Add RMStateStore apis to store and load accepted reservations for
- failover (adhoot via asuresh)
-
- YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks)
-
- YARN-3873. PendingApplications in LeafQueue should also use OrderingPolicy.
- (Sunil G via wangda)
-
- YARN-3887. Support changing Application priority during runtime. (Sunil G
- via jianhe)
-
- YARN-4023. Publish Application Priority to TimelineServer. (Sunil G
- via rohithsharmaks)
-
- YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha)
-
- YARN-4055. Report node resource utilization in heartbeat.
- (Inigo Goiri via kasha)
-
- YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed
- Node Label Configuration Setup. (Naganarasimha G R)
-
- YARN-221. NM should provide a way for AM to tell it not to aggregate logs.
- (Ming Ma via xgong)
-
- YARN-4014. Support user cli interface in for Application Priority.
- (Rohith Sharma K S via jianhe)
-
- YARN-3250. Support admin cli interface in for Application Priority.
- (Rohith Sharma K S via jianhe)
-
- YARN-3970. Add REST api support for Application Priority.
- (Naganarasimha G R via vvasudev)
-
- YARN-2884. Added a proxy service in NM to proxy the the communication
- between AM and RM. (Kishore Chaliparambil via jianhe)
-
- YARN-313. Add Admin API for supporting node resource configuration in
- command line. (Inigo Goiri, Kenji Kikushima and Junping Du
- via junping_du)
-
- YARN-4034. Render cluster Max Priority in scheduler metrics in RM web
- UI. (Rohith Sharma K S via jianhe)
-
- YARN-3212. RMNode State Transition Update with DECOMMISSIONING state.
- (Junping Du via wangda)
-
- YARN-3866. AM-RM protocol changes to support container resizing. (Meng Ding
- via jianhe)
-
- YARN-1449. AM-NM protocol changes to support container resizing.
- (Meng Ding & Wangda Tan via jianhe)
-
- YARN-1645. ContainerManager implementation to support container resizing.
- (Meng Ding & Wangda Tan via jianhe)
-
- YARN-3867. ContainerImpl changes to support container resizing. (Meng Ding
- via jianhe)
-
- YARN-1643. Make ContainersMonitor support changing monitoring size of an
- allocated container. (Meng Ding and Wangda Tan)
-
- YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to
- support container resizing. (Meng Ding via jianhe)
-
- YARN-3868. Recovery support for container resizing. (Meng Ding via jianhe)
-
- YARN-1651. CapacityScheduler side changes to support container resize.
- (Wangda Tan via jianhe)
-
- YARN-1897. CLI and core support for signal container functionality.
- (Ming Ma via xgong)
-
- YARN-261. Ability to fail AM attempts (Andrey Klochkov and
- Rohith Sharma K S via jlowe)
-
- YARN-3964. Support NodeLabelsProvider at Resource Manager side.
- (Dian Fu via devaraj)
-
- YARN-2556. Tool to measure the performance of the timeline server (Chang Li
- via sjlee)
-
- YARN-4262. Allow whitelisted users to run privileged docker containers.
- (Sidharta Seethana via vvasudev)
-
- YARN-3739. Add reservation system recovery to RM recovery process.
- (Subru Krishnan via adhoot)
-
- YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label
- Configuration Setup. (Naganarasimha G R via rohithsharmaks)
-
- YARN-1510. Make NMClient support change container resources.
- (Meng Ding via wangda)
-
- YARN-1509. Make AMRMClient support send increase container request and
- get increased/decreased containers. (Meng Ding via wangda)
-
- YARN-4184. Remove update reservation state api from state store as its not used by
- ReservationSystem (Sean Po via asuresh)
-
- YARN-4349. Support CallerContext in YARN. (wtan via jianhe)
-
- YARN-3623. Add a new config to indicate the Timeline Service version.
- (Xuan Gong via junping_du)
-
- YARN-3226. UI changes for decommissioning node. (Sunil G via
- junping_du)
-
- YARN-3458. CPU resource monitoring in Windows. (Inigo Goiri via cnauroth)
-
- YARN-4234. New put APIs in TimelineClient for ats v1.5. (Xuan Gong via
- junping_du)
-
- YARN-4265. Provide new timeline plugin storage to support fine-grained entity
- caching. (Li Lu and Jason Lowe via junping_du)
-
- YARN-4219. New levelDB cache storage for timeline v1.5. (Li Lu via xgong)
-
- YARN-4411. RMAppAttemptImpl#createApplicationAttemptReport throws
- IllegalArgumentException. (Bibin A Chundatt, yarntime via devaraj)
-
- YARN-3223. Resource update during NM graceful decommission. (Brook Zhou
- via junping_du)
-
- IMPROVEMENTS
-
- YARN-644. Basic null check is not performed on passed in arguments before
- using them in ContainerManagerImpl.startContainer (Varun Saxena via bobby)
-
- YARN-1880. Cleanup TestApplicationClientProtocolOnHA
- (ozawa via harsh)
-
- YARN-3357. Move TestFifoScheduler to FIFO package. (Rohith Sharmaks
- via devaraj)
-
- YARN-3356. Capacity Scheduler FiCaSchedulerApp should use ResourceUsage to
- track used-resources-by-label. (Wangda Tan via jianhe)
-
- YARN-3350. YARN RackResolver spams logs with messages at info level.
- (Wilfred Spiegelenburg via junping_du)
-
- YARN-2868. FairScheduler: Metric for latency to allocate first container
- for an application. (Ray Chiang via kasha)
-
- YARN-3397. yarn rmadmin should skip -failover. (J.Andreina via kasha)
-
- YARN-3288. Document and fix indentation in the DockerContainerExecutor code
-
- YARN-2495. Allow admin specify labels from each NM (Distributed
- configuration for node label). (Naganarasimha G R via wangda)
-
- YARN-3258. FairScheduler: Need to add more logging to investigate
- allocations. (Anubhav Dhoot via ozawa)
-
- YARN-3428. Debug log resources to be localized for a container. (kasha)
-
- YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring
- from info to debug. (Anubhav Dhoot via ozawa)
-
- YARN-2901. Add errors and warning metrics page to RM, NM web UI.
- (Varun Vasudev via wangda)
-
- YARN-3294. Allow dumping of Capacity Scheduler debug logs via
- web UI for a fixed time period. (Varun Vasudev via xgong)
-
- YARN-3293. Track and display capacity scheduler health metrics
- in web UI. (Varun Vasudev via xgong)
-
- YARN-3394. Enrich WebApplication proxy documentation. (Naganarasimha G R
- via jianhe)
-
- YARN-3404. Display queue name on application page. (Ryu Kobayashi via jianhe)
-
- YARN-2696. Queue sorting in CapacityScheduler should consider node label.
- (Wangda Tan via jianhe)
-
- YARN-3451. Display attempt start time and elapsed time on the web UI.
- (Rohith Sharmaks via jianhe)
-
- YARN-3494. Expose AM resource limit and usage in CS QueueMetrics. (Rohith
- Sharmaks via jianhe)
-
- YARN-3503. Expose disk utilization percentage and bad local and log dir
- counts in NM metrics. (Varun Vasudev via jianhe)
-
- YARN-3511. Add errors and warnings page to ATS. (Varun Vasudev via xgong)
-
- YARN-3406. Display count of running containers in the RM's Web UI.
- (Ryu Kobayashi via ozawa)
-
- YARN-3363. add localization and container launch time to ContainerMetrics
- at NM to show these timing information for each active container.
- (zxu via rkanter)
-
- YARN-3396. Handle URISyntaxException in ResourceLocalizationService.
- (Brahma Reddy Battula via junping_du)
-
- YARN-3491. PublicLocalizer#addResource is too slow. (zxu via rkanter)
-
- YARN-3523. Cleanup ResourceManagerAdministrationProtocol interface audience.
- (Naganarasimha G R via junping_du)
-
- YARN-3169. Drop YARN's overview document. (Brahma Reddy Battula via ozawa)
-
- YARN-2784. Make POM project names consistent. (Rohith via devaraj)
-
- YARN-20. More information for "yarn.resourcemanager.webapp.address" in
- yarn-default.xml (Bartosz Ługowski vai tgraves)
-
- YARN-3593. Add label-type and Improve "DEFAULT_PARTITION" in Node Labels Page.
- (Naganarasimha G R via wangda)
-
- YARN-3018. Unified the default value for the configuration property
- yarn.scheduler.capacity.node-locality-delay in code and default xml file.
- (Nijel SF via vinodkv)
-
- YARN-2331. Distinguish shutdown during supervision vs. shutdown for
- rolling upgrade. (Jason Lowe via xgong)
-
- YARN-1050. Document the Fair Scheduler REST API.
- (Kenji Kikushima and Roman Shaposhnik via kasha)
-
- YARN-3271. FairScheduler: Move tests related to max-runnable-apps from
- TestFairScheduler to TestAppRunnability. (nijel via kasha)
-
- YARN-1912. ResourceLocalizer started without any jvm memory control.
- (Masatake Iwasaki via xgong)
-
- YARN-1287. Consolidate MockClocks.
- (Sebastian Wong and Anubhav Dhoot via kasha)
-
- YARN-3395. FairScheduler: Trim whitespaces when using username for
- queuename. (Zhihai Xu via kasha)
-
- YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc.
- projects. (Gabor Liptak via junping_du)
-
- YARN-3513. Remove unused variables in ContainersMonitorImpl and add debug
- log for overall resource usage by all containers. (Naganarasimha G R via devaraj)
-
- YARN-3613. TestContainerManagerSecurity should init and start Yarn cluster in
- setup instead of individual methods. (nijel via kasha)
-
- YARN-3579. CommonNodeLabelsManager should support NodeLabel instead of string
- label name when getting node-to-label/label-to-label mappings. (Sunil G via wangda)
-
- YARN-3521. Support return structured NodeLabel objects in REST API (Sunil G via wangda)
-
- YARN-3362. Add node label usage in RM CapacityScheduler web UI.
- (Naganarasimha G R via wangda)
-
- YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use
- NodeLabel object instead of String. (Naganarasimha G R via wangda)
-
- YARN-3583. Support of NodeLabel object instead of plain String
- in YarnClient side. (Sunil G via wangda)
-
- YARN-3684. Changed ContainerExecutor's primary lifecycle methods to use a more
- extensible mechanism of context objects. (Sidharta Seethana via vinodkv)
-
- YARN-3594. WintuilsProcessStubExecutor.startStreamReader leaks streams.
- (Lars Francke via junping_du)
-
- YARN-3647. RMWebServices api's should use updated api from CommonNodeLabelsManager
- to get NodeLabel object. (Sunil G via wangda)
-
- YARN-3581. Deprecate -directlyAccessNodeLabelStore in RMAdminCLI.
- (Naganarasimha G R via wangda)
-
- YARN-3722. Merge multiple TestWebAppUtils into o.a.h.yarn.webapp.util.TestWebAppUtils.
- (Masatake Iwasaki via devaraj)
-
- YARN-3713. Remove duplicate function call storeContainerDiagnostics in
- ContainerDiagnosticsUpdateTransition (zxu via rkanter)
-
- YARN-3467. Expose allocatedMB, allocatedVCores, and runningContainers metrics on
- running Applications in RM Web UI. (Anubhav Dhoot via kasha)
-
- YARN-2392. Add more diags about app retry limits on AM failures. (Steve
- Loughran via jianhe)
-
- YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. (xgong)
-
- YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator.
- (Karthik Kambatla via jianhe)
-
- YARN-3786. Document yarn class path options.
- (Brahma Reddy Battula via cnauroth)
-
- YARN-3787. Allowed generic history service to load a number of applications whose
- started time is within the given range. (Xuan Gong via zjshen)
-
- YARN-3785. Support for Resource as an argument during submitApp call in MockRM
- test class. (Sunil G via xgong)
-
- YARN-3789. Improve logs for LeafQueue#activateApplications().
- (Bibin A Chundatt via devaraj)
-
- YARN-3148. Allow CORS related headers to passthrough in WebAppProxyServlet.
- (Varun Saxena via devaraj)
-
- YARN-3834. Scrub debug logging of tokens during resource localization.
- (Chris Nauroth via xgong)
-
- YARN-3827. Migrate YARN native build to new CMake framework (Alan Burlison
- via Colin P. McCabe)
-
- YARN-3800. Reduce storage footprint for ReservationAllocation. (Anubhav Dhoot
- via curino)
-
- YARN-3069. Document missing properties in yarn-default.xml.
- (Ray Chiang via aajisaka)
-
- YARN-3844. Make hadoop-yarn-project Native code -Wall-clean (Alan Burlison
- via Colin P. McCabe)
-
- YARN-3026. Move application-specific container allocation logic from
- LeafQueue to FiCaSchedulerApp. (Wangda Tan via jianhe)
-
- YARN-3950. Add unique SHELL_ID environment variable to DistributedShell
- (Robert Kanter via jlowe)
-
- YARN-3965. Add startup timestamp to nodemanager UI (Hong Zhiguo via jlowe)
-
- YARN-3543. ApplicationReport should be able to tell whether the Application
- is AM managed or not. (Rohith Sharma K S via xgong)
-
- YARN-4004. container-executor should print output of docker logs if the docker
- container exits with non-0 exit status. (Varun Vasudev via xgong)
-
- YARN-3961. Expose pending, running and reserved containers of a queue in REST
- api and yarn top (adhoot via asuresh)
-
- YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. (Robert Kanter
- via junping_du)
-
- YARN-3974. Refactor the reservation system test cases to use parameterized
- base test. (subru via curino)
-
- YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda)
-
- YARN-4026. Refactored ContainerAllocator to accept a list of priorites
- rather than a single priority. (Wangda Tan via jianhe)
-
- YARN-4031. Add JvmPauseMonitor to ApplicationHistoryServer and
- WebAppProxyServer (djp via rkanter)
-
- YARN-4057. If ContainersMonitor is not enabled, only print
- related log info one time. (Jun Gong via zxu)
-
- YARN-1556. NPE getting application report with a null appId. (Weiwei Yang via
- junping_du)
-
- YARN-4121. Fix typos in capacity scheduler documentation.
- (Kai Sasaki via vvasudev)
-
- YARN-4086. Allow Aggregated Log readers to handle HAR files (rkanter)
-
- YARN-4145. Make RMHATestBase abstract so its not run when running all
- tests under that namespace (adhoot via rkanter)
-
- YARN-2005. Blacklisting support for scheduling AMs. (Anubhav Dhoot via kasha)
-
- YARN-3717. Expose app/am/queue's node-label-expression to RM web UI /
- CLI / REST-API. (Naganarasimha G R via wangda)
-
- YARN-4149. yarn logs -am should provide an option to fetch all the log files
- (Varun Vasudev via xgong)
-
- YARN-2597. MiniYARNCluster should propagate reason for AHS not starting.
- (stevel)
-
- MAPREDUCE-6478. Add an option to skip cleanupJob stage or ignore cleanup
- failure during commitJob. (Junping Du via wangda)
-
- YARN-3920. FairScheduler container reservation on a node should be
- configurable to limit it to large containers (adhoot via asuresh)
-
- HADOOP-12428. Fix inconsistency between log-level guards and statements.
- (Jagadesh Kiran N and Jackie Chang via ozawa)
-
- YARN-4095. Avoid sharing AllocatorPerContext object in LocalDirAllocator
- between ShuffleHandler and LocalDirsHandlerService. (Zhihai Xu via jlowe)
-
- YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels.
- (Bibin A Chundatt via wangda)
-
- YARN-4215. RMNodeLabels Manager Need to verify and replace node labels for the
- only modified Node Label Mappings in the request. (Naganarasimha G R via wangda)
-
- YARN-3943. Use separate threshold configurations for disk-full detection
- and disk-not-full detection. (Zhihai Xu via jlowe)
-
- YARN-4252. Log container-executor invocation details when exit code is non-zero.
- (Sidharta Seethana via vvasudev)
-
- YARN-4258. Add support for controlling capabilities for docker containers.
- (Sidharta Seethana via vvasudev)
-
- YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity
- by partition to REST API. (Naganarasimha G R via wangda)
-
- YARN-4170. AM need to be notified with priority in AllocateResponse.
- (Sunil G via jianhe)
-
- YARN-4267. Add additional logging to container launch implementations in
- container-executor. (Sidharta Seethana via vvasudev)
-
- YARN-3985. Make ReservationSystem persist state using RMStateStore
- reservation APIs. (adhoot via asuresh)
-
- YARN-4243. Add retry on establishing Zookeeper conenction in
- EmbeddedElectorService#serviceInit. (Xuan Gong via junping_du)
-
- YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for
- each queue. (Siqi Li via mingma)
-
- YARN-4296. DistributedShell Log.info is not friendly.
- (Xiaowei Wang via stevel)
-
- YARN-3738. Add support for recovery of reserved apps running under dynamic
- queues (subru via asuresh)
-
- YARN-4285. Display resource usage as percentage of queue and cluster in the
- RM UI (Varun Vasudev via wangda)
-
- YARN-3216. Max-AM-Resource-Percentage should respect node labels.
- (Sunil G via wangda)
-
- YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh)
-
- YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public,
- @Unstable. (stevel)
-
- YARN-3454. Add efficient merge operation to RLESparseResourceAllocation
- (Carlo Curino via asuresh)
-
- YARN-3980. Plumb resource-utilization info in node heartbeat through to the
- scheduler. (Inigo Goiri via kasha)
-
- YARN-4132. Separate configs for nodemanager to resourcemanager connection
- timeout and retries (Chang Li via jlowe)
-
- YARN-4292. ResourceUtilization should be a part of NodeInfo REST API.
- (Sunil G via wangda)
-
- YARN-4405. Support node label store in non-appendable file system. (Wangda
- Tan via jianhe)
-
- YARN-4358. Reservation System: Improve relationship between SharingPolicy
- and ReservationAgent. (Carlo Curino via asuresh)
-
- YARN-3456. Improve handling of incomplete TimelineEntities. (Varun Saxena
- via rohithsharmaks)
-
- YARN-4248. REST API for submit/update/delete Reservations. (curino)
-
- YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to
- app's diagnostic message. (Naganarasimha G R via wangda)
-
- YARN-4309. Add container launch related debug information to container logs
- when a container fails. (Varun Vasudev via wangda)
-
- YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well.
- (Sunil G via wangda)
-
- YARN-4207. Add a non-judgemental YARN app completion status. (Rich Haase via sseth)
-
- YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda)
-
- YARN-4164. Changed updateApplicationPriority API to return the updated
- application priority. (Rohith Sharma K S via jianhe)
-
- YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh)
-
- YARN-4290. Add -showDetails option to YARN Nodes CLI to print all nodes reports
- information. (Sunil G via wangda)
-
- YARN-4098. Document ApplicationPriority feature. (Rohith Sharma K S via jianhe)
-
- YARN-4544. All the log messages about rolling monitoring interval are
- shown with WARN level. (Takashi Ohnishi via aajisaka)
-
- YARN-4537. Pull out priority comparison from fifocomparator and use compound
- comparator for FifoOrdering policy. (Rohith Sharma K S via jianhe)
-
- YARN-4582. Label-related invalid resource request exception should be able to
- properly handled by application. (Bibin A Chundatt via wangda)
-
- YARN-4304. AM max resource configuration per partition to be displayed/updated
- correctly in UI and in various partition related metrics. (Sunil G via wangda)
-
- YARN-4524. Cleanup AppSchedulingInfo. (Karthik Kambatla via wangda)
-
- YARN-4371. "yarn application -kill" should take multiple application ids
- (Sunil G via jlowe)
-
- YARN-4340. Add "list" API to reservation system. (Sean Po via wangda)
-
- YARN-4100. Add Documentation for Distributed and Delegated-Centralized
- Node Labels feature. (Naganarasimha G R via devaraj)
-
- YARN-4662. Document some newly added metrics. (Jian He via xgong)
-
- YARN-4360. Improve GreedyReservationAgent to support "early" allocations,
- and performance improvements (curino via asuresh)
-
- YARN-4420. Add REST API for List Reservations. (Sean Po via curino)
-
- YARN-4519. Potential deadlock of CapacityScheduler between decrease container
- and assign containers. (Meng Ding via jianhe)
-
- YARN-4138. Roll back container resource allocation after resource
- increase token expires. (Meng Ding via jianhe)
-
- YARN-2575. Create separate ACLs for Reservation create/update/delete/list
- ops (Sean Po via asuresh)
-
- YARN-4682. AMRM client to log when AMRM token updated.
- (Prabhu Joseph via stevel)
-
- YARN-4579. Allow DefaultContainerExecutor container log directory
- permissions to be configurable (rchiang via rkanter)
-
- OPTIMIZATIONS
-
- YARN-3339. TestDockerContainerExecutor should pull a single image and not
- the entire centos repository. (Ravindra Kumar Naik via raviprak)
-
- YARN-3469. ZKRMStateStore: Avoid setting watches that are not required.
- (Jun Gong via kasha)
-
- YARN-3006. Improve the error message when attempting manual failover with
- auto-failover enabled. (Akira AJISAKA via wangda)
-
- YARN-3547. FairScheduler: Apps that have no resource demand should not participate
- scheduling. (Xianyin Xin via kasha)
-
- YARN-3259. FairScheduler: Trigger fairShare updates on node events.
- (Anubhav Dhoot via kasha)
-
- YARN-2768. Avoid cloning Resource in FSAppAttempt#updateDemand.
- (Hong Zhiguo via kasha)
-
- YARN-3983. Refactored CapacityScheduleri#FiCaSchedulerApp to easier extend
- container allocation logic. (Wangda Tan via jianhe)
-
- YARN-3635. Refactored current queue mapping implementation in CapacityScheduler
- to use a generic PlacementManager framework. (Wangda Tan via jianhe)
-
- YARN-4066. Large number of queues choke fair scheduler.
- (Johan Gustavsson via kasha)
-
- YARN-4720. Skip unnecessary NN operations in log aggregation.
- (Jun Gong via mingma)
-
- BUG FIXES
-
- YARN-3197. Confusing log generated by CapacityScheduler. (Varun Saxena
- via devaraj)
-
- YARN-3305. Normalize AM resource request on app submission. (Rohith Sharmaks
- via jianhe)
-
- YARN-3205 FileSystemRMStateStore should disable FileSystem Cache to avoid
- get a Filesystem with an old configuration. (Zhihai Xu via ozawa)
-
- YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to
- fully qualified path. (Xuan Gong via junping_du)
-
- YARN-3241. FairScheduler handles "invalid" queue names inconsistently.
- (Zhihai Xu via kasha)
-
- YARN-3383. AdminService should use "warn" instead of "info" to log exception
- when operation fails. (Li Lu via wangda)
-
- YARN-3400. [JDK 8] Build Failure due to unreported exceptions in
- RPCUtil (rkanter)
-
- YARN-3412. RM tests should use MockRM where possible. (kasha)
-
- YARN-3425. NPE from RMNodeLabelsManager.serviceStop when
- NodeLabelsManager.serviceInit failed. (Bibin A Chundatt via wangda)
-
- YARN-3415. Non-AM containers can be counted towards amResourceUsage of a
- Fair Scheduler queue (Zhihai Xu via Sandy Ryza)
-
- YARN-3435. AM container to be allocated Appattempt AM container shown as null.
- (Bibin A Chundatt via xgong)
-
- YARN-2666. TestFairScheduler.testContinuousScheduling fails Intermittently.
- (Zhihai Xu via ozawa)
-
- YARN-3429. TestAMRMTokens.testTokenExpiry fails Intermittently with
- error message:Invalid AMRMToken (zxu via rkanter)
-
- YARN-3110. Few issues in ApplicationHistory web ui. (Naganarasimha G R via xgong)
-
- YARN-3459. Fix failiure of TestLog4jWarningErrorMetricsAppender.
- (Varun Vasudev via wangda)
-
- YARN-3266. RMContext#inactiveNodes should have NodeId as map key.
- (Chengbing Liu via jianhe)
-
- YARN-3436. Fix URIs in documantion of YARN web service REST APIs.
- (Bibin A Chundatt via ozawa)
-
- YARN-3021. YARN's delegation-token handling disallows certain trust setups
- to operate properly over DistCp. (Yongjun Zhang via jianhe)
-
- YARN-3136. Fixed a synchronization problem of
- AbstractYarnScheduler#getTransferredContainers. (Sunil G via jianhe)
-
- YARN-3495. Confusing log generated by FairScheduler.
- (Brahma Reddy Battula via ozawa)
-
- YARN-3387. Previous AM's container completed status couldn't pass to current
- AM if AM and RM restarted during the same time. (sandflee via jianhe)
-
- YARN-3444. Fix typo capabililty. (Gabor Liptak via aajisaka)
-
- YARN-3530. ATS throws exception on trying to filter results without otherinfo.
- (zhijie shen via xgong)
-
- YARN-2740. Fix NodeLabelsManager to properly handle node label modifications
- when distributed node label configuration enabled. (Naganarasimha G R via wangda)
-
- YARN-3517. RM web ui for dumping scheduler logs should be for admins only
- (Varun Vasudev via tgraves)
-
- YARN-3533. Test: Fix launchAM in MockRM to wait for attempt to be scheduled.
- (Anubhav Dhoot via jianhe)
-
- YARN-3564. Fix TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable
- fails randomly. (Jian He via wangda)
-
- YARN-2893. AMLaucher: sporadic job failures due to EOFException in
- readTokenStorageStream. (Zhihai Xu via gera)
-
- YARN-2454. Fix compareTo of variable UNBOUNDED in o.a.h.y.util.resource.Resources.
- (Xu Yang via junping_du)
-
- YARN-1993. Cross-site scripting vulnerability in TextView.java. (Kenji Kikushima
- via ozawa)
-
- YARN-3097. Logging of resource recovery on NM restart has redundancies
- (Eric Payne via jlowe)
-
- YARN-3375. NodeHealthScriptRunner.shouldRun() check is performing 3 times for
- starting NodeHealthScriptRunner. (Devaraj K via wangda)
-
- YARN-2725. Added test cases of retrying creating znode in ZKRMStateStore.
- (Tsuyoshi Ozawa via jianhe)
-
- YARN-3552. RM Web UI shows -1 running containers for completed apps
- (Rohith via jlowe)
-
- YARN-2123. Progress bars in Web UI always at 100% due to non-US locale.
- (Akira AJISAKA via xgong)
-
- YARN-3343. Increased TestCapacitySchedulerNodeLabelUpdate#testNodeUpdate
- timeout. (Rohith Sharmaks via jianhe)
-
- YARN-3582. NPE in WebAppProxyServlet. (jian he via xgong)
-
- YARN-3577. Misspelling of threshold in log4j.properties for tests.
- (Brahma Reddy Battula via aajisaka)
-
- YARN-3584. Fixed attempt diagnostics format shown on the UI. (nijel via
- jianhe)
-
- YARN-1832. Fix wrong MockLocalizerStatus#equals implementation.
- (Hong Zhiguo via aajisaka)
-
- YARN-3572. Correct typos in WritingYarnApplications.md.
- (Gabor Liptak via aajisaka)
-
- YARN-3592. Fix typos in RMNodeLabelsManager. (Sunil G via devaraj)
-
- YARN-3589. RM and AH web UI display DOCTYPE wrongly. (Rohith via ozawa)
-
- YARN-3600. AM container link is broken (Naganarasimha G R via tgraves)
-
- YARN-3604. Fixed ZKRMStateStore#removeApplication to also disable watch.
- (zhihai xu via jianhe)
-
- YARN-3473. Fix RM Web UI configuration for some properties (rchiang via rkanter)
-
- YARN-2206. Updated document for applications REST API response examples. (Kenji
- Kikushima and Brahma Reddy Battula via zjshen)
-
- YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir
- fails Intermittently due to IOException from cleanup. (zhihai xu via xgong)
-
- YARN-3629. NodeID is always printed as "null" in node manager initialization log.
- (nijel via devaraj)
-
- YARN-2921. Fix MockRM/MockAM#waitForState sleep too long.
- (Tsuyoshi Ozawa via wangda)
-
- YARN-1519. Check in container-executor if sysconf is implemented before
- using it (Radim Kolar and Eric Payne via raviprak)
-
- YARN-2421. RM still allocates containers to an app in the FINISHING
- state (Chang Li via jlowe)
-
- YARN-3302. TestDockerContainerExecutor should run automatically if it can
- detect docker in the usual place (Ravindra Kumar Naik via raviprak)
-
- YARN-2821. Fixed a problem that DistributedShell AM may hang if restarted.
- (Varun Vasudev via jianhe)
-
- YARN-3654. ContainerLogsPage web UI should not have meta-refresh. (Xuan Gong
- via jianhe)
-
- YARN-3707. RM Web UI queue filter doesn't work. (Wangda Tan via jianhe)
-
- YARN-3632. Ordering policy should be allowed to reorder an application when
- demand changes. (Craig Welch via jianhe)
-
- YARN-3716. Node-label-expression should be included by
- ResourceRequestPBImpl.toString. (Xianyin Xin via wangda)
-
- YARN-3751. Fixed AppInfo to check if used resources are null. (Sunil G via
- zjshen)
-
- YARN-3762. FairScheduler: CME on FSParentQueue#getQueueUserAclInfo. (kasha)
-
- YARN-3749. We should make a copy of configuration when init MiniYARNCluster
- with multiple RMs. (Chun Chen via xgong)
-
- YARN-3766. Fixed the apps table column error of generic history web UI.
- (Xuan Gong via zjshen)
-
- YARN-3655. FairScheduler: potential livelock due to maxAMShare limitation
- and container reservation. (Zhihai Xu via kasha)
-
- YARN-3747. TestLocalDirsHandlerService should delete the created test directory logDir2.
- (David Moore via devaraj)
-
- YARN-3778. Fix Yarn resourcemanger CLI usage. (Brahma Reddy Battula via xgong)
-
- YARN-3794. TestRMEmbeddedElector fails because of ambiguous LOG reference.
- (Chengbing Liu via devaraj)
-
- YARN-3714. AM proxy filter can not get RM webapp address from
- yarn.resourcemanager.hostname.rm-id. (Masatake Iwasaki via xgong)
-
- YARN-3617. Fix WindowsResourceCalculatorPlugin.getCpuFrequency()
- returning always -1. (J.Andreina via devaraj)
-
- YARN-3824. Fix two minor nits in member variable properties
- of YarnConfiguration. (Ray Chiang via devaraj)
-
- YARN-3835. hadoop-yarn-server-resourcemanager test package bundles
- core-site.xml, yarn-site.xml (vamsee via rkanter)
-
- YARN-3790. usedResource from rootQueue metrics may get stale data for FS
- scheduler after recovering the container (Zhihai Xu via rohithsharmaks)
-
- YARN-3826. Race condition in ResourceTrackerService leads to
- wrong diagnostics messages. (Chengbing Liu via devaraj)
-
- YARN-3745. SerializedException should also try to instantiate internal
- exception with the default constructor. (Lavkesh Lahngir via devaraj)
-
- YARN-2871. TestRMRestart#testRMRestartGetApplicationList sometime fails in trunk.
- (zhihai xu via xgong)
-
- YARN-3859. LeafQueue doesn't print user properly for application add.
- (Varun Saxena via devaraj)
-
- YARN-3860. rmadmin -transitionToActive should check the state of non-target node.
- (Masatake Iwasaki via junping_du)
-
- YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non
- network exception. (Raju Bairishetti via jianhe)
-
- YARN-3770. SerializedException should also handle java.lang.Error on
- de-serialization. (Lavkesh Lahngir via jianhe)
-
- YARN-3768. ArrayIndexOutOfBoundsException with empty environment variables.
- (Zhihai Xu via gera)
-
- YARN-3823. Fix mismatch in default values for
- yarn.scheduler.maximum-allocation-vcores property. (Ray Chiang via devaraj)
-
- YARN-3830. AbstractYarnScheduler.createReleaseCache may try to clean a null
- attempt. (nijel via devaraj)
-
- YARN-3875. FSSchedulerNode#reserveResource() doesn't print Application Id
- properly in log. (Bibin A Chundatt via devaraj)
-
- YARN-3882. AggregatedLogFormat should close aclScanner and ownerScanner
- after create them. (zhihai xu via xgong)
-
- YARN-3837. javadocs of TimelineAuthenticationFilterInitializer give wrong
- prefix for auth options. (Bibin A Chundatt via devaraj)
-
- YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7.
- (Wei Yan via vvasudev)
-
- YARN-3892. Fixed NPE on RMStateStore#serviceStop when
- CapacityScheduler#serviceInit fails. (Bibin A Chundatt via jianhe)
-
- YARN-3888. ApplicationMaster link is broken in RM WebUI when appstate is NEW.
- (Bibin A Chundatt via xgong)
-
- YARN-3917. getResourceCalculatorPlugin for the default should intercept all
- exceptions. (gera)
-
- YARN-3894. RM startup should fail for wrong CS xml NodeLabel capacity
- configuration. (Bibin A Chundatt via wangda)
-
- YARN-3381. Fix typo InvalidStateTransitonException.
- (Brahma Reddy Battula via aajisaka)
-
- YARN-3453. Ensure preemption logic in FairScheduler uses DominantResourceCalculator
- in DRF queues to prevent unnecessary thrashing. (asuresh)
-
- YARN-3174. Consolidate the NodeManager and NodeManagerRestart documentation
- into one. (Masatake Iwasaki via ozawa)
-
- YARN-3805. Update the documentation of Disk Checker based on YARN-90.
- (Masatake Iwasaki via ozawa)
-
- YARN-3930. FileSystemNodeLabelsStore should make sure edit log file closed when
- exception is thrown. (Dian Fu via wangda)
-
- YARN-3885. ProportionalCapacityPreemptionPolicy doesn't preempt if queue is
- more than 2 level. (Ajith S via wangda)
-
- YARN-3932. SchedulerApplicationAttempt#getResourceUsageReport and UserInfo
- should based on total-used-resources. (Bibin A Chundatt via wangda)
-
- YARN-3954. Fix TestYarnConfigurationFields#testCompareConfigurationClassAgainstXml.
- (varun saxena via rohithsharmaks)
-
- YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda)
-
- YARN-3941. Proportional Preemption policy should try to avoid sending duplicate
- PREEMPT_CONTAINER event to scheduler. (Sunil G via wangda)
-
- YARN-3900. Protobuf layout of yarn_security_token causes errors in other protos
- that include it (adhoot via rkanter)
-
- YARN-3845. Scheduler page does not render RGBA color combinations in IE11.
- (Contributed by Mohammad Shahid Khan)
-
- YARN-3957. FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to
- return 500. (Anubhav Dhoot via kasha)
-
- YARN-3973. Recent changes to application priority management break
- reservation system from YARN-1051. (Carlo Curino via wangda)
-
- YARN-3958. TestYarnConfigurationFields should be moved to hadoop-yarn-api
- module. (Varun Saxena via aajisaka)
-
- YARN-3846. RM Web UI queue filter is not working for sub queue.
- (Mohammad Shahid Khan via jianhe)
-
- YARN-3982. container-executor parsing of container-executor.cfg broken in
- trunk and branch-2. (Varun Vasudev via xgong)
-
- YARN-3919. NPEs' while stopping service after exception during
- CommonNodeLabelsManager#start. (varun saxane via rohithsharmaks)
-
- YARN-3963. AddNodeLabel on duplicate label addition shows success.
- (Bibin A Chundatt via wangda)
-
- YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue
- on nodelabel recovery. (Bibin A Chundatt via wangda)
-
- YARN-433. When RM is catching up with node updates then it should not expire
- acquired containers. (Xuan Gong via zxu)
-
- YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails
- intermittently. (Contributed by Sunil G)
-
- YARN-3987. Send AM container completed msg to NM once AM finishes.
- (sandflee via jianhe)
-
- YARN-4028. AppBlock page key update and diagnostics value null on recovery
- (Bibin A Chundatt via xgong)
-
- YARN-3986. getTransferredContainers in AbstractYarnScheduler should be present
- in YarnScheduler interface instead. (Varun Saxena via rohithsharmaks)
-
- YARN-4082. Container shouldn't be killed when node's label updated.
- (Wangda Tan via vvasudev)
-
- YARN-4073. Removed unused ApplicationACLsManager in ContainerManagerImpl constructor.
- (Naganarasimha G R via rohithsharmaks)
-
- YARN-4024. YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat.
- (Hong Zhiguo via wangda)
-
- YARN-3591. Resource localization on a bad disk causes subsequent containers failure.
- (Lavkesh Lahngir via vvasudev)
-
- YARN-4106. NodeLabels for NM in distributed mode is not updated even after
- clusterNodelabel addition in RM. (Bibin A Chundatt via wangda)
-
- YARN-4115. Reduce loglevel of ContainerManagementProtocolProxy to Debug
- (adhoot via rkanter)
-
- YARN-4151. Fix findbugs errors in hadoop-yarn-server-common module.
- (Meng Ding via wangda)
-
- YARN-4078. Add getPendingResourceRequestForAttempt in YarnScheduler interface.
- (Naganarasimha G R via jianhe)
-
- YARN-4135. Improve the assertion message in MockRM while failing after waiting for the state.
- (Nijel S F via rohithsharmaks)
-
- YARN-4167. NPE on RMActiveServices#serviceStop when store is null. (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4113. RM should respect retry-interval when uses RetryPolicies.RETRY_FOREVER.
- (Sunil G via wangda)
-
- YARN-4188. Make MoveApplicationAcrossQueues abstract, newInstance static.
- (Giovanni Matteo Fumarola via cdouglas)
-
- YARN-4171. Fix findbugs warnings in YARN-1197 branch. (Wangda Tan via jianhe)
-
- YARN-4152. NodeManager crash with NPE when LogAggregationService#stopContainer called for
- absent container. (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4044. Running applications information changes such as movequeue is not published to
- TimeLine server. (Sunil G via rohithsharmaks)
-
- YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot)
-
- YARN-4141. Runtime Application Priority change should not throw exception
- for applications at finishing states (Sunil G via jlowe)
-
- YARN-4228. FileSystemRMStateStore use IOUtils#close instead of fs#close. (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4235. FairScheduler PrimaryGroup does not handle empty groups returned
- for a user. (Anubhav Dhoot via rohithsharmaks)
-
- YARN-4140. RM container allocation delayed incase of app submitted to
- Nodelabel partition. (Bibin A Chundatt via wangda)
-
- YARN-4201. AMBlacklist does not work for minicluster. (Jun Gong via zxu)
-
- YARN-4230. RM crashes with NPE when increasing container resource if there is no headroom left.
- (Meng Ding via jianhe)
-
- YARN-4017. container-executor overuses PATH_MAX. (Sidharta Seethana via vvasudev)
-
- YARN-4253. Standardize on using PrivilegedOperationExecutor for all
- invocations of container-executor in LinuxContainerExecutor. (Sidharta Seethana via vvasudev)
-
- YARN-4255. container-executor does not clean up docker operation command files.
- (Sidharta Seethana via vvasudev)
-
- YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks)
-
- YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart.
- (Varun Saxena via jianhe)
-
- YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing
- (Bibin A Chundatt via stevel)
-
- YARN-4270. Limit application resource reservation on nodes for non-node/rack
- specific requests (asuresh)
-
- YARN-4256. YARN fair scheduler vcores with decimal values. (Jun Gong via zxu)
-
- YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal
- tag. (aajisaka)
-
- YARN-4289. TestDistributedShell failing with bind exception.
- (Brahma Reddy Battula via stevel)
-
- YARN-3724. Use POSIX nftw(3) instead of fts(3) (Alan Burlison via aw)
-
- YARN-4246. NPE while listing app attempt. (Nijel S F via rohithsharmaks)
-
- YARN-3528. Tests with 12345 as hard-coded port break jenkins.
- (Brahma Reddy Battula via ozawa)
-
- YARN-4223. Fixed findbugs warnings in hadoop-yarn-server-nodemanager project
- (varun saxena via rohithsharmaks)
-
- YARN-4284. condition for AM blacklisting is too narrow (Sangjin Lee via
- jlowe)
-
- YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels.
- (Naganarasimha G R via wangda)
-
- YARN-4300. [JDK8] Fix javadoc errors caused by wrong tags. (aajisaka)
-
- YARN-3573. MiniMRYarnCluster constructor that starts the timeline server
- using a boolean should be marked deprecated. (Brahma Reddy Battula via ozawa)
-
- YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt
- (Bibin A Chundatt via vvasudev)
-
- YARN-4251. TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart
- is failing. (Brahma Reddy Battula via ozawa)
-
- YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method.
- (Kai Sasaki via rohithsharmaks)
-
- YARN-4288. Fixed RMProxy to retry on IOException from local host.
- (Junping Du via jianhe)
-
- YARN-4127. RM fail with noAuth error if switched from failover to non-failover.
- (Varun Saxena via jianhe)
-
- YARN-4345. yarn rmadmin -updateNodeResource doesn't work (Junping Du via
- jlowe)
-
- YARN-4367. SLS webapp doesn't load. (kasha).
-
- YARN-4298. Fix findbugs warnings in hadoop-yarn-common.
- (Sunil G via aajisaka)
-
- YARN-4387. Fix typo in FairScheduler log message. (Xin Wang via ozawa)
-
- YARN-4384. updateNodeResource CLI should not accept negative values for resource.
- (Junping Du via wangda)
-
- YARN-4408. Fix issue that NodeManager reports negative running containers.
- (Robert Kanter via junping_du)
-
- YARN-4392. ApplicationCreatedEvent event time resets after RM restart/failover.
- (Naganarasimha G R and Xuan Gong via xgong)
-
- YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect
- to RM. (Junping Du via rohithsharmaks)
-
- YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition.
- (Daniel Templeton via rohithsharmaks)
-
- YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time
- when calculating period. (Junping Du via jianhe)
-
- YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with
- bind exception. (Brahma Reddy Battula via jianhe)
-
- YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the
- lastScheduler time. (Lin Yiqun via zxu)
-
- YARN-4452. NPE when submit Unmanaged application. (Naganarasimha G R
- via junping_du)
-
- YARN-4225. Add preemption status to yarn queue -status for capacity scheduler.
- (Eric Payne via wangda)
-
- YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue.
- (Naganarasimha G R via wangda)
-
- YARN-4461. Redundant nodeLocalityDelay log in LeafQueue (Eric Payne via
- jlowe)
-
- YARN-4454. NM to nodelabel mapping going wrong after RM restart.
- (Bibin A Chundatt via wangda)
-
- YARN-4477. FairScheduler: Handle condition which can result in an
- infinite loop in attemptScheduling. (Tao Jie via asuresh)
-
- YARN-4400. AsyncDispatcher.waitForDrained should be final. (Daniel Templeton
- via junping_du)
-
- YARN-1382. Remove unusableRMNodesConcurrentSet (never used) in NodeListManager
- to get rid of memory leak. (Rohith Sharma K S via junping_du)
-
- YARN-4546. ResourceManager crash due to scheduling opportunity overflow.
- (Jason Lowe via junping_du)
-
- YARN-4479. Change CS LeafQueue pendingOrderingPolicy to hornor recovered apps.
- (Rohith Sharma K S via jianhe)
-
- YARN-4534. Remove the redundant symbol in yarn rmadmin help msg.
- (Lin Yiqun via aajisaka)
-
- YARN-4389. "yarn.am.blacklisting.enabled" and "yarn.am.blacklisting.disable-
- failure-threshold" should be app specific rather than a setting for whole
- YARN cluster. (Sunil G via rohithsharmaks)
-
- YARN-4581. AHS writer thread leak makes RM crash while RM is recovering.
- (sandflee via junping_du)
-
- YARN-4538. QueueMetrics pending cores and memory metrics wrong.
- (Bibin A Chundatt via wangda)
-
- YARN-4596. SystemMetricPublisher should not swallow error messages from
- TimelineClient#putEntities. (Li Lu via jianhe)
-
- YARN-4502. Fix two AM containers get allocated when AM restart.
- (Vinod Kumar Vavilapalli via wangda)
-
- YARN-4565. Fix a bug that leads to AM resource limit not hornored when
- sizeBasedWeight enabled for FairOrderingPolicy. (wtan via jianhe)
-
- YARN-4557. Fix improper Queues sorting in PartitionedQueueComparator
- when accessible-node-labels=*. (Naganarasimha G R via wangda)
-
- YARN-4608. Redundant code statement in WritingYarnApplications.
- (Kai Sasaki via aajisaka)
-
- YARN-4605. Spelling mistake in the help message of "yarn applicationattempt"
- command. (Weiwei Yang via aajisaka)
-
- YARN-4614. Fix random failure in TestApplicationPriority#testApplicationPriority
- AllocationWithChangeInPriority. (Sunil G via rohithsharmaks)
-
- YARN-4592. Remove unused GetContainerStatus proto. (Chang Li via aajisaka)
-
- YARN-4520. Finished app info is unnecessarily persisted in NM state-store
- if container is acquired but not lunched on this node. (sandflee via jianeh)
-
- YARN-4643. Container recovery is broken with delegating container runtime
- (Sidharta Seethana via jlowe)
-
- YARN-4617. LeafQueue#pendingOrderingPolicy should always use fixed ordering
- policy instead of using same as active applications ordering policy.
- (Rohith Sharma K S via jianhe)
-
- YARN-4667. RM Admin CLI for refreshNodesResources throws NPE when nothing
- is configured. (Naganarasimha G R via devaraj)
-
- YARN-4654. Yarn node label CLI should parse "=" correctly when trying to
- remove all labels on a node. (Naganarasimha G R via rohithsharmaks)
-
- YARN-4386. refreshNodesGracefully() should send recommission event to active
- RMNodes only. (Kuhu Shukla via junping_du)
-
- YARN-4709. NMWebServices produces incorrect JSON for containers.
- (Varun Saxena via vvasudev)
-
- YARN-4748. ApplicationHistoryManagerOnTimelineStore should not
- swallow exceptions on generateApplicationReport. (Li Lu via jianhe)
-
- YARN-4671. There is no need to acquire CS lock when completing a container.
- (Meng Ding via jianhe)
-
-Release 2.7.3 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda)
-
- YARN-4492. Add documentation for preemption supported in Capacity
- scheduler (Naganarasimha G R via jlowe)
-
- YARN-4653. Document YARN security model from the perspective of
- Application Developers. (Steve Loughran via jianhe)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer
- binds to default port 8188. (Meng Ding via wangda)
-
- YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda)
-
- YARN-4374. RM capacity scheduler UI rounds user limit factor (Chang Li via
- jlowe)
-
- YARN-3849. Too much of preemption activity causing continuos killing of
- containers across queues. (Sunil G via wangda)
-
- YARN-3769. Consider user limit when calculating total pending resource for
- preemption policy in Capacity Scheduler. (Eric Payne via wangda)
-
- YARN-4344. NMs reconnecting with changed capabilities can lead to wrong
- cluster resource calculations (Varun Vasudev via jlowe)
-
- YARN-4365. FileSystemNodeLabelStore should check for root dir existence on
- startup (Kuhu Shukla via jlowe)
-
- YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill
- fails intermittently. (Varun Saxena via ozawa)
-
- YARN-4398. Remove unnecessary synchronization in RMStateStore. (Ning Ding via jianhe)
-
- YARN-3840. Resource Manager web ui issue when sorting application by id
- (with application having id > 9999) （Mohammad Shahid Khan & Varun Saxena
- via jianhe)
-
- YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page
- (Eric Payne via jeagles)
-
- YARN-4439. Clarify NMContainerStatus#toString method. (Jian He via xgong)
-
- YARN-4452. NPE when submit Unmanaged application. (Naganarasimha G R via
- junping_du)
-
- YARN-4315. NaN in Queue percentage for cluster apps page. (Bibin A Chundatt via wangda)
-
- YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via wangda)
-
- YARN-4546. ResourceManager crash due to scheduling opportunity overflow.
- (Jason Lowe via junping_du)
-
- YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non
- network exception. (Raju Bairishetti via jianhe)
-
- YARN-4414. Nodemanager connection errors are retried at multiple levels
- (Chang Li via jlowe)
-
- YARN-4581. AHS writer thread leak makes RM crash while RM is recovering.
- (sandflee via junping_du)
-
- YARN-4610. Reservations continue looking for one app causes other apps to
- starve (jlowe)
-
- YARN-4598. Invalid event: RESOURCE_FAILED at
- CONTAINER_CLEANEDUP_AFTER_KILL (tangshangwen via jlowe)
-
- YARN-4428. Redirect RM page to AHS page when AHS turned on and RM page is
- not available (Chang Li via jlowe)
-
- YARN-3102. Decommisioned Nodes not listed in Web UI (Kuhu Shukla via
- jlowe)
-
- YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir
- fails Intermittently due to IOException from cleanup. (zhihai xu via xgong)
-
- YARN-4707. Remove the extra char (>) from SecureContainer.md.
- (Brahma Reddy Battula via aajisaka)
-
- YARN-2046. Out of band heartbeats are sent only on container kill and
- possibly too early (Ming Ma via jlowe)
-
- YARN-4723. NodesListManager$UnknownNodeId ClassCastException (Kuhu Shukla
- via jlowe)
-
-Release 2.7.2 - 2016-01-25
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles)
-
- YARN-3170. YARN architecture document needs updating. (Brahma Reddy Battula
- via ozawa)
-
- YARN-3967. Fetch the application report from the AHS if the RM does not know about it.
- (Mit Desai via xgong)
-
- YARN-2801. Add documentation for node labels feature. (Wangda Tan and Naganarasimha
- G R via ozawa)
-
- YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- YARN-3793. Several NPEs when deleting local files on NM recovery (Varun
- Saxena via jlowe)
-
- YARN-3508. Prevent processing preemption events on the main RM dispatcher.
- (Varun Saxena via wangda)
-
- YARN-3690. [JDK8] 'mvn site' fails. (Brahma Reddy Battula via aajisaka)
-
- YARN-3905. Application History Server UI NPEs when accessing apps run after
- RM restart (Eric Payne via jeagles)
-
- YARN-3535. Scheduler must re-request container resources when RMContainer transitions
- from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh)
-
- YARN-3878. AsyncDispatcher can hang while stopping if it is configured for
- draining events on stop. (Varun Saxena via jianhe)
-
- YARN-3969. Allow jobs to be submitted to reservation that is active
- but does not have any allocations. (subru via curino)
-
- YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container
- log files from full disks. (zhihai xu via jlowe)
-
- YARN-3857: Memory leak in ResourceManager with SIMPLE mode.
- (mujunchao via zxu)
-
- YARN-3893. Both RM in active state when Admin#transitionToActive failure
- from refeshAll() (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4103. RM WebServices missing scheme for appattempts logLinks.
- (Jonathan Eagles via vvasudeb)
-
- YARN-4105. Capacity Scheduler headroom for DRF is wrong (Chang Li via
- jlowe)
-
- YARN-4096. App local logs are leaked if log aggregation fails to initialize
- for the app. (Jason Lowe via zxu)
-
- YARN-4136. LinuxContainerExecutor loses info when forwarding
- ResourceHandlerException. (Bibin A Chundatt via vvasudev)
-
- YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown.
- (Zhihai Xu via kasha)
-
- YARN-4126. RM should not issue delegation tokens in unsecure mode.
- (Bibin A Chundatt via jianhe)
-
- YARN-4158. Remove duplicate close for LogWriter in
- AppLogAggregatorImpl#uploadLogsForContainers (Zhihai Xu via jlowe)
-
- YARN-3433. Jersey tests failing with Port in Use -again.
- (Brahma Reddy Battula)
-
- YARN-3975. WebAppProxyServlet should not redirect to RM page if AHS is
- enabled (Mit Desai via jlowe)
-
- YARN-3624. ApplicationHistoryServer should not reverse the order of the
- filters it gets. (Mit Desai via xgong)
-
- YARN-4180. AMLauncher does not retry on failures when talking to NM.
- (adhoot)
-
- YARN-3619. ContainerMetrics unregisters during getMetrics and leads to
- ConcurrentModificationException (Zhihai Xu via jlowe)
-
- YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called
- by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks)
-
- YARN-4041. Slow delegation token renewal can severely prolong RM recovery
- (Sunil G via jlowe)
-
- YARN-2902. Killing a container that is localizing can orphan resources in
- the DOWNLOADING state (Varun Saxena via jlowe)
-
- YARN-4313. Race condition in MiniMRYarnCluster when getting history server
- address. (Jian He via xgong)
-
- YARN-3580. [JDK8] TestClientRMService.testGetLabelsToNodes fails. (Robert Kanter
- via junping_du)
-
- YARN-4312. TestSubmitApplicationWithRMHA fails on branch-2.7 and branch-2.6
- as some of the test cases time out. (Varun Saxena via ozawa)
-
- YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer
- binds to default port 8188. (Varun Saxena via ozawa)
-
- YARN-4354. Public resource localization fails with NPE. (Jason Lowe via
- junping_du)
-
-Release 2.7.1 - 2015-07-06
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- YARN-3243. CapacityScheduler should pass headroom from parent to children
- to make sure ParentQueue obey its capacity limits. (Wangda Tan via jianhe)
-
- YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue
- info once. (Varun Saxena via wangda)
-
- YARN-3539. Updated timeline server documentation and marked REST APIs evolving.
- (Steve Loughran via zjshen)
-
- YARN-3723. Need to clearly document primaryFilter and otherInfo value type.
- (Zhijie Shen via xgong)
-
- YARN-3711. Documentation of ResourceManager HA should explain configurations
- about listen addresses. (Masatake Iwasaki via ozawa)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- YARN-3497. ContainerManagementProtocolProxy modifies IPC timeout conf
- without making a copy. (Jason Lowe via jianhe)
-
- YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly.
- (Xuan Gong via stevel)
-
- YARN-3522. Fixed DistributedShell to instantiate TimeLineClient as the
- correct user. (Zhijie Shen via jianhe)
-
- YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha)
-
- YARN-3382. Some of UserMetricsInfo metrics are incorrectly set to root
- queue metrics. (Rohit Agarwal via jianhe)
-
- YARN-3472. Fixed possible leak in DelegationTokenRenewer#allTokens.
- (Rohith Sharmaks via jianhe)
-
- YARN-3465. Use LinkedHashMap to preserve order of resource requests.
- (Zhihai Xu via kasha)
-
- YARN-3516. killing ContainerLocalizer action doesn't take effect when
- private localizer receives FETCH_FAILURE status.(zhihai xu via xgong)
-
- YARN-3485. FairScheduler headroom calculation doesn't consider
- maxResources for Fifo and FairShare policies. (kasha)
-
- YARN-3301. Fixed the format issue of the new RM attempt web page.
- (Xuan Gong via jianhe)
-
- YARN-3385. Fixed a race-condition in ResourceManager's ZooKeeper based
- state-store to avoid crashing on duplicate deletes. (Zhihai Xu via vinodkv)
-
- YARN-3358. Audit log not present while refreshing Service ACLs.
- (Varun Saxena via devaraj)
-
- YARN-3476. Nodemanager can fail to delete local logs if log aggregation
- fails (Rohith via jlowe)
-
- YARN-3434. Interaction between reservations and userlimit can result in
- significant ULF violation (tgraves)
-
- YARN-3626. On Windows localized resources are not moved to the front
- of the classpath when they should be. (Craig Welch via xgong)
-
- YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called.
- (Bibin A Chundatt via ozawa)
-
- YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore
- invoked (Brahma Reddy Battula via jlowe)
-
- YARN-3601. Fix UT TestRMFailover.testRMWebAppRedirect. (Weiwei Yang via xgong)
-
- YARN-3677. Fix findbugs warnings in yarn-server-resourcemanager.
- (Vinod Kumar Vavilapalli via ozawa)
-
- YARN-3681. yarn cmd says "could not find main class 'queue'" in windows.
- (Craig Welch and Varun Saxena via xgong)
-
- YARN-3609. Load node labels from storage inside RM serviceStart. (Wangda
- Tan via jianhe)
-
- YARN-3694. Fix dead link for TimelineServer REST API.
- (Jagadesh Kiran N via aajisaka)
-
- YARN-3646. Applications are getting stuck some times in case of retry
- policy forever. (Raju Bairishetti via devaraj)
-
- YARN-3675. FairScheduler: RM quits when node removal races with
- continuous-scheduling on the same node. (Anubhav Dhoot via kasha)
-
- YARN-3701. Isolating the error of generating a single app report when
- getting all apps from generic history service. (Zhijie Shen via xgong)
-
- YARN-2238. filtering on UI sticks even if I move away from the page.
- (Jian He via xgong)
-
- YARN-3686. CapacityScheduler should trim default_node_label_expression.
- (Sunil G via wangda)
-
- YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent
- to another. (Wangda Tan via jianhe)
-
- YARN-3804. Both RM are on standBy state when kerberos user not in yarn.admin.acl
- (Varun Saxena via xgong)
-
- YARN-3842. NMProxy should retry on NMNotYetReadyException.
- (Robert Kanter via kasha)
-
- YARN-3809. Failed to launch new attempts because
- ApplicationMasterLauncher's threads all hang (Jun Gong via jlowe)
-
-Release 2.7.0 - 2015-04-20
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- YARN-2179. [YARN-1492] Initial cache manager structure and context.
- (Chris Trezzo via kasha)
-
- YARN-2180. [YARN-1492] In-memory backing store for cache manager.
- (Chris Trezzo via kasha)
-
- YARN-2183. [YARN-1492] Cleaner service for cache manager.
- (Chris Trezzo and Sangjin Lee via kasha)
-
- YARN-2186. [YARN-1492] Node Manager uploader service for cache manager.
- (Chris Trezzo and Sangjin Lee via kasha)
-
- YARN-2236. [YARN-1492] Shared Cache uploader service on the Node
- Manager. (Chris Trezzo and Sangjin Lee via kasha)
-
- YARN-2188. [YARN-1492] Client service for cache manager.
- (Chris Trezzo and Sangjin Lee via kasha)
-
- YARN-2189. [YARN-1492] Admin service for cache manager.
- (Chris Trezzo via kasha)
-
- YARN-2765. Added leveldb-based implementation for RMStateStore. (Jason Lowe
- via jianhe)
-
- YARN-2203. [YARN-1492] Web UI for cache manager. (Chris Trezzo via kasha)
-
- YARN-2738. [YARN-2574] Add FairReservationSystem for FairScheduler.
- (Anubhav Dhoot via kasha)
-
- YARN-2881. [YARN-2574] Implement PlanFollower for FairScheduler.
- (Anubhav Dhoot via kasha)
-
- YARN-2427. Added the API of moving apps between queues in RM web services.
- (Varun Vasudev via zjshen)
-
- YARN-2217. [YARN-1492] Shared cache client side changes.
- (Chris Trezzo via kasha)
-
- YARN-2616 [YARN-913] Add CLI client to the registry to list, view
- and manipulate entries. (Akshay Radia via stevel)
-
- YARN-2994. Document work-preserving RM restart. (Jian He via ozawa)
-
- YARN-2786. Created a yarn cluster CLI and seeded with one command for listing
- node-labels collection. (Wangda Tan via vinodkv)
-
- IMPROVEMENTS
-
- YARN-3005. [JDK7] Use switch statement for String instead of if-else
- statement in RegistrySecurity.java (Kengo Seki via aajisaka)
-
- YARN-2950. Change message to mandate, not suggest JS requirement on UI.
- (Dustin Cote via harsh)
-
- YARN-2891. Failed Container Executor does not provide a clear error
- message. (Dustin Cote via harsh)
-
- YARN-1979. TestDirectoryCollection fails when the umask is unusual.
- (Vinod Kumar Vavilapalli and Tsuyoshi OZAWA via junping_du)
-
- YARN-2641. Decommission nodes on -refreshNodes instead of next
- NM-RM heartbeat. (Zhihai Xu via kasha)
-
- YARN-2742. FairSchedulerConfiguration should allow extra spaces
- between value and unit. (Wei Yan via kasha)
-
- YARN-2712. TestWorkPreservingRMRestart: Augment FS tests with
- queue and headroom checks. (Tsuyoshi Ozawa via kasha)
-
- YARN-2735. diskUtilizationPercentageCutoff and diskUtilizationSpaceCutoff
- are initialized twice in DirectoryCollection. (Zhihai Xu via kasha)
-
- YARN-570. Time strings are formated in different timezone.
- (Akira Ajisaka and Peng Zhang via kasha)
-
- YARN-2780. Log aggregated resource allocation in rm-appsummary.log (Eric
- Payne via jlowe)
-
- YARN-2690. [YARN-2574] Make ReservationSystem and its dependent classes
- independent of Scheduler type. (Anubhav Dhoot via kasha)
-
- YARN-2157. Added YARN metrics in the documentaion. (Akira AJISAKA via
- jianhe)
-
- YARN-2802. ClusterMetrics to include AM launch and register delays.
- (Zhihai Xu via kasha)
-
- YARN-2375. Allow enabling/disabling timeline server per framework.
- (Mit Desai via jeagles)
-
- YARN-2604. Scheduler should consider max-allocation-* in conjunction
- with the largest node. (Robert Kanter via kasha)
-
- YARN-2679. Add metric for container launch duration. (Zhihai Xu via kasha)
-
- YARN-2669. FairScheduler: queue names shouldn't allow periods
- (Wei Yan via Sandy Ryza)
-
- YARN-2404. Removed ApplicationAttemptState and ApplicationState class in
- RMStateStore. (Tsuyoshi OZAWA via jianhe)
-
- YARN-2165. Added the sanity check for the numeric configuration values of
- the timeline service. (Vasanth kumar RJ via zjshen)
-
- YARN-2907. SchedulerNode#toString should print all resource detail instead
- of only memory. (Rohith via junping_du)
-
- YARN-2136. Changed RMStateStore to ignore store opearations when fenced.
- (Varun Saxena via jianhe)
-
- YARN-1156. Enhance NodeManager AllocatedGB and AvailableGB metrics
- for aggregation of decimal values. (Tsuyoshi OZAWA via junping_du)
-
- YARN-2056. Disable preemption at Queue level (Eric Payne via jlowe)
-
- YARN-2762. Fixed RMAdminCLI to trim and check node-label related arguments
- before sending to RM. (Rohith Sharmaks via jianhe)
-
- YARN-2972. DelegationTokenRenewer thread pool never expands. (Jason Lowe
- via junping_du)
-
- YARN-2949. Add documentation for CGroups (Varun Vasudev via junping_du)
-
- YARN-2970. NodeLabel operations in RMAdmin CLI get missing in help command.
- (Varun Saxena via junping_du)
-
- YARN-2837. Support TimeLine server to recover delegation token when
- restarting. (Zhijie Shen via jianhe)
-
- YARN-2993. Several fixes (missing acl check, error log msg ...) and some
- refinement in AdminService. (Yi Liu via junping_du)
-
- YARN-2943. Added node-labels page on RM web UI. (Wangda Tan via jianhe)
-
- YARN-2998. Abstract out scheduler independent PlanFollower components.
- (Anubhav Dhoot via kasha)
-
- YARN-2360. Fair Scheduler: Display dynamic fair share for queues on the
- scheduler page. (Ashwin Shankar and Wei Yan via kasha)
-
- YARN-2880. Added a test to make sure node labels will be recovered
- if RM restart is enabled. (Rohith Sharmaks via jianhe)
-
- YARN-2996. Improved synchronization and I/O operations of FS- and Mem-
- RMStateStore. (Yi Liu via zjshen)
-
- YARN-2956. Added missing links in YARN documentation. (Masatake Iwasaki via
- jianhe)
-
- YARN-2957. Create unit test to automatically compare YarnConfiguration
- and yarn-default.xml. (rchiang via rkanter)
-
- YARN-2643. Don't create a new DominantResourceCalculator on every
- FairScheduler.allocate call. (kasha via rkanter)
-
- YARN-3019. Make work-preserving-recovery the default mechanism for RM
- recovery. (Jian He via junping_du)
-
- YARN-2807. Option "--forceactive" not works as described in usage of
- "yarn rmadmin -transitionToActive". (Masatake Iwasaki via xgong)
-
- YARN-2984. Metrics for container's actual memory usage. (kasha)
-
- YARN-2800. Remove MemoryNodeLabelsStore and add a way to enable/disable
- node labels feature. (Wangda Tan via ozawa)
-
- YARN-3086. Make NodeManager memory configurable in MiniYARNCluster.
- (Robert Metzger via ozawa)
-
- YARN-2897. CrossOriginFilter needs more log statements (Mit Desai via
- jeagles)
-
- YARN-3028. Better syntax for replaceLabelsOnNode in RMAdmin CLI
- (Rohith Sharmaks via wangda)
-
- YARN-2932. Add entry for "preemptable" status (enabled/disabled) to
- scheduler web UI and queue initialize/refresh logging.
- (Eric Payne via wangda)
-
- YARN-3108. ApplicationHistoryServer doesn't process -D arguments (Chang Li
- via jeagles)
-
- YARN-2808. Made YARN CLI list attempt’s finished containers of a running
- application. (Naganarasimha G R via zjshen)
-
- YARN-3085. Application summary should include the application type (Rohith
- via jlowe)
-
- YARN-3022. Expose Container resource information from NodeManager for
- monitoring (adhoot via ranter)
-
- YARN-3075. NodeLabelsManager implementation to retrieve label to node
- mapping (Varun Saxena via wangda)
-
- YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha)
-
- YARN-1723. AMRMClientAsync missing blacklist addition and removal
- functionality. (Bartosz Ługowski via sseth)
-
- YARN-3123. Made YARN CLI show a single completed container even if the app
- is running. (Naganarasimha G R via zjshen)
-
- YARN-1582. Capacity Scheduler: add a maximum-allocation-mb setting per
- queue (Thomas Graves via jlowe)
-
- YARN-1904. Ensure exceptions thrown in ClientRMService &
- ApplicationHistoryClientService are uniform when application-attempt is
- not found. (zjshen via acmurthy)
-
- YARN-3144. Configuration for making delegation token failures to timeline
- server not-fatal (Jonathan Eagles via jlowe)
-
- YARN-3155. Refactor the exception handling code for TimelineClientImpl's
- retryOn method (Li Lu via wangda)
-
- YARN-3100. Made YARN authorization pluggable. (Jian He via zjshen)
-
- YARN-2683. [YARN-913] registry config options: document and move to
- core-default. (stevel)
-
- YARN-1237. Description for yarn.nodemanager.aux-services in
- yarn-default.xml is misleading. (Brahma Reddy Battula via ozawa)
-
- YARN-3157. Refactor the exception handling in ConverterUtils#to*Id.
- (Bibin A Chundatt via ozawa)
-
- YARN-3147. Clean up RM web proxy code. (Steve Loughran via xgong)
-
- YARN-2079. Recover NonAggregatingLogHandler state upon nodemanager
- restart. (Jason Lowe via junping_du)
-
- YARN-3158. Correct log messages in ResourceTrackerService.
- (Varun Saxena via xgong)
-
- YARN-3179. Update use of Iterator to Iterable in RMAdminCLI and
- CommonNodeLabelsManager. (Ray Chiang via xgong)
-
- YARN-3182. Cleanup switch statement in ApplicationMasterLauncher#handle().
- (Ray Chiang via ozawa)
-
- YARN-3203. Correct a log message in AuxServices. (Brahma Reddy Battula
- via ozawa)
-
- YARN-1299. Improve a log message in AppSchedulingInfo by adding application
- id. (Ashutosh Jindal and Devaraj K via ozawa)
-
- YARN-1514. Utility to benchmark ZKRMStateStore#loadState for RM HA.
- (Tsuyoshi OZAWA via jianhe)
-
- YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node
- mapping. (Varun Saxena via wangda)
-
- YARN-2799. Cleanup TestLogAggregationService based on the change in YARN-90.
- (Zhihai Xu via junping_du)
-
- YARN-3237. AppLogAggregatorImpl fails to log error cause.
- (Rushabh S Shah via xgong)
-
- YARN-3236. Cleanup RMAuthenticationFilter#AUTH_HANDLER_PROPERTY.
- (zhihai xu via xgong)
-
- YARN-2797. TestWorkPreservingRMRestart should use ParametrizedSchedulerTestBase
- (Karthik Kambatla via xgong)
-
- YARN-2797. Add -help to yarn logs and nodes CLI command.
- (Jagadesh Kiran N via devaraj)
-
- YARN-3217. Remove httpclient dependency from hadoop-yarn-server-web-proxy.
- (Brahma Reddy Battula via ozawa).
-
- YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main()
- should support generic options. (shv)
-
- YARN-2820. Retry in FileSystemRMStateStore when FS's operations fail
- due to IOException. (Zhihai Xu via ozawa)
-
- YARN-3262. Surface application outstanding resource requests table
- in RM web UI. (Jian He via wangda)
-
- YARN-3281. Added RMStateStore to StateMachine visualization list.
- (Chengbing Liu via jianhe)
-
- YARN-3272. Surface container locality info in RM web UI.
- (Jian He via wangda)
-
- YARN-3122. Metrics for container's actual CPU usage.
- (Anubhav Dhoot via kasha)
-
- YARN-2190. Added CPU and memory limit options to the default container
- executor for Windows containers. (Chuan Liu via jianhe)
-
- YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable
- resource monitoring. (Hitesh Shah via junping_du)
-
- YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user
- or group. (Gururaj Shetty via jianhe)
-
- YARN-2854. Updated the documentation of the timeline service and the generic
- history service. (Naganarasimha G R via zjshen)
-
- YARN-2777. Mark the end of individual log in aggregated log.
- (Varun Saxena via xgong)
-
- YARN-3273. Improve scheduler UI to facilitate scheduling analysis and
- debugging. (Rohith Sharmaks via jianhe)
-
- OPTIMIZATIONS
-
- YARN-2990. FairScheduler's delay-scheduling always waits for node-local and
- rack-local delays, even for off-rack-only requests. (kasha)
-
- BUG FIXES
-
- YARN-3071. Remove invalid char from sample conf in doc of FairScheduler.
- (Masatake Iwasaki via aajisaka)
-
- YARN-2254. TestRMWebServicesAppsModification should run against both
- CS and FS. (Zhihai Xu via kasha)
-
- YARN-2713. "RM Home" link in NM should point to one of the RMs in an
- HA setup. (kasha)
-
- YARN-2857. ConcurrentModificationException in ContainerLogAppender
- (Mohammad Kamrul Islam via jlowe)
-
- YARN-2432. RMStateStore should process the pending events before close.
- (Varun Saxena via jianhe)
-
- YARN-1703. Fixed ResourceManager web-proxy to close connections correctly.
- (Rohith Sharma via vinodkv)
-
- YARN-2870. Updated the command to run the timeline server in the document.
- (Masatake Iwasaki via zjshen)
-
- YARN-2878. Fix DockerContainerExecutor.apt.vm formatting. (Abin Shahab via
- jianhe)
-
- YARN-2315. FairScheduler: Set current capacity in addition to capacity.
- (Zhihai Xu via kasha)
-
- YARN-2697. Remove useless RMAuthenticationHandler. (Haosong Huang via zjshen)
-
- YARN-2461. Fix PROCFS_USE_SMAPS_BASED_RSS_ENABLED property in
- YarnConfiguration. (rchiang via rkanter)
-
- YARN-2869. CapacityScheduler should trim sub queue names when parse
- configuration. (Wangda Tan via jianhe)
-
- YARN-2927. [YARN-1492] InMemorySCMStore properties are inconsistent.
- (Ray Chiang via kasha)
-
- YARN-2931. PublicLocalizer may fail until directory is initialized by
- LocalizeRunner. (Anubhav Dhoot via kasha)
-
- YARN-2930. Fixed TestRMRestart#testRMRestartRecoveringNodeLabelManager
- intermittent failure. (Wangda Tan via jianhe)
-
- YARN-2924. Fixed RMAdminCLI to not convert node labels to lower case.
- (Wangda Tan via jianhe)
-
- YARN-2243. Order of arguments for Preconditions.checkNotNull() is wrong in
- SchedulerApplicationAttempt ctor. (devaraj)
-
- YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel)
-
- YARN-2356. yarn status command for non-existent application/application
- attempt/container is too verbose. (Sunil G via devaraj)
-
- YARN-2914. [YARN-1492] Potential race condition in Singleton implementation of
- SharedCacheUploaderMetrics, CleanerMetrics, ClientSCMMetrics. (Varun Saxena via kasha)
-
- YARN-2945. FSLeafQueue#assignContainer - document the reason for using both write and
- read locks. (Tsuyoshi Ozawa via kasha)
-
- YARN-2944. InMemorySCMStore can not be instantiated with ReflectionUtils#newInstance.
- (Chris Trezzo via kasha)
-
- YARN-2675. containersKilled metrics is not updated when the container is killed
- during localization. (Zhihai Xu via kasha)
-
- YARN-2975. FSLeafQueue app lists are accessed without required locks. (kasha)
-
- YARN-2977. Fixed intermittent TestNMClient failure.
- (Junping Du via ozawa)
-
- YARN-2939. Fix new findbugs warnings in hadoop-yarn-common. (Li Lu via junping_du)
-
- YARN-2940. Fix new findbugs warnings in rest of the hadoop-yarn components. (Li Lu
- via junping_du)
-
- YARN-2937. Fixed new findbugs warnings in hadoop-yarn-nodemanager. (Varun Saxena
- via zjshen)
-
- YARN-2946. Fixed potential deadlock in RMStateStore. (Rohith Sharmaks via
- jianhe)
-
- YARN-2988. Graph#save() may leak file descriptors. (Ted Yu via ozawa)
-
- YARN-2938. Fixed new findbugs warnings in hadoop-yarn-resourcemanager and
- hadoop-yarn-applicationhistoryservice. (Varun Saxena via zjshen)
-
- YARN-2987. Fixed ClientRMService#getQueueInfo to check against queue and
- app ACLs. (Varun Saxena via jianhe)
-
- YARN-2991. Fixed DrainDispatcher to reuse the draining code path in
- AsyncDispatcher. (Rohith Sharmaks via zjshen)
-
- YARN-2958. Made RMStateStore not update the last sequence number when updating the
- delegation token. (Varun Saxena via zjshen)
-
- YARN-2230. Fixed few configs description in yarn-default.xml. (Vijay Bhat
- via jianhe)
-
- YARN-3010. Fixed findbugs warning in AbstractYarnScheduler. (Yi Liu via
- jianhe)
-
- YARN-2936. Changed YARNDelegationTokenIdentifier to set proto fields on
- getProto method. (Varun Saxena via jianhe)
-
- YARN-3014. Replaces labels on a host should update all NM's labels on that
- host. (Wangda Tan via jianhe)
-
- YARN-3027. Scheduler should use totalAvailable resource from node instead of
- availableResource for maxAllocation. (adhoot via rkanter)
-
- YARN-2861. Fixed Timeline DT secret manager to not reuse RM's configs.
- (Zhijie Shen via jianhe)
-
- YARN-3064. TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync
- failure with allocation timeout. (Jian He via junping_du)
-
- YARN-2815. Excluded transitive dependency of JLine in hadoop-yarn-server-common.
- (Ferdinand Xu via zjshen)
-
- YARN-3070. TestRMAdminCLI#testHelp fails for transitionToActive command.
- (Contributed by Junping Du)
-
- YARN-3015. yarn classpath command should support same options as hadoop
- classpath. (Contributed by Varun Saxena)
-
- YARN-2933. Capacity Scheduler preemption policy should only consider capacity
- without labels temporarily. (Mayank Bansal via wangda)
-
- YARN-2731. Fixed RegisterApplicationMasterResponsePBImpl
<TRUNCATED>