You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by wa...@apache.org on 2016/03/04 06:03:24 UTC
[1/5] hadoop git commit: HADOOP-11792. Remove all of the CHANGES.txt
files.
Repository: hadoop
Updated Branches:
refs/heads/trunk ff0ee84d7 -> 1bc454615
http://git-wip-us.apache.org/repos/asf/hadoop/blob/1bc45461/hadoop-yarn-project/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt
deleted file mode 100644
index 4934e89..0000000
--- a/hadoop-yarn-project/CHANGES.txt
+++ /dev/null
@@ -1,6905 +0,0 @@
-Hadoop YARN Change Log
-
-Trunk - Unreleased
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- YARN-2882. Add an OPPORTUNISTIC ExecutionType.
- (Konstantinos Karanasos and Inigo Goiri via kasha)
-
- IMPROVEMENTS
-
- YARN-2438. yarn-env.sh cleanup (aw)
-
- YARN-2472. yarn-daemons.sh should jsut call yarn directly (Masatake Iwasaki
- via aw)
-
- YARN-2437. start-yarn.sh/stop-yarn should give info (Varun Saxena via aw)
-
- YARN-2796. deprecate sbin/yarn-daemon.sh (aw)
-
- YARN-2980. Move health check script related functionality to hadoop-common
- (Varun Saxena via aw)
-
- YARN-3168. Convert site documentation from apt to markdown (Gururaj Shetty
- via aw)
-
- YARN-3199. Fair Scheduler documentation improvements (Rohit Agarwal via
- aw)
-
- YARN-2280. Resource manager web service fields are not accessible
- (Krisztian Horvath via aw)
-
- YARN-3261. rewrite resourcemanager restart doc to remove roadmap bits (Gururaj Shetty via aw)
-
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via gtcarrera9)
-
- YARN-524 TestYarnVersionInfo failing if generated properties doesn't
- include an SVN URL. (stevel)
-
- YARN-1471. The SLS simulator is not running the preemption policy
- for CapacityScheduler (Carlo Curino via cdouglas)
-
- YARN-2436. [post-HADOOP-9902] yarn application help doesn't work (aw)
-
- YARN-2525. yarn logs command gives error on trunk (Akira AJISAKA via aw)
-
- YARN-3002. YARN documentation needs updating post-shell rewrite (aw)
-
- YARN-2428. LCE default banned user list should have yarn (Varun
- Saxena via aw)
-
- YARN-2355. MAX_APP_ATTEMPTS_ENV may no longer be a useful env var
- for a container (Darrell Taylor via aw)
-
- YARN-3915. scmadmin help message correction (Bibin A Chundatt via aw)
-
-Release 2.9.0 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- YARN-1856. Added cgroups based memory monitoring for containers as another
- alternative to custom memory-monitoring. (Varun Vasudev via vinodkv)
-
- YARN-4462. FairScheduler: Disallow preemption from a queue. (Tao Jie via kasha)
-
- IMPROVEMENTS
-
- YARN-4072. ApplicationHistoryServer, WebAppProxyServer, NodeManager and
- ResourceManager to support JvmPauseMonitor as a service.
- (Sunil G via Stevel)
-
- YARN-4341. add doc about timeline performance tool usage (Chang Li via
- sjlee)
-
- YARN-4417. Make RM and Timeline-server REST APIs more consistent.
- (wtan via jianhe)
-
- YARN-3480. Remove attempts that are beyond max-attempt limit from state
- store. (Jun Gong via jianhe)
-
- YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda)
-
- YARN-4438. Implement RM leader election with curator. (Jian He via xgong)
-
- YARN-4571. Make app id/name available to the yarn authorizer provider for
- better auditing. (Jian He via wangda)
-
- YARN-4551. Address the duplication between StatusUpdateWhenHealthy and
- StatusUpdateWhenUnhealthy transitions. (Sunil G via kasha)
-
- YARN-4553. Add cgroups support for docker containers.
- (Sidharta Seethana via vvasudev)
-
- YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it.
- (kasha)
-
- YARN-4559. Make leader elector and zk store share the same curator client.
- (Jian He via xgong)
-
- YARN-4603. FairScheduler should mention user requested queuename in error
- message when failed in queue ACL check. (Tao Jie via kasha)
-
- YARN-4496. Improve HA ResourceManager Failover detection on the client.
- (Jian He via xgong)
-
- YARN-3542. Refactored existing CPU cgroups support to use the newer and
- integrated ResourceHandler mechanism, and also deprecated the old
- LCEResourceHandler inteface hierarchy. (Varun Vasudev via vinodkv)
-
- YARN-4647. Make RegisterNodeManagerRequestPBImpl thread-safe. (kasha)
-
- YARN-4649. Add additional logging to some NM state store operations.
- (Sidharta Seethana via vvasudev)
-
- YARN-4625. Make ApplicationSubmissionContext and
- ApplicationSubmissionContextInfo more consistent.
- (Xuan Gong via vvasudev)
-
- YARN-4307. Display blacklisted nodes for AM container in the RM web UI.
- (Naganarasimha G R via vvasudev)
-
- YARN-4628. Display application priority in yarn top.
- (Bibin A Chundatt via vvasudev)
-
- YARN-4655. Log uncaught exceptions/errors in various thread pools in YARN.
- (Sidharta Seethana via vvasudev)
-
- YARN-4689. FairScheduler: Cleanup preemptContainer to be more readable.
- (Kai Sasaki via kasha)
-
- YARN-4708. Missing default mapper type in TimelineServer performance test tool
- usage. (Kai Sasaki via ozawa)
-
- YARN-4648. Move preemption related tests from TestFairScheduler to
- TestFairSchedulerPreemption. (Kai Sasaki via ozawa)
-
- YARN-4697. NM aggregation thread pool is not bound by
- limits (haibochen via rkanter)
-
- YARN-4718. Rename variables in SchedulerNode to reduce ambiguity post
- YARN-1011. (Inigo Goiri via kasha)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- YARN-4109. Exception on RM scheduler page loading with labels.
- (Mohammad Shahid Khan via rohithsharmaks)
-
- YARN-4156. TestAMRestart#testAMBlacklistPreventsRestartOnSameNode
- assumes CapacityScheduler. (Anubhav Dhoot via kasha)
-
- YARN-2934. Improve handling of container's stderr.
- (Naganarasimha G R via gera)
-
- YARN-4530. LocalizedResource trigger a NPE Cause the NodeManager exit
- (tangshangwen via rohithsharmaks)
-
- YARN-4535. Fix checkstyle error in CapacityScheduler.java
- (Naganarasimha G R via rohithsharmaks)
-
- YARN-4393. Fix intermittent test failure for TestResourceLocalizationService
- #testFailedDirsResourceRelease (Varun Saxana via rohithsharmaks)
-
- YARN-4550. Some tests in TestContainerLanch fails on non-english locale
- environment. (Takashi Ohnishi via stevel)
-
- YARN-4567. javadoc failing on java 8. (Steve Loughran via aajisaka)
-
- YARN-3446. FairScheduler headroom calculation should exclude nodes in the
- blacklist. (Zhihai Xu via kasha)
-
- YARN-4611. Fix scheduler load simulator to support multi-layer network
- location. (Ming Ma via xgong)
-
- YARN-4584. RM startup failure when AM attempts greater than max-attempts.
- (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4578. Directories that are mounted in docker containers need to be more
- restrictive/container-specific. (Sidharta Seethana via vvasudev)
-
- YARN-4497. RM might fail to restart when recovering apps whose attempts are missing.
- (Jun Gong via rohithsharmaks)
-
- YARN-4574. Fix random failure in TestAMRMClientOnRMRestart.
- (Takashi Ohnishi via rohithsharmaks)
-
- YARN-4613. Fix test failure in TestClientRMService#testGetClusterNodes.
- (Takashi Ohnishi via rohithsharmaks)
-
- YARN-4612. Fix rumen and scheduler load simulator handle killed tasks properly.
- (Ming Ma via xgong)
-
- YARN-4573. Fix test failure in TestRMAppTransitions#testAppRunningKill and
- testAppKilledKilled. (Takashi Ohnishi via rohithsharmaks)
-
- YARN-4633. Fix random test failure in TestRMRestart#testRMRestartAfterPreemption
- (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant
- (Akihiro Suda via rohithsharmaks)
-
- YARN-4615. Fix random test failure in TestAbstractYarnScheduler#testResource
- RequestRecoveryToTheRightAppAttempt. (Sunil G via rohithsharmaks)
-
- YARN-4594. container-executor fails to remove directory tree when chmod
- required (Colin Patrick McCabe via jlowe)
-
- YARN-4669. Fix logging statements in resource manager's Application class.
- (Seethana Sidharta via vvasudev)
-
- YARN-4629. Distributed shell breaks under strong security.
- (Daniel Templeton via stevel)
-
- YARN-4684. TestYarnCLI#testGetContainers failing in CN locale.
- (Bibin A Chundatt via vvasudev)
-
- YARN-4569. Remove incorrect part of maxResources in FairScheduler
- documentation. (Ray Chiang via kasha)
-
- YARN-4651. Document movetoqueue option in 'YARN Commands'
- (Takashi Ohnishi via rohithsharmaks)
-
- YARN-4729. SchedulerApplicationAttempt#getTotalRequiredResources can throw
- an NPE. (kasha)
-
- YARN-4701. When task logs are not available, port 8041 is referenced
- instead of port 8042 (haibochen via rkanter)
-
- YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization.
- (Takashi Ohnishi via rohithsharmaks)
-
- YARN-4731. container-executor should not follow symlinks in
- recursive_unlink_children (Colin Patrick McCabe via jlowe)
-
- YARN-4704. TestResourceManager#testResourceAllocation() fails when using
- FairScheduler. (Yufei Gu via kasha)
-
-Release 2.8.0 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- YARN-2336. Fair scheduler's REST API returns a missing '[' bracket JSON for
- deep queue tree. (Kenji Kikushima and Akira Ajisaka via ozawa)
-
- NEW FEATURES
-
- YARN-3360. Add JMX metrics to TimelineDataManager (Jason Lowe via jeagles)
-
- YARN-3345. Add non-exclusive node label API. (Wangda Tan via jianhe)
-
- YARN-3365. Enhanced NodeManager to support using the 'tc' tool via
- container-executor for outbound network traffic control. (Sidharta Seethana
- via vinodkv)
-
- YARN-1376. NM need to notify the log aggregation status to RM through
- heartbeat. (Xuan Gong via junping_du)
-
- YARN-3348. Add a 'yarn top' tool to help understand cluster usage. (Varun
- Vasudev via jianhe)
-
- YARN-3347. Improve YARN log command to get AMContainer logs as well as
- running containers logs. (Xuan Gong via junping_du)
-
- YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support
- for new resource types on the NM. (Sidharta Seethana via junping_du)
-
- YARN-3361. CapacityScheduler side changes to support non-exclusive node
- labels. (Wangda Tan via jianhe)
-
- YARN-3318. Create Initial OrderingPolicy Framework and FifoOrderingPolicy.
- (Craig Welch via wangda)
-
- YARN-3326. Support RESTful API for getLabelsToNodes. (Naganarasimha G R
- via ozawa)
-
- YARN-3354. Add node label expression in ContainerTokenIdentifier to support
- RM recovery. (Wangda Tan via jianhe)
-
- YARN-1402. Update related Web UI and CLI with exposing client API to check
- log aggregation status. (Xuan Gong via junping_du)
-
- YARN-3463. Integrate OrderingPolicy Framework with CapacityScheduler.
- (Craig Welch via wangda)
-
- YARN-3410. YARN admin should be able to remove individual application
- records from RMStateStore. (Rohith Sharmaks via wangda)
-
- YARN-3225. New parameter of CLI for decommissioning node gracefully in
- RMAdmin CLI. (Devaraj K via junping_du)
-
- YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing
- network bandwidth traffic originating from YARN containers (Sidharta Seethana
- via vinodkv)
-
- YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda)
-
- YARN-2498. Respect labels in preemption policy of capacity scheduler for
- inter-queue preemption. (Wangda Tan via jianhe)
-
- YARN-2619. Added NodeManager support for disk io isolation through cgroups.
- (Varun Vasudev and Wei Yan via vinodkv)
-
- YARN-3448. Added a rolling time-to-live LevelDB timeline store implementation.
- (Jonathan Eagles via zjshen)
-
- YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in
- RMApps. (Xuan Gong via junping_du)
-
- YARN-3541. Add version info on timeline service / generic history web UI
- and REST API. (Zhijie Shen via xgong)
-
- YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from
- underlying OS when configured to do so. (Varun Vasudev via vinodkv)
-
- YARN-41. The RM should handle the graceful shutdown of the NM. (Devaraj K via
- junping_du)
-
- YARN-1012. Report NM aggregated container resource utilization in heartbeat.
- (Inigo Goiri via kasha)
-
- YARN-3116. RM notifies NM whether a container is an AM container or normal
- task container. (Giovanni Matteo Fumarola via zjshen)
-
- YARN-2003. Support for Application priority : Changes in RM and Capacity
- Scheduler. (Sunil G via wangda)
-
- YARN-3656. LowCost: A Cost-Based Placement Agent for YARN Reservations.
- (Jonathan Yaniv and Ishai Menache via curino)
-
- YARN-3852. Add docker container support to container-executor
- (Abin Shahab via vvasudev)
-
- YARN-3853. Add docker container runtime support to LinuxContainterExecutor.
- (Sidharta Seethana via vvasudev)
-
- YARN-3736. Add RMStateStore apis to store and load accepted reservations for
- failover (adhoot via asuresh)
-
- YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks)
-
- YARN-3873. PendingApplications in LeafQueue should also use OrderingPolicy.
- (Sunil G via wangda)
-
- YARN-3887. Support changing Application priority during runtime. (Sunil G
- via jianhe)
-
- YARN-4023. Publish Application Priority to TimelineServer. (Sunil G
- via rohithsharmaks)
-
- YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha)
-
- YARN-4055. Report node resource utilization in heartbeat.
- (Inigo Goiri via kasha)
-
- YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed
- Node Label Configuration Setup. (Naganarasimha G R)
-
- YARN-221. NM should provide a way for AM to tell it not to aggregate logs.
- (Ming Ma via xgong)
-
- YARN-4014. Support user cli interface in for Application Priority.
- (Rohith Sharma K S via jianhe)
-
- YARN-3250. Support admin cli interface in for Application Priority.
- (Rohith Sharma K S via jianhe)
-
- YARN-3970. Add REST api support for Application Priority.
- (Naganarasimha G R via vvasudev)
-
- YARN-2884. Added a proxy service in NM to proxy the the communication
- between AM and RM. (Kishore Chaliparambil via jianhe)
-
- YARN-313. Add Admin API for supporting node resource configuration in
- command line. (Inigo Goiri, Kenji Kikushima and Junping Du
- via junping_du)
-
- YARN-4034. Render cluster Max Priority in scheduler metrics in RM web
- UI. (Rohith Sharma K S via jianhe)
-
- YARN-3212. RMNode State Transition Update with DECOMMISSIONING state.
- (Junping Du via wangda)
-
- YARN-3866. AM-RM protocol changes to support container resizing. (Meng Ding
- via jianhe)
-
- YARN-1449. AM-NM protocol changes to support container resizing.
- (Meng Ding & Wangda Tan via jianhe)
-
- YARN-1645. ContainerManager implementation to support container resizing.
- (Meng Ding & Wangda Tan via jianhe)
-
- YARN-3867. ContainerImpl changes to support container resizing. (Meng Ding
- via jianhe)
-
- YARN-1643. Make ContainersMonitor support changing monitoring size of an
- allocated container. (Meng Ding and Wangda Tan)
-
- YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to
- support container resizing. (Meng Ding via jianhe)
-
- YARN-3868. Recovery support for container resizing. (Meng Ding via jianhe)
-
- YARN-1651. CapacityScheduler side changes to support container resize.
- (Wangda Tan via jianhe)
-
- YARN-1897. CLI and core support for signal container functionality.
- (Ming Ma via xgong)
-
- YARN-261. Ability to fail AM attempts (Andrey Klochkov and
- Rohith Sharma K S via jlowe)
-
- YARN-3964. Support NodeLabelsProvider at Resource Manager side.
- (Dian Fu via devaraj)
-
- YARN-2556. Tool to measure the performance of the timeline server (Chang Li
- via sjlee)
-
- YARN-4262. Allow whitelisted users to run privileged docker containers.
- (Sidharta Seethana via vvasudev)
-
- YARN-3739. Add reservation system recovery to RM recovery process.
- (Subru Krishnan via adhoot)
-
- YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label
- Configuration Setup. (Naganarasimha G R via rohithsharmaks)
-
- YARN-1510. Make NMClient support change container resources.
- (Meng Ding via wangda)
-
- YARN-1509. Make AMRMClient support send increase container request and
- get increased/decreased containers. (Meng Ding via wangda)
-
- YARN-4184. Remove update reservation state api from state store as its not used by
- ReservationSystem (Sean Po via asuresh)
-
- YARN-4349. Support CallerContext in YARN. (wtan via jianhe)
-
- YARN-3623. Add a new config to indicate the Timeline Service version.
- (Xuan Gong via junping_du)
-
- YARN-3226. UI changes for decommissioning node. (Sunil G via
- junping_du)
-
- YARN-3458. CPU resource monitoring in Windows. (Inigo Goiri via cnauroth)
-
- YARN-4234. New put APIs in TimelineClient for ats v1.5. (Xuan Gong via
- junping_du)
-
- YARN-4265. Provide new timeline plugin storage to support fine-grained entity
- caching. (Li Lu and Jason Lowe via junping_du)
-
- YARN-4219. New levelDB cache storage for timeline v1.5. (Li Lu via xgong)
-
- YARN-4411. RMAppAttemptImpl#createApplicationAttemptReport throws
- IllegalArgumentException. (Bibin A Chundatt, yarntime via devaraj)
-
- YARN-3223. Resource update during NM graceful decommission. (Brook Zhou
- via junping_du)
-
- IMPROVEMENTS
-
- YARN-644. Basic null check is not performed on passed in arguments before
- using them in ContainerManagerImpl.startContainer (Varun Saxena via bobby)
-
- YARN-1880. Cleanup TestApplicationClientProtocolOnHA
- (ozawa via harsh)
-
- YARN-3357. Move TestFifoScheduler to FIFO package. (Rohith Sharmaks
- via devaraj)
-
- YARN-3356. Capacity Scheduler FiCaSchedulerApp should use ResourceUsage to
- track used-resources-by-label. (Wangda Tan via jianhe)
-
- YARN-3350. YARN RackResolver spams logs with messages at info level.
- (Wilfred Spiegelenburg via junping_du)
-
- YARN-2868. FairScheduler: Metric for latency to allocate first container
- for an application. (Ray Chiang via kasha)
-
- YARN-3397. yarn rmadmin should skip -failover. (J.Andreina via kasha)
-
- YARN-3288. Document and fix indentation in the DockerContainerExecutor code
-
- YARN-2495. Allow admin specify labels from each NM (Distributed
- configuration for node label). (Naganarasimha G R via wangda)
-
- YARN-3258. FairScheduler: Need to add more logging to investigate
- allocations. (Anubhav Dhoot via ozawa)
-
- YARN-3428. Debug log resources to be localized for a container. (kasha)
-
- YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring
- from info to debug. (Anubhav Dhoot via ozawa)
-
- YARN-2901. Add errors and warning metrics page to RM, NM web UI.
- (Varun Vasudev via wangda)
-
- YARN-3294. Allow dumping of Capacity Scheduler debug logs via
- web UI for a fixed time period. (Varun Vasudev via xgong)
-
- YARN-3293. Track and display capacity scheduler health metrics
- in web UI. (Varun Vasudev via xgong)
-
- YARN-3394. Enrich WebApplication proxy documentation. (Naganarasimha G R
- via jianhe)
-
- YARN-3404. Display queue name on application page. (Ryu Kobayashi via jianhe)
-
- YARN-2696. Queue sorting in CapacityScheduler should consider node label.
- (Wangda Tan via jianhe)
-
- YARN-3451. Display attempt start time and elapsed time on the web UI.
- (Rohith Sharmaks via jianhe)
-
- YARN-3494. Expose AM resource limit and usage in CS QueueMetrics. (Rohith
- Sharmaks via jianhe)
-
- YARN-3503. Expose disk utilization percentage and bad local and log dir
- counts in NM metrics. (Varun Vasudev via jianhe)
-
- YARN-3511. Add errors and warnings page to ATS. (Varun Vasudev via xgong)
-
- YARN-3406. Display count of running containers in the RM's Web UI.
- (Ryu Kobayashi via ozawa)
-
- YARN-3363. add localization and container launch time to ContainerMetrics
- at NM to show these timing information for each active container.
- (zxu via rkanter)
-
- YARN-3396. Handle URISyntaxException in ResourceLocalizationService.
- (Brahma Reddy Battula via junping_du)
-
- YARN-3491. PublicLocalizer#addResource is too slow. (zxu via rkanter)
-
- YARN-3523. Cleanup ResourceManagerAdministrationProtocol interface audience.
- (Naganarasimha G R via junping_du)
-
- YARN-3169. Drop YARN's overview document. (Brahma Reddy Battula via ozawa)
-
- YARN-2784. Make POM project names consistent. (Rohith via devaraj)
-
- YARN-20. More information for "yarn.resourcemanager.webapp.address" in
- yarn-default.xml (Bartosz Ługowski vai tgraves)
-
- YARN-3593. Add label-type and Improve "DEFAULT_PARTITION" in Node Labels Page.
- (Naganarasimha G R via wangda)
-
- YARN-3018. Unified the default value for the configuration property
- yarn.scheduler.capacity.node-locality-delay in code and default xml file.
- (Nijel SF via vinodkv)
-
- YARN-2331. Distinguish shutdown during supervision vs. shutdown for
- rolling upgrade. (Jason Lowe via xgong)
-
- YARN-1050. Document the Fair Scheduler REST API.
- (Kenji Kikushima and Roman Shaposhnik via kasha)
-
- YARN-3271. FairScheduler: Move tests related to max-runnable-apps from
- TestFairScheduler to TestAppRunnability. (nijel via kasha)
-
- YARN-1912. ResourceLocalizer started without any jvm memory control.
- (Masatake Iwasaki via xgong)
-
- YARN-1287. Consolidate MockClocks.
- (Sebastian Wong and Anubhav Dhoot via kasha)
-
- YARN-3395. FairScheduler: Trim whitespaces when using username for
- queuename. (Zhihai Xu via kasha)
-
- YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc.
- projects. (Gabor Liptak via junping_du)
-
- YARN-3513. Remove unused variables in ContainersMonitorImpl and add debug
- log for overall resource usage by all containers. (Naganarasimha G R via devaraj)
-
- YARN-3613. TestContainerManagerSecurity should init and start Yarn cluster in
- setup instead of individual methods. (nijel via kasha)
-
- YARN-3579. CommonNodeLabelsManager should support NodeLabel instead of string
- label name when getting node-to-label/label-to-label mappings. (Sunil G via wangda)
-
- YARN-3521. Support return structured NodeLabel objects in REST API (Sunil G via wangda)
-
- YARN-3362. Add node label usage in RM CapacityScheduler web UI.
- (Naganarasimha G R via wangda)
-
- YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use
- NodeLabel object instead of String. (Naganarasimha G R via wangda)
-
- YARN-3583. Support of NodeLabel object instead of plain String
- in YarnClient side. (Sunil G via wangda)
-
- YARN-3684. Changed ContainerExecutor's primary lifecycle methods to use a more
- extensible mechanism of context objects. (Sidharta Seethana via vinodkv)
-
- YARN-3594. WintuilsProcessStubExecutor.startStreamReader leaks streams.
- (Lars Francke via junping_du)
-
- YARN-3647. RMWebServices api's should use updated api from CommonNodeLabelsManager
- to get NodeLabel object. (Sunil G via wangda)
-
- YARN-3581. Deprecate -directlyAccessNodeLabelStore in RMAdminCLI.
- (Naganarasimha G R via wangda)
-
- YARN-3722. Merge multiple TestWebAppUtils into o.a.h.yarn.webapp.util.TestWebAppUtils.
- (Masatake Iwasaki via devaraj)
-
- YARN-3713. Remove duplicate function call storeContainerDiagnostics in
- ContainerDiagnosticsUpdateTransition (zxu via rkanter)
-
- YARN-3467. Expose allocatedMB, allocatedVCores, and runningContainers metrics on
- running Applications in RM Web UI. (Anubhav Dhoot via kasha)
-
- YARN-2392. Add more diags about app retry limits on AM failures. (Steve
- Loughran via jianhe)
-
- YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. (xgong)
-
- YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator.
- (Karthik Kambatla via jianhe)
-
- YARN-3786. Document yarn class path options.
- (Brahma Reddy Battula via cnauroth)
-
- YARN-3787. Allowed generic history service to load a number of applications whose
- started time is within the given range. (Xuan Gong via zjshen)
-
- YARN-3785. Support for Resource as an argument during submitApp call in MockRM
- test class. (Sunil G via xgong)
-
- YARN-3789. Improve logs for LeafQueue#activateApplications().
- (Bibin A Chundatt via devaraj)
-
- YARN-3148. Allow CORS related headers to passthrough in WebAppProxyServlet.
- (Varun Saxena via devaraj)
-
- YARN-3834. Scrub debug logging of tokens during resource localization.
- (Chris Nauroth via xgong)
-
- YARN-3827. Migrate YARN native build to new CMake framework (Alan Burlison
- via Colin P. McCabe)
-
- YARN-3800. Reduce storage footprint for ReservationAllocation. (Anubhav Dhoot
- via curino)
-
- YARN-3069. Document missing properties in yarn-default.xml.
- (Ray Chiang via aajisaka)
-
- YARN-3844. Make hadoop-yarn-project Native code -Wall-clean (Alan Burlison
- via Colin P. McCabe)
-
- YARN-3026. Move application-specific container allocation logic from
- LeafQueue to FiCaSchedulerApp. (Wangda Tan via jianhe)
-
- YARN-3950. Add unique SHELL_ID environment variable to DistributedShell
- (Robert Kanter via jlowe)
-
- YARN-3965. Add startup timestamp to nodemanager UI (Hong Zhiguo via jlowe)
-
- YARN-3543. ApplicationReport should be able to tell whether the Application
- is AM managed or not. (Rohith Sharma K S via xgong)
-
- YARN-4004. container-executor should print output of docker logs if the docker
- container exits with non-0 exit status. (Varun Vasudev via xgong)
-
- YARN-3961. Expose pending, running and reserved containers of a queue in REST
- api and yarn top (adhoot via asuresh)
-
- YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. (Robert Kanter
- via junping_du)
-
- YARN-3974. Refactor the reservation system test cases to use parameterized
- base test. (subru via curino)
-
- YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda)
-
- YARN-4026. Refactored ContainerAllocator to accept a list of priorites
- rather than a single priority. (Wangda Tan via jianhe)
-
- YARN-4031. Add JvmPauseMonitor to ApplicationHistoryServer and
- WebAppProxyServer (djp via rkanter)
-
- YARN-4057. If ContainersMonitor is not enabled, only print
- related log info one time. (Jun Gong via zxu)
-
- YARN-1556. NPE getting application report with a null appId. (Weiwei Yang via
- junping_du)
-
- YARN-4121. Fix typos in capacity scheduler documentation.
- (Kai Sasaki via vvasudev)
-
- YARN-4086. Allow Aggregated Log readers to handle HAR files (rkanter)
-
- YARN-4145. Make RMHATestBase abstract so its not run when running all
- tests under that namespace (adhoot via rkanter)
-
- YARN-2005. Blacklisting support for scheduling AMs. (Anubhav Dhoot via kasha)
-
- YARN-3717. Expose app/am/queue's node-label-expression to RM web UI /
- CLI / REST-API. (Naganarasimha G R via wangda)
-
- YARN-4149. yarn logs -am should provide an option to fetch all the log files
- (Varun Vasudev via xgong)
-
- YARN-2597. MiniYARNCluster should propagate reason for AHS not starting.
- (stevel)
-
- MAPREDUCE-6478. Add an option to skip cleanupJob stage or ignore cleanup
- failure during commitJob. (Junping Du via wangda)
-
- YARN-3920. FairScheduler container reservation on a node should be
- configurable to limit it to large containers (adhoot via asuresh)
-
- HADOOP-12428. Fix inconsistency between log-level guards and statements.
- (Jagadesh Kiran N and Jackie Chang via ozawa)
-
- YARN-4095. Avoid sharing AllocatorPerContext object in LocalDirAllocator
- between ShuffleHandler and LocalDirsHandlerService. (Zhihai Xu via jlowe)
-
- YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels.
- (Bibin A Chundatt via wangda)
-
- YARN-4215. RMNodeLabels Manager Need to verify and replace node labels for the
- only modified Node Label Mappings in the request. (Naganarasimha G R via wangda)
-
- YARN-3943. Use separate threshold configurations for disk-full detection
- and disk-not-full detection. (Zhihai Xu via jlowe)
-
- YARN-4252. Log container-executor invocation details when exit code is non-zero.
- (Sidharta Seethana via vvasudev)
-
- YARN-4258. Add support for controlling capabilities for docker containers.
- (Sidharta Seethana via vvasudev)
-
- YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity
- by partition to REST API. (Naganarasimha G R via wangda)
-
- YARN-4170. AM need to be notified with priority in AllocateResponse.
- (Sunil G via jianhe)
-
- YARN-4267. Add additional logging to container launch implementations in
- container-executor. (Sidharta Seethana via vvasudev)
-
- YARN-3985. Make ReservationSystem persist state using RMStateStore
- reservation APIs. (adhoot via asuresh)
-
- YARN-4243. Add retry on establishing Zookeeper conenction in
- EmbeddedElectorService#serviceInit. (Xuan Gong via junping_du)
-
- YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for
- each queue. (Siqi Li via mingma)
-
- YARN-4296. DistributedShell Log.info is not friendly.
- (Xiaowei Wang via stevel)
-
- YARN-3738. Add support for recovery of reserved apps running under dynamic
- queues (subru via asuresh)
-
- YARN-4285. Display resource usage as percentage of queue and cluster in the
- RM UI (Varun Vasudev via wangda)
-
- YARN-3216. Max-AM-Resource-Percentage should respect node labels.
- (Sunil G via wangda)
-
- YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh)
-
- YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public,
- @Unstable. (stevel)
-
- YARN-3454. Add efficient merge operation to RLESparseResourceAllocation
- (Carlo Curino via asuresh)
-
- YARN-3980. Plumb resource-utilization info in node heartbeat through to the
- scheduler. (Inigo Goiri via kasha)
-
- YARN-4132. Separate configs for nodemanager to resourcemanager connection
- timeout and retries (Chang Li via jlowe)
-
- YARN-4292. ResourceUtilization should be a part of NodeInfo REST API.
- (Sunil G via wangda)
-
- YARN-4405. Support node label store in non-appendable file system. (Wangda
- Tan via jianhe)
-
- YARN-4358. Reservation System: Improve relationship between SharingPolicy
- and ReservationAgent. (Carlo Curino via asuresh)
-
- YARN-3456. Improve handling of incomplete TimelineEntities. (Varun Saxena
- via rohithsharmaks)
-
- YARN-4248. REST API for submit/update/delete Reservations. (curino)
-
- YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to
- app's diagnostic message. (Naganarasimha G R via wangda)
-
- YARN-4309. Add container launch related debug information to container logs
- when a container fails. (Varun Vasudev via wangda)
-
- YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well.
- (Sunil G via wangda)
-
- YARN-4207. Add a non-judgemental YARN app completion status. (Rich Haase via sseth)
-
- YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda)
-
- YARN-4164. Changed updateApplicationPriority API to return the updated
- application priority. (Rohith Sharma K S via jianhe)
-
- YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh)
-
- YARN-4290. Add -showDetails option to YARN Nodes CLI to print all nodes reports
- information. (Sunil G via wangda)
-
- YARN-4098. Document ApplicationPriority feature. (Rohith Sharma K S via jianhe)
-
- YARN-4544. All the log messages about rolling monitoring interval are
- shown with WARN level. (Takashi Ohnishi via aajisaka)
-
- YARN-4537. Pull out priority comparison from fifocomparator and use compound
- comparator for FifoOrdering policy. (Rohith Sharma K S via jianhe)
-
- YARN-4582. Label-related invalid resource request exception should be able to
- properly handled by application. (Bibin A Chundatt via wangda)
-
- YARN-4304. AM max resource configuration per partition to be displayed/updated
- correctly in UI and in various partition related metrics. (Sunil G via wangda)
-
- YARN-4524. Cleanup AppSchedulingInfo. (Karthik Kambatla via wangda)
-
- YARN-4371. "yarn application -kill" should take multiple application ids
- (Sunil G via jlowe)
-
- YARN-4340. Add "list" API to reservation system. (Sean Po via wangda)
-
- YARN-4100. Add Documentation for Distributed and Delegated-Centralized
- Node Labels feature. (Naganarasimha G R via devaraj)
-
- YARN-4662. Document some newly added metrics. (Jian He via xgong)
-
- YARN-4360. Improve GreedyReservationAgent to support "early" allocations,
- and performance improvements (curino via asuresh)
-
- YARN-4420. Add REST API for List Reservations. (Sean Po via curino)
-
- YARN-4519. Potential deadlock of CapacityScheduler between decrease container
- and assign containers. (Meng Ding via jianhe)
-
- YARN-4138. Roll back container resource allocation after resource
- increase token expires. (Meng Ding via jianhe)
-
- YARN-2575. Create separate ACLs for Reservation create/update/delete/list
- ops (Sean Po via asuresh)
-
- YARN-4682. AMRM client to log when AMRM token updated.
- (Prabhu Joseph via stevel)
-
- YARN-4579. Allow DefaultContainerExecutor container log directory
- permissions to be configurable (rchiang via rkanter)
-
- OPTIMIZATIONS
-
- YARN-3339. TestDockerContainerExecutor should pull a single image and not
- the entire centos repository. (Ravindra Kumar Naik via raviprak)
-
- YARN-3469. ZKRMStateStore: Avoid setting watches that are not required.
- (Jun Gong via kasha)
-
- YARN-3006. Improve the error message when attempting manual failover with
- auto-failover enabled. (Akira AJISAKA via wangda)
-
- YARN-3547. FairScheduler: Apps that have no resource demand should not participate
- scheduling. (Xianyin Xin via kasha)
-
- YARN-3259. FairScheduler: Trigger fairShare updates on node events.
- (Anubhav Dhoot via kasha)
-
- YARN-2768. Avoid cloning Resource in FSAppAttempt#updateDemand.
- (Hong Zhiguo via kasha)
-
- YARN-3983. Refactored CapacityScheduleri#FiCaSchedulerApp to easier extend
- container allocation logic. (Wangda Tan via jianhe)
-
- YARN-3635. Refactored current queue mapping implementation in CapacityScheduler
- to use a generic PlacementManager framework. (Wangda Tan via jianhe)
-
- YARN-4066. Large number of queues choke fair scheduler.
- (Johan Gustavsson via kasha)
-
- YARN-4720. Skip unnecessary NN operations in log aggregation.
- (Jun Gong via mingma)
-
- BUG FIXES
-
- YARN-3197. Confusing log generated by CapacityScheduler. (Varun Saxena
- via devaraj)
-
- YARN-3305. Normalize AM resource request on app submission. (Rohith Sharmaks
- via jianhe)
-
- YARN-3205 FileSystemRMStateStore should disable FileSystem Cache to avoid
- get a Filesystem with an old configuration. (Zhihai Xu via ozawa)
-
- YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to
- fully qualified path. (Xuan Gong via junping_du)
-
- YARN-3241. FairScheduler handles "invalid" queue names inconsistently.
- (Zhihai Xu via kasha)
-
- YARN-3383. AdminService should use "warn" instead of "info" to log exception
- when operation fails. (Li Lu via wangda)
-
- YARN-3400. [JDK 8] Build Failure due to unreported exceptions in
- RPCUtil (rkanter)
-
- YARN-3412. RM tests should use MockRM where possible. (kasha)
-
- YARN-3425. NPE from RMNodeLabelsManager.serviceStop when
- NodeLabelsManager.serviceInit failed. (Bibin A Chundatt via wangda)
-
- YARN-3415. Non-AM containers can be counted towards amResourceUsage of a
- Fair Scheduler queue (Zhihai Xu via Sandy Ryza)
-
- YARN-3435. AM container to be allocated Appattempt AM container shown as null.
- (Bibin A Chundatt via xgong)
-
- YARN-2666. TestFairScheduler.testContinuousScheduling fails Intermittently.
- (Zhihai Xu via ozawa)
-
- YARN-3429. TestAMRMTokens.testTokenExpiry fails Intermittently with
- error message:Invalid AMRMToken (zxu via rkanter)
-
- YARN-3110. Few issues in ApplicationHistory web ui. (Naganarasimha G R via xgong)
-
- YARN-3459. Fix failiure of TestLog4jWarningErrorMetricsAppender.
- (Varun Vasudev via wangda)
-
- YARN-3266. RMContext#inactiveNodes should have NodeId as map key.
- (Chengbing Liu via jianhe)
-
- YARN-3436. Fix URIs in documantion of YARN web service REST APIs.
- (Bibin A Chundatt via ozawa)
-
- YARN-3021. YARN's delegation-token handling disallows certain trust setups
- to operate properly over DistCp. (Yongjun Zhang via jianhe)
-
- YARN-3136. Fixed a synchronization problem of
- AbstractYarnScheduler#getTransferredContainers. (Sunil G via jianhe)
-
- YARN-3495. Confusing log generated by FairScheduler.
- (Brahma Reddy Battula via ozawa)
-
- YARN-3387. Previous AM's container completed status couldn't pass to current
- AM if AM and RM restarted during the same time. (sandflee via jianhe)
-
- YARN-3444. Fix typo capabililty. (Gabor Liptak via aajisaka)
-
- YARN-3530. ATS throws exception on trying to filter results without otherinfo.
- (zhijie shen via xgong)
-
- YARN-2740. Fix NodeLabelsManager to properly handle node label modifications
- when distributed node label configuration enabled. (Naganarasimha G R via wangda)
-
- YARN-3517. RM web ui for dumping scheduler logs should be for admins only
- (Varun Vasudev via tgraves)
-
- YARN-3533. Test: Fix launchAM in MockRM to wait for attempt to be scheduled.
- (Anubhav Dhoot via jianhe)
-
- YARN-3564. Fix TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable
- fails randomly. (Jian He via wangda)
-
- YARN-2893. AMLaucher: sporadic job failures due to EOFException in
- readTokenStorageStream. (Zhihai Xu via gera)
-
- YARN-2454. Fix compareTo of variable UNBOUNDED in o.a.h.y.util.resource.Resources.
- (Xu Yang via junping_du)
-
- YARN-1993. Cross-site scripting vulnerability in TextView.java. (Kenji Kikushima
- via ozawa)
-
- YARN-3097. Logging of resource recovery on NM restart has redundancies
- (Eric Payne via jlowe)
-
- YARN-3375. NodeHealthScriptRunner.shouldRun() check is performing 3 times for
- starting NodeHealthScriptRunner. (Devaraj K via wangda)
-
- YARN-2725. Added test cases of retrying creating znode in ZKRMStateStore.
- (Tsuyoshi Ozawa via jianhe)
-
- YARN-3552. RM Web UI shows -1 running containers for completed apps
- (Rohith via jlowe)
-
- YARN-2123. Progress bars in Web UI always at 100% due to non-US locale.
- (Akira AJISAKA via xgong)
-
- YARN-3343. Increased TestCapacitySchedulerNodeLabelUpdate#testNodeUpdate
- timeout. (Rohith Sharmaks via jianhe)
-
- YARN-3582. NPE in WebAppProxyServlet. (jian he via xgong)
-
- YARN-3577. Misspelling of threshold in log4j.properties for tests.
- (Brahma Reddy Battula via aajisaka)
-
- YARN-3584. Fixed attempt diagnostics format shown on the UI. (nijel via
- jianhe)
-
- YARN-1832. Fix wrong MockLocalizerStatus#equals implementation.
- (Hong Zhiguo via aajisaka)
-
- YARN-3572. Correct typos in WritingYarnApplications.md.
- (Gabor Liptak via aajisaka)
-
- YARN-3592. Fix typos in RMNodeLabelsManager. (Sunil G via devaraj)
-
- YARN-3589. RM and AH web UI display DOCTYPE wrongly. (Rohith via ozawa)
-
- YARN-3600. AM container link is broken (Naganarasimha G R via tgraves)
-
- YARN-3604. Fixed ZKRMStateStore#removeApplication to also disable watch.
- (zhihai xu via jianhe)
-
- YARN-3473. Fix RM Web UI configuration for some properties (rchiang via rkanter)
-
- YARN-2206. Updated document for applications REST API response examples. (Kenji
- Kikushima and Brahma Reddy Battula via zjshen)
-
- YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir
- fails Intermittently due to IOException from cleanup. (zhihai xu via xgong)
-
- YARN-3629. NodeID is always printed as "null" in node manager initialization log.
- (nijel via devaraj)
-
- YARN-2921. Fix MockRM/MockAM#waitForState sleep too long.
- (Tsuyoshi Ozawa via wangda)
-
- YARN-1519. Check in container-executor if sysconf is implemented before
- using it (Radim Kolar and Eric Payne via raviprak)
-
- YARN-2421. RM still allocates containers to an app in the FINISHING
- state (Chang Li via jlowe)
-
- YARN-3302. TestDockerContainerExecutor should run automatically if it can
- detect docker in the usual place (Ravindra Kumar Naik via raviprak)
-
- YARN-2821. Fixed a problem that DistributedShell AM may hang if restarted.
- (Varun Vasudev via jianhe)
-
- YARN-3654. ContainerLogsPage web UI should not have meta-refresh. (Xuan Gong
- via jianhe)
-
- YARN-3707. RM Web UI queue filter doesn't work. (Wangda Tan via jianhe)
-
- YARN-3632. Ordering policy should be allowed to reorder an application when
- demand changes. (Craig Welch via jianhe)
-
- YARN-3716. Node-label-expression should be included by
- ResourceRequestPBImpl.toString. (Xianyin Xin via wangda)
-
- YARN-3751. Fixed AppInfo to check if used resources are null. (Sunil G via
- zjshen)
-
- YARN-3762. FairScheduler: CME on FSParentQueue#getQueueUserAclInfo. (kasha)
-
- YARN-3749. We should make a copy of configuration when init MiniYARNCluster
- with multiple RMs. (Chun Chen via xgong)
-
- YARN-3766. Fixed the apps table column error of generic history web UI.
- (Xuan Gong via zjshen)
-
- YARN-3655. FairScheduler: potential livelock due to maxAMShare limitation
- and container reservation. (Zhihai Xu via kasha)
-
- YARN-3747. TestLocalDirsHandlerService should delete the created test directory logDir2.
- (David Moore via devaraj)
-
- YARN-3778. Fix Yarn resourcemanger CLI usage. (Brahma Reddy Battula via xgong)
-
- YARN-3794. TestRMEmbeddedElector fails because of ambiguous LOG reference.
- (Chengbing Liu via devaraj)
-
- YARN-3714. AM proxy filter can not get RM webapp address from
- yarn.resourcemanager.hostname.rm-id. (Masatake Iwasaki via xgong)
-
- YARN-3617. Fix WindowsResourceCalculatorPlugin.getCpuFrequency()
- returning always -1. (J.Andreina via devaraj)
-
- YARN-3824. Fix two minor nits in member variable properties
- of YarnConfiguration. (Ray Chiang via devaraj)
-
- YARN-3835. hadoop-yarn-server-resourcemanager test package bundles
- core-site.xml, yarn-site.xml (vamsee via rkanter)
-
- YARN-3790. usedResource from rootQueue metrics may get stale data for FS
- scheduler after recovering the container (Zhihai Xu via rohithsharmaks)
-
- YARN-3826. Race condition in ResourceTrackerService leads to
- wrong diagnostics messages. (Chengbing Liu via devaraj)
-
- YARN-3745. SerializedException should also try to instantiate internal
- exception with the default constructor. (Lavkesh Lahngir via devaraj)
-
- YARN-2871. TestRMRestart#testRMRestartGetApplicationList sometime fails in trunk.
- (zhihai xu via xgong)
-
- YARN-3859. LeafQueue doesn't print user properly for application add.
- (Varun Saxena via devaraj)
-
- YARN-3860. rmadmin -transitionToActive should check the state of non-target node.
- (Masatake Iwasaki via junping_du)
-
- YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non
- network exception. (Raju Bairishetti via jianhe)
-
- YARN-3770. SerializedException should also handle java.lang.Error on
- de-serialization. (Lavkesh Lahngir via jianhe)
-
- YARN-3768. ArrayIndexOutOfBoundsException with empty environment variables.
- (Zhihai Xu via gera)
-
- YARN-3823. Fix mismatch in default values for
- yarn.scheduler.maximum-allocation-vcores property. (Ray Chiang via devaraj)
-
- YARN-3830. AbstractYarnScheduler.createReleaseCache may try to clean a null
- attempt. (nijel via devaraj)
-
- YARN-3875. FSSchedulerNode#reserveResource() doesn't print Application Id
- properly in log. (Bibin A Chundatt via devaraj)
-
- YARN-3882. AggregatedLogFormat should close aclScanner and ownerScanner
- after create them. (zhihai xu via xgong)
-
- YARN-3837. javadocs of TimelineAuthenticationFilterInitializer give wrong
- prefix for auth options. (Bibin A Chundatt via devaraj)
-
- YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7.
- (Wei Yan via vvasudev)
-
- YARN-3892. Fixed NPE on RMStateStore#serviceStop when
- CapacityScheduler#serviceInit fails. (Bibin A Chundatt via jianhe)
-
- YARN-3888. ApplicationMaster link is broken in RM WebUI when appstate is NEW.
- (Bibin A Chundatt via xgong)
-
- YARN-3917. getResourceCalculatorPlugin for the default should intercept all
- exceptions. (gera)
-
- YARN-3894. RM startup should fail for wrong CS xml NodeLabel capacity
- configuration. (Bibin A Chundatt via wangda)
-
- YARN-3381. Fix typo InvalidStateTransitonException.
- (Brahma Reddy Battula via aajisaka)
-
- YARN-3453. Ensure preemption logic in FairScheduler uses DominantResourceCalculator
- in DRF queues to prevent unnecessary thrashing. (asuresh)
-
- YARN-3174. Consolidate the NodeManager and NodeManagerRestart documentation
- into one. (Masatake Iwasaki via ozawa)
-
- YARN-3805. Update the documentation of Disk Checker based on YARN-90.
- (Masatake Iwasaki via ozawa)
-
- YARN-3930. FileSystemNodeLabelsStore should make sure edit log file closed when
- exception is thrown. (Dian Fu via wangda)
-
- YARN-3885. ProportionalCapacityPreemptionPolicy doesn't preempt if queue is
- more than 2 level. (Ajith S via wangda)
-
- YARN-3932. SchedulerApplicationAttempt#getResourceUsageReport and UserInfo
- should based on total-used-resources. (Bibin A Chundatt via wangda)
-
- YARN-3954. Fix TestYarnConfigurationFields#testCompareConfigurationClassAgainstXml.
- (varun saxena via rohithsharmaks)
-
- YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda)
-
- YARN-3941. Proportional Preemption policy should try to avoid sending duplicate
- PREEMPT_CONTAINER event to scheduler. (Sunil G via wangda)
-
- YARN-3900. Protobuf layout of yarn_security_token causes errors in other protos
- that include it (adhoot via rkanter)
-
- YARN-3845. Scheduler page does not render RGBA color combinations in IE11.
- (Contributed by Mohammad Shahid Khan)
-
- YARN-3957. FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to
- return 500. (Anubhav Dhoot via kasha)
-
- YARN-3973. Recent changes to application priority management break
- reservation system from YARN-1051. (Carlo Curino via wangda)
-
- YARN-3958. TestYarnConfigurationFields should be moved to hadoop-yarn-api
- module. (Varun Saxena via aajisaka)
-
- YARN-3846. RM Web UI queue filter is not working for sub queue.
- (Mohammad Shahid Khan via jianhe)
-
- YARN-3982. container-executor parsing of container-executor.cfg broken in
- trunk and branch-2. (Varun Vasudev via xgong)
-
- YARN-3919. NPEs' while stopping service after exception during
- CommonNodeLabelsManager#start. (varun saxane via rohithsharmaks)
-
- YARN-3963. AddNodeLabel on duplicate label addition shows success.
- (Bibin A Chundatt via wangda)
-
- YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue
- on nodelabel recovery. (Bibin A Chundatt via wangda)
-
- YARN-433. When RM is catching up with node updates then it should not expire
- acquired containers. (Xuan Gong via zxu)
-
- YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails
- intermittently. (Contributed by Sunil G)
-
- YARN-3987. Send AM container completed msg to NM once AM finishes.
- (sandflee via jianhe)
-
- YARN-4028. AppBlock page key update and diagnostics value null on recovery
- (Bibin A Chundatt via xgong)
-
- YARN-3986. getTransferredContainers in AbstractYarnScheduler should be present
- in YarnScheduler interface instead. (Varun Saxena via rohithsharmaks)
-
- YARN-4082. Container shouldn't be killed when node's label updated.
- (Wangda Tan via vvasudev)
-
- YARN-4073. Removed unused ApplicationACLsManager in ContainerManagerImpl constructor.
- (Naganarasimha G R via rohithsharmaks)
-
- YARN-4024. YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat.
- (Hong Zhiguo via wangda)
-
- YARN-3591. Resource localization on a bad disk causes subsequent containers failure.
- (Lavkesh Lahngir via vvasudev)
-
- YARN-4106. NodeLabels for NM in distributed mode is not updated even after
- clusterNodelabel addition in RM. (Bibin A Chundatt via wangda)
-
- YARN-4115. Reduce loglevel of ContainerManagementProtocolProxy to Debug
- (adhoot via rkanter)
-
- YARN-4151. Fix findbugs errors in hadoop-yarn-server-common module.
- (Meng Ding via wangda)
-
- YARN-4078. Add getPendingResourceRequestForAttempt in YarnScheduler interface.
- (Naganarasimha G R via jianhe)
-
- YARN-4135. Improve the assertion message in MockRM while failing after waiting for the state.
- (Nijel S F via rohithsharmaks)
-
- YARN-4167. NPE on RMActiveServices#serviceStop when store is null. (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4113. RM should respect retry-interval when uses RetryPolicies.RETRY_FOREVER.
- (Sunil G via wangda)
-
- YARN-4188. Make MoveApplicationAcrossQueues abstract, newInstance static.
- (Giovanni Matteo Fumarola via cdouglas)
-
- YARN-4171. Fix findbugs warnings in YARN-1197 branch. (Wangda Tan via jianhe)
-
- YARN-4152. NodeManager crash with NPE when LogAggregationService#stopContainer called for
- absent container. (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4044. Running applications information changes such as movequeue is not published to
- TimeLine server. (Sunil G via rohithsharmaks)
-
- YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot)
-
- YARN-4141. Runtime Application Priority change should not throw exception
- for applications at finishing states (Sunil G via jlowe)
-
- YARN-4228. FileSystemRMStateStore use IOUtils#close instead of fs#close. (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4235. FairScheduler PrimaryGroup does not handle empty groups returned
- for a user. (Anubhav Dhoot via rohithsharmaks)
-
- YARN-4140. RM container allocation delayed incase of app submitted to
- Nodelabel partition. (Bibin A Chundatt via wangda)
-
- YARN-4201. AMBlacklist does not work for minicluster. (Jun Gong via zxu)
-
- YARN-4230. RM crashes with NPE when increasing container resource if there is no headroom left.
- (Meng Ding via jianhe)
-
- YARN-4017. container-executor overuses PATH_MAX. (Sidharta Seethana via vvasudev)
-
- YARN-4253. Standardize on using PrivilegedOperationExecutor for all
- invocations of container-executor in LinuxContainerExecutor. (Sidharta Seethana via vvasudev)
-
- YARN-4255. container-executor does not clean up docker operation command files.
- (Sidharta Seethana via vvasudev)
-
- YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks)
-
- YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart.
- (Varun Saxena via jianhe)
-
- YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing
- (Bibin A Chundatt via stevel)
-
- YARN-4270. Limit application resource reservation on nodes for non-node/rack
- specific requests (asuresh)
-
- YARN-4256. YARN fair scheduler vcores with decimal values. (Jun Gong via zxu)
-
- YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal
- tag. (aajisaka)
-
- YARN-4289. TestDistributedShell failing with bind exception.
- (Brahma Reddy Battula via stevel)
-
- YARN-3724. Use POSIX nftw(3) instead of fts(3) (Alan Burlison via aw)
-
- YARN-4246. NPE while listing app attempt. (Nijel S F via rohithsharmaks)
-
- YARN-3528. Tests with 12345 as hard-coded port break jenkins.
- (Brahma Reddy Battula via ozawa)
-
- YARN-4223. Fixed findbugs warnings in hadoop-yarn-server-nodemanager project
- (varun saxena via rohithsharmaks)
-
- YARN-4284. condition for AM blacklisting is too narrow (Sangjin Lee via
- jlowe)
-
- YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels.
- (Naganarasimha G R via wangda)
-
- YARN-4300. [JDK8] Fix javadoc errors caused by wrong tags. (aajisaka)
-
- YARN-3573. MiniMRYarnCluster constructor that starts the timeline server
- using a boolean should be marked deprecated. (Brahma Reddy Battula via ozawa)
-
- YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt
- (Bibin A Chundatt via vvasudev)
-
- YARN-4251. TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart
- is failing. (Brahma Reddy Battula via ozawa)
-
- YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method.
- (Kai Sasaki via rohithsharmaks)
-
- YARN-4288. Fixed RMProxy to retry on IOException from local host.
- (Junping Du via jianhe)
-
- YARN-4127. RM fail with noAuth error if switched from failover to non-failover.
- (Varun Saxena via jianhe)
-
- YARN-4345. yarn rmadmin -updateNodeResource doesn't work (Junping Du via
- jlowe)
-
- YARN-4367. SLS webapp doesn't load. (kasha).
-
- YARN-4298. Fix findbugs warnings in hadoop-yarn-common.
- (Sunil G via aajisaka)
-
- YARN-4387. Fix typo in FairScheduler log message. (Xin Wang via ozawa)
-
- YARN-4384. updateNodeResource CLI should not accept negative values for resource.
- (Junping Du via wangda)
-
- YARN-4408. Fix issue that NodeManager reports negative running containers.
- (Robert Kanter via junping_du)
-
- YARN-4392. ApplicationCreatedEvent event time resets after RM restart/failover.
- (Naganarasimha G R and Xuan Gong via xgong)
-
- YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect
- to RM. (Junping Du via rohithsharmaks)
-
- YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition.
- (Daniel Templeton via rohithsharmaks)
-
- YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time
- when calculating period. (Junping Du via jianhe)
-
- YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with
- bind exception. (Brahma Reddy Battula via jianhe)
-
- YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the
- lastScheduler time. (Lin Yiqun via zxu)
-
- YARN-4452. NPE when submit Unmanaged application. (Naganarasimha G R
- via junping_du)
-
- YARN-4225. Add preemption status to yarn queue -status for capacity scheduler.
- (Eric Payne via wangda)
-
- YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue.
- (Naganarasimha G R via wangda)
-
- YARN-4461. Redundant nodeLocalityDelay log in LeafQueue (Eric Payne via
- jlowe)
-
- YARN-4454. NM to nodelabel mapping going wrong after RM restart.
- (Bibin A Chundatt via wangda)
-
- YARN-4477. FairScheduler: Handle condition which can result in an
- infinite loop in attemptScheduling. (Tao Jie via asuresh)
-
- YARN-4400. AsyncDispatcher.waitForDrained should be final. (Daniel Templeton
- via junping_du)
-
- YARN-1382. Remove unusableRMNodesConcurrentSet (never used) in NodeListManager
- to get rid of memory leak. (Rohith Sharma K S via junping_du)
-
- YARN-4546. ResourceManager crash due to scheduling opportunity overflow.
- (Jason Lowe via junping_du)
-
- YARN-4479. Change CS LeafQueue pendingOrderingPolicy to hornor recovered apps.
- (Rohith Sharma K S via jianhe)
-
- YARN-4534. Remove the redundant symbol in yarn rmadmin help msg.
- (Lin Yiqun via aajisaka)
-
- YARN-4389. "yarn.am.blacklisting.enabled" and "yarn.am.blacklisting.disable-
- failure-threshold" should be app specific rather than a setting for whole
- YARN cluster. (Sunil G via rohithsharmaks)
-
- YARN-4581. AHS writer thread leak makes RM crash while RM is recovering.
- (sandflee via junping_du)
-
- YARN-4538. QueueMetrics pending cores and memory metrics wrong.
- (Bibin A Chundatt via wangda)
-
- YARN-4596. SystemMetricPublisher should not swallow error messages from
- TimelineClient#putEntities. (Li Lu via jianhe)
-
- YARN-4502. Fix two AM containers get allocated when AM restart.
- (Vinod Kumar Vavilapalli via wangda)
-
- YARN-4565. Fix a bug that leads to AM resource limit not hornored when
- sizeBasedWeight enabled for FairOrderingPolicy. (wtan via jianhe)
-
- YARN-4557. Fix improper Queues sorting in PartitionedQueueComparator
- when accessible-node-labels=*. (Naganarasimha G R via wangda)
-
- YARN-4608. Redundant code statement in WritingYarnApplications.
- (Kai Sasaki via aajisaka)
-
- YARN-4605. Spelling mistake in the help message of "yarn applicationattempt"
- command. (Weiwei Yang via aajisaka)
-
- YARN-4614. Fix random failure in TestApplicationPriority#testApplicationPriority
- AllocationWithChangeInPriority. (Sunil G via rohithsharmaks)
-
- YARN-4592. Remove unused GetContainerStatus proto. (Chang Li via aajisaka)
-
- YARN-4520. Finished app info is unnecessarily persisted in NM state-store
- if container is acquired but not lunched on this node. (sandflee via jianeh)
-
- YARN-4643. Container recovery is broken with delegating container runtime
- (Sidharta Seethana via jlowe)
-
- YARN-4617. LeafQueue#pendingOrderingPolicy should always use fixed ordering
- policy instead of using same as active applications ordering policy.
- (Rohith Sharma K S via jianhe)
-
- YARN-4667. RM Admin CLI for refreshNodesResources throws NPE when nothing
- is configured. (Naganarasimha G R via devaraj)
-
- YARN-4654. Yarn node label CLI should parse "=" correctly when trying to
- remove all labels on a node. (Naganarasimha G R via rohithsharmaks)
-
- YARN-4386. refreshNodesGracefully() should send recommission event to active
- RMNodes only. (Kuhu Shukla via junping_du)
-
- YARN-4709. NMWebServices produces incorrect JSON for containers.
- (Varun Saxena via vvasudev)
-
- YARN-4748. ApplicationHistoryManagerOnTimelineStore should not
- swallow exceptions on generateApplicationReport. (Li Lu via jianhe)
-
- YARN-4671. There is no need to acquire CS lock when completing a container.
- (Meng Ding via jianhe)
-
-Release 2.7.3 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda)
-
- YARN-4492. Add documentation for preemption supported in Capacity
- scheduler (Naganarasimha G R via jlowe)
-
- YARN-4653. Document YARN security model from the perspective of
- Application Developers. (Steve Loughran via jianhe)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer
- binds to default port 8188. (Meng Ding via wangda)
-
- YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda)
-
- YARN-4374. RM capacity scheduler UI rounds user limit factor (Chang Li via
- jlowe)
-
- YARN-3849. Too much of preemption activity causing continuos killing of
- containers across queues. (Sunil G via wangda)
-
- YARN-3769. Consider user limit when calculating total pending resource for
- preemption policy in Capacity Scheduler. (Eric Payne via wangda)
-
- YARN-4344. NMs reconnecting with changed capabilities can lead to wrong
- cluster resource calculations (Varun Vasudev via jlowe)
-
- YARN-4365. FileSystemNodeLabelStore should check for root dir existence on
- startup (Kuhu Shukla via jlowe)
-
- YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill
- fails intermittently. (Varun Saxena via ozawa)
-
- YARN-4398. Remove unnecessary synchronization in RMStateStore. (Ning Ding via jianhe)
-
- YARN-3840. Resource Manager web ui issue when sorting application by id
- (with application having id > 9999) （Mohammad Shahid Khan & Varun Saxena
- via jianhe)
-
- YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page
- (Eric Payne via jeagles)
-
- YARN-4439. Clarify NMContainerStatus#toString method. (Jian He via xgong)
-
- YARN-4452. NPE when submit Unmanaged application. (Naganarasimha G R via
- junping_du)
-
- YARN-4315. NaN in Queue percentage for cluster apps page. (Bibin A Chundatt via wangda)
-
- YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via wangda)
-
- YARN-4546. ResourceManager crash due to scheduling opportunity overflow.
- (Jason Lowe via junping_du)
-
- YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non
- network exception. (Raju Bairishetti via jianhe)
-
- YARN-4414. Nodemanager connection errors are retried at multiple levels
- (Chang Li via jlowe)
-
- YARN-4581. AHS writer thread leak makes RM crash while RM is recovering.
- (sandflee via junping_du)
-
- YARN-4610. Reservations continue looking for one app causes other apps to
- starve (jlowe)
-
- YARN-4598. Invalid event: RESOURCE_FAILED at
- CONTAINER_CLEANEDUP_AFTER_KILL (tangshangwen via jlowe)
-
- YARN-4428. Redirect RM page to AHS page when AHS turned on and RM page is
- not available (Chang Li via jlowe)
-
- YARN-3102. Decommisioned Nodes not listed in Web UI (Kuhu Shukla via
- jlowe)
-
- YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir
- fails Intermittently due to IOException from cleanup. (zhihai xu via xgong)
-
- YARN-4707. Remove the extra char (>) from SecureContainer.md.
- (Brahma Reddy Battula via aajisaka)
-
- YARN-2046. Out of band heartbeats are sent only on container kill and
- possibly too early (Ming Ma via jlowe)
-
- YARN-4723. NodesListManager$UnknownNodeId ClassCastException (Kuhu Shukla
- via jlowe)
-
-Release 2.7.2 - 2016-01-25
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles)
-
- YARN-3170. YARN architecture document needs updating. (Brahma Reddy Battula
- via ozawa)
-
- YARN-3967. Fetch the application report from the AHS if the RM does not know about it.
- (Mit Desai via xgong)
-
- YARN-2801. Add documentation for node labels feature. (Wangda Tan and Naganarasimha
- G R via ozawa)
-
- YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- YARN-3793. Several NPEs when deleting local files on NM recovery (Varun
- Saxena via jlowe)
-
- YARN-3508. Prevent processing preemption events on the main RM dispatcher.
- (Varun Saxena via wangda)
-
- YARN-3690. [JDK8] 'mvn site' fails. (Brahma Reddy Battula via aajisaka)
-
- YARN-3905. Application History Server UI NPEs when accessing apps run after
- RM restart (Eric Payne via jeagles)
-
- YARN-3535. Scheduler must re-request container resources when RMContainer transitions
- from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh)
-
- YARN-3878. AsyncDispatcher can hang while stopping if it is configured for
- draining events on stop. (Varun Saxena via jianhe)
-
- YARN-3969. Allow jobs to be submitted to reservation that is active
- but does not have any allocations. (subru via curino)
-
- YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container
- log files from full disks. (zhihai xu via jlowe)
-
- YARN-3857: Memory leak in ResourceManager with SIMPLE mode.
- (mujunchao via zxu)
-
- YARN-3893. Both RM in active state when Admin#transitionToActive failure
- from refeshAll() (Bibin A Chundatt via rohithsharmaks)
-
- YARN-4103. RM WebServices missing scheme for appattempts logLinks.
- (Jonathan Eagles via vvasudeb)
-
- YARN-4105. Capacity Scheduler headroom for DRF is wrong (Chang Li via
- jlowe)
-
- YARN-4096. App local logs are leaked if log aggregation fails to initialize
- for the app. (Jason Lowe via zxu)
-
- YARN-4136. LinuxContainerExecutor loses info when forwarding
- ResourceHandlerException. (Bibin A Chundatt via vvasudev)
-
- YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown.
- (Zhihai Xu via kasha)
-
- YARN-4126. RM should not issue delegation tokens in unsecure mode.
- (Bibin A Chundatt via jianhe)
-
- YARN-4158. Remove duplicate close for LogWriter in
- AppLogAggregatorImpl#uploadLogsForContainers (Zhihai Xu via jlowe)
-
- YARN-3433. Jersey tests failing with Port in Use -again.
- (Brahma Reddy Battula)
-
- YARN-3975. WebAppProxyServlet should not redirect to RM page if AHS is
- enabled (Mit Desai via jlowe)
-
- YARN-3624. ApplicationHistoryServer should not reverse the order of the
- filters it gets. (Mit Desai via xgong)
-
- YARN-4180. AMLauncher does not retry on failures when talking to NM.
- (adhoot)
-
- YARN-3619. ContainerMetrics unregisters during getMetrics and leads to
- ConcurrentModificationException (Zhihai Xu via jlowe)
-
- YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called
- by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks)
-
- YARN-4041. Slow delegation token renewal can severely prolong RM recovery
- (Sunil G via jlowe)
-
- YARN-2902. Killing a container that is localizing can orphan resources in
- the DOWNLOADING state (Varun Saxena via jlowe)
-
- YARN-4313. Race condition in MiniMRYarnCluster when getting history server
- address. (Jian He via xgong)
-
- YARN-3580. [JDK8] TestClientRMService.testGetLabelsToNodes fails. (Robert Kanter
- via junping_du)
-
- YARN-4312. TestSubmitApplicationWithRMHA fails on branch-2.7 and branch-2.6
- as some of the test cases time out. (Varun Saxena via ozawa)
-
- YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer
- binds to default port 8188. (Varun Saxena via ozawa)
-
- YARN-4354. Public resource localization fails with NPE. (Jason Lowe via
- junping_du)
-
-Release 2.7.1 - 2015-07-06
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- YARN-3243. CapacityScheduler should pass headroom from parent to children
- to make sure ParentQueue obey its capacity limits. (Wangda Tan via jianhe)
-
- YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue
- info once. (Varun Saxena via wangda)
-
- YARN-3539. Updated timeline server documentation and marked REST APIs evolving.
- (Steve Loughran via zjshen)
-
- YARN-3723. Need to clearly document primaryFilter and otherInfo value type.
- (Zhijie Shen via xgong)
-
- YARN-3711. Documentation of ResourceManager HA should explain configurations
- about listen addresses. (Masatake Iwasaki via ozawa)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- YARN-3497. ContainerManagementProtocolProxy modifies IPC timeout conf
- without making a copy. (Jason Lowe via jianhe)
-
- YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly.
- (Xuan Gong via stevel)
-
- YARN-3522. Fixed DistributedShell to instantiate TimeLineClient as the
- correct user. (Zhijie Shen via jianhe)
-
- YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha)
-
- YARN-3382. Some of UserMetricsInfo metrics are incorrectly set to root
- queue metrics. (Rohit Agarwal via jianhe)
-
- YARN-3472. Fixed possible leak in DelegationTokenRenewer#allTokens.
- (Rohith Sharmaks via jianhe)
-
- YARN-3465. Use LinkedHashMap to preserve order of resource requests.
- (Zhihai Xu via kasha)
-
- YARN-3516. killing ContainerLocalizer action doesn't take effect when
- private localizer receives FETCH_FAILURE status.(zhihai xu via xgong)
-
- YARN-3485. FairScheduler headroom calculation doesn't consider
- maxResources for Fifo and FairShare policies. (kasha)
-
- YARN-3301. Fixed the format issue of the new RM attempt web page.
- (Xuan Gong via jianhe)
-
- YARN-3385. Fixed a race-condition in ResourceManager's ZooKeeper based
- state-store to avoid crashing on duplicate deletes. (Zhihai Xu via vinodkv)
-
- YARN-3358. Audit log not present while refreshing Service ACLs.
- (Varun Saxena via devaraj)
-
- YARN-3476. Nodemanager can fail to delete local logs if log aggregation
- fails (Rohith via jlowe)
-
- YARN-3434. Interaction between reservations and userlimit can result in
- significant ULF violation (tgraves)
-
- YARN-3626. On Windows localized resources are not moved to the front
- of the classpath when they should be. (Craig Welch via xgong)
-
- YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called.
- (Bibin A Chundatt via ozawa)
-
- YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore
- invoked (Brahma Reddy Battula via jlowe)
-
- YARN-3601. Fix UT TestRMFailover.testRMWebAppRedirect. (Weiwei Yang via xgong)
-
- YARN-3677. Fix findbugs warnings in yarn-server-resourcemanager.
- (Vinod Kumar Vavilapalli via ozawa)
-
- YARN-3681. yarn cmd says "could not find main class 'queue'" in windows.
- (Craig Welch and Varun Saxena via xgong)
-
- YARN-3609. Load node labels from storage inside RM serviceStart. (Wangda
- Tan via jianhe)
-
- YARN-3694. Fix dead link for TimelineServer REST API.
- (Jagadesh Kiran N via aajisaka)
-
- YARN-3646. Applications are getting stuck some times in case of retry
- policy forever. (Raju Bairishetti via devaraj)
-
- YARN-3675. FairScheduler: RM quits when node removal races with
- continuous-scheduling on the same node. (Anubhav Dhoot via kasha)
-
- YARN-3701. Isolating the error of generating a single app report when
- getting all apps from generic history service. (Zhijie Shen via xgong)
-
- YARN-2238. filtering on UI sticks even if I move away from the page.
- (Jian He via xgong)
-
- YARN-3686. CapacityScheduler should trim default_node_label_expression.
- (Sunil G via wangda)
-
- YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent
- to another. (Wangda Tan via jianhe)
-
- YARN-3804. Both RM are on standBy state when kerberos user not in yarn.admin.acl
- (Varun Saxena via xgong)
-
- YARN-3842. NMProxy should retry on NMNotYetReadyException.
- (Robert Kanter via kasha)
-
- YARN-3809. Failed to launch new attempts because
- ApplicationMasterLauncher's threads all hang (Jun Gong via jlowe)
-
-Release 2.7.0 - 2015-04-20
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- YARN-2179. [YARN-1492] Initial cache manager structure and context.
- (Chris Trezzo via kasha)
-
- YARN-2180. [YARN-1492] In-memory backing store for cache manager.
- (Chris Trezzo via kasha)
-
- YARN-2183. [YARN-1492] Cleaner service for cache manager.
- (Chris Trezzo and Sangjin Lee via kasha)
-
- YARN-2186. [YARN-1492] Node Manager uploader service for cache manager.
- (Chris Trezzo and Sangjin Lee via kasha)
-
- YARN-2236. [YARN-1492] Shared Cache uploader service on the Node
- Manager. (Chris Trezzo and Sangjin Lee via kasha)
-
- YARN-2188. [YARN-1492] Client service for cache manager.
- (Chris Trezzo and Sangjin Lee via kasha)
-
- YARN-2189. [YARN-1492] Admin service for cache manager.
- (Chris Trezzo via kasha)
-
- YARN-2765. Added leveldb-based implementation for RMStateStore. (Jason Lowe
- via jianhe)
-
- YARN-2203. [YARN-1492] Web UI for cache manager. (Chris Trezzo via kasha)
-
- YARN-2738. [YARN-2574] Add FairReservationSystem for FairScheduler.
- (Anubhav Dhoot via kasha)
-
- YARN-2881. [YARN-2574] Implement PlanFollower for FairScheduler.
- (Anubhav Dhoot via kasha)
-
- YARN-2427. Added the API of moving apps between queues in RM web services.
- (Varun Vasudev via zjshen)
-
- YARN-2217. [YARN-1492] Shared cache client side changes.
- (Chris Trezzo via kasha)
-
- YARN-2616 [YARN-913] Add CLI client to the registry to list, view
- and manipulate entries. (Akshay Radia via stevel)
-
- YARN-2994. Document work-preserving RM restart. (Jian He via ozawa)
-
- YARN-2786. Created a yarn cluster CLI and seeded with one command for listing
- node-labels collection. (Wangda Tan via vinodkv)
-
- IMPROVEMENTS
-
- YARN-3005. [JDK7] Use switch statement for String instead of if-else
- statement in RegistrySecurity.java (Kengo Seki via aajisaka)
-
- YARN-2950. Change message to mandate, not suggest JS requirement on UI.
- (Dustin Cote via harsh)
-
- YARN-2891. Failed Container Executor does not provide a clear error
- message. (Dustin Cote via harsh)
-
- YARN-1979. TestDirectoryCollection fails when the umask is unusual.
- (Vinod Kumar Vavilapalli and Tsuyoshi OZAWA via junping_du)
-
- YARN-2641. Decommission nodes on -refreshNodes instead of next
- NM-RM heartbeat. (Zhihai Xu via kasha)
-
- YARN-2742. FairSchedulerConfiguration should allow extra spaces
- between value and unit. (Wei Yan via kasha)
-
- YARN-2712. TestWorkPreservingRMRestart: Augment FS tests with
- queue and headroom checks. (Tsuyoshi Ozawa via kasha)
-
- YARN-2735. diskUtilizationPercentageCutoff and diskUtilizationSpaceCutoff
- are initialized twice in DirectoryCollection. (Zhihai Xu via kasha)
-
- YARN-570. Time strings are formated in different timezone.
- (Akira Ajisaka and Peng Zhang via kasha)
-
- YARN-2780. Log aggregated resource allocation in rm-appsummary.log (Eric
- Payne via jlowe)
-
- YARN-2690. [YARN-2574] Make ReservationSystem and its dependent classes
- independent of Scheduler type. (Anubhav Dhoot via kasha)
-
- YARN-2157. Added YARN metrics in the documentaion. (Akira AJISAKA via
- jianhe)
-
- YARN-2802. ClusterMetrics to include AM launch and register delays.
- (Zhihai Xu via kasha)
-
- YARN-2375. Allow enabling/disabling timeline server per framework.
- (Mit Desai via jeagles)
-
- YARN-2604. Scheduler should consider max-allocation-* in conjunction
- with the largest node. (Robert Kanter via kasha)
-
- YARN-2679. Add metric for container launch duration. (Zhihai Xu via kasha)
-
- YARN-2669. FairScheduler: queue names shouldn't allow periods
- (Wei Yan via Sandy Ryza)
-
- YARN-2404. Removed ApplicationAttemptState and ApplicationState class in
- RMStateStore. (Tsuyoshi OZAWA via jianhe)
-
- YARN-2165. Added the sanity check for the numeric configuration values of
- the timeline service. (Vasanth kumar RJ via zjshen)
-
- YARN-2907. SchedulerNode#toString should print all resource detail instead
- of only memory. (Rohith via junping_du)
-
- YARN-2136. Changed RMStateStore to ignore store opearations when fenced.
- (Varun Saxena via jianhe)
-
- YARN-1156. Enhance NodeManager AllocatedGB and AvailableGB metrics
- for aggregation of decimal values. (Tsuyoshi OZAWA via junping_du)
-
- YARN-2056. Disable preemption at Queue level (Eric Payne via jlowe)
-
- YARN-2762. Fixed RMAdminCLI to trim and check node-label related arguments
- before sending to RM. (Rohith Sharmaks via jianhe)
-
- YARN-2972. DelegationTokenRenewer thread pool never expands. (Jason Lowe
- via junping_du)
-
- YARN-2949. Add documentation for CGroups (Varun Vasudev via junping_du)
-
- YARN-2970. NodeLabel operations in RMAdmin CLI get missing in help command.
- (Varun Saxena via junping_du)
-
- YARN-2837. Support TimeLine server to recover delegation token when
- restarting. (Zhijie Shen via jianhe)
-
- YARN-2993. Several fixes (missing acl check, error log msg ...) and some
- refinement in AdminService. (Yi Liu via junping_du)
-
- YARN-2943. Added node-labels page on RM web UI. (Wangda Tan via jianhe)
-
- YARN-2998. Abstract out scheduler independent PlanFollower components.
- (Anubhav Dhoot via kasha)
-
- YARN-2360. Fair Scheduler: Display dynamic fair share for queues on the
- scheduler page. (Ashwin Shankar and Wei Yan via kasha)
-
- YARN-2880. Added a test to make sure node labels will be recovered
- if RM restart is enabled. (Rohith Sharmaks via jianhe)
-
- YARN-2996. Improved synchronization and I/O operations of FS- and Mem-
- RMStateStore. (Yi Liu via zjshen)
-
- YARN-2956. Added missing links in YARN documentation. (Masatake Iwasaki via
- jianhe)
-
- YARN-2957. Create unit test to automatically compare YarnConfiguration
- and yarn-default.xml. (rchiang via rkanter)
-
- YARN-2643. Don't create a new DominantResourceCalculator on every
- FairScheduler.allocate call. (kasha via rkanter)
-
- YARN-3019. Make work-preserving-recovery the default mechanism for RM
- recovery. (Jian He via junping_du)
-
- YARN-2807. Option "--forceactive" not works as described in usage of
- "yarn rmadmin -transitionToActive". (Masatake Iwasaki via xgong)
-
- YARN-2984. Metrics for container's actual memory usage. (kasha)
-
- YARN-2800. Remove MemoryNodeLabelsStore and add a way to enable/disable
- node labels feature. (Wangda Tan via ozawa)
-
- YARN-3086. Make NodeManager memory configurable in MiniYARNCluster.
- (Robert Metzger via ozawa)
-
- YARN-2897. CrossOriginFilter needs more log statements (Mit Desai via
- jeagles)
-
- YARN-3028. Better syntax for replaceLabelsOnNode in RMAdmin CLI
- (Rohith Sharmaks via wangda)
-
- YARN-2932. Add entry for "preemptable" status (enabled/disabled) to
- scheduler web UI and queue initialize/refresh logging.
- (Eric Payne via wangda)
-
- YARN-3108. ApplicationHistoryServer doesn't process -D arguments (Chang Li
- via jeagles)
-
- YARN-2808. Made YARN CLI list attempt’s finished containers of a running
- application. (Naganarasimha G R via zjshen)
-
- YARN-3085. Application summary should include the application type (Rohith
- via jlowe)
-
- YARN-3022. Expose Container resource information from NodeManager for
- monitoring (adhoot via ranter)
-
- YARN-3075. NodeLabelsManager implementation to retrieve label to node
- mapping (Varun Saxena via wangda)
-
- YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha)
-
- YARN-1723. AMRMClientAsync missing blacklist addition and removal
- functionality. (Bartosz Ługowski via sseth)
-
- YARN-3123. Made YARN CLI show a single completed container even if the app
- is running. (Naganarasimha G R via zjshen)
-
- YARN-1582. Capacity Scheduler: add a maximum-allocation-mb setting per
- queue (Thomas Graves via jlowe)
-
- YARN-1904. Ensure exceptions thrown in ClientRMService &
- ApplicationHistoryClientService are uniform when application-attempt is
- not found. (zjshen via acmurthy)
-
- YARN-3144. Configuration for making delegation token failures to timeline
- server not-fatal (Jonathan Eagles via jlowe)
-
- YARN-3155. Refactor the exception handling code for TimelineClientImpl's
- retryOn method (Li Lu via wangda)
-
- YARN-3100. Made YARN authorization pluggable. (Jian He via zjshen)
-
- YARN-2683. [YARN-913] registry config options: document and move to
- core-default. (stevel)
-
- YARN-1237. Description for yarn.nodemanager.aux-services in
- yarn-default.xml is misleading. (Brahma Reddy Battula via ozawa)
-
- YARN-3157. Refactor the exception handling in ConverterUtils#to*Id.
- (Bibin A Chundatt via ozawa)
-
- YARN-3147. Clean up RM web proxy code. (Steve Loughran via xgong)
-
- YARN-2079. Recover NonAggregatingLogHandler state upon nodemanager
- restart. (Jason Lowe via junping_du)
-
- YARN-3158. Correct log messages in ResourceTrackerService.
- (Varun Saxena via xgong)
-
- YARN-3179. Update use of Iterator to Iterable in RMAdminCLI and
- CommonNodeLabelsManager. (Ray Chiang via xgong)
-
- YARN-3182. Cleanup switch statement in ApplicationMasterLauncher#handle().
- (Ray Chiang via ozawa)
-
- YARN-3203. Correct a log message in AuxServices. (Brahma Reddy Battula
- via ozawa)
-
- YARN-1299. Improve a log message in AppSchedulingInfo by adding application
- id. (Ashutosh Jindal and Devaraj K via ozawa)
-
- YARN-1514. Utility to benchmark ZKRMStateStore#loadState for RM HA.
- (Tsuyoshi OZAWA via jianhe)
-
- YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node
- mapping. (Varun Saxena via wangda)
-
- YARN-2799. Cleanup TestLogAggregationService based on the change in YARN-90.
- (Zhihai Xu via junping_du)
-
- YARN-3237. AppLogAggregatorImpl fails to log error cause.
- (Rushabh S Shah via xgong)
-
- YARN-3236. Cleanup RMAuthenticationFilter#AUTH_HANDLER_PROPERTY.
- (zhihai xu via xgong)
-
- YARN-2797. TestWorkPreservingRMRestart should use ParametrizedSchedulerTestBase
- (Karthik Kambatla via xgong)
-
- YARN-2797. Add -help to yarn logs and nodes CLI command.
- (Jagadesh Kiran N via devaraj)
-
- YARN-3217. Remove httpclient dependency from hadoop-yarn-server-web-proxy.
- (Brahma Reddy Battula via ozawa).
-
- YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main()
- should support generic options. (shv)
-
- YARN-2820. Retry in FileSystemRMStateStore when FS's operations fail
- due to IOException. (Zhihai Xu via ozawa)
-
- YARN-3262. Surface application outstanding resource requests table
- in RM web UI. (Jian He via wangda)
-
- YARN-3281. Added RMStateStore to StateMachine visualization list.
- (Chengbing Liu via jianhe)
-
- YARN-3272. Surface container locality info in RM web UI.
- (Jian He via wangda)
-
- YARN-3122. Metrics for container's actual CPU usage.
- (Anubhav Dhoot via kasha)
-
- YARN-2190. Added CPU and memory limit options to the default container
- executor for Windows containers. (Chuan Liu via jianhe)
-
- YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable
- resource monitoring. (Hitesh Shah via junping_du)
-
- YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user
- or group. (Gururaj Shetty via jianhe)
-
- YARN-2854. Updated the documentation of the timeline service and the generic
- history service. (Naganarasimha G R via zjshen)
-
- YARN-2777. Mark the end of individual log in aggregated log.
- (Varun Saxena via xgong)
-
- YARN-3273. Improve scheduler UI to facilitate scheduling analysis and
- debugging. (Rohith Sharmaks via jianhe)
-
- OPTIMIZATIONS
-
- YARN-2990. FairScheduler's delay-scheduling always waits for node-local and
- rack-local delays, even for off-rack-only requests. (kasha)
-
- BUG FIXES
-
- YARN-3071. Remove invalid char from sample conf in doc of FairScheduler.
- (Masatake Iwasaki via aajisaka)
-
- YARN-2254. TestRMWebServicesAppsModification should run against both
- CS and FS. (Zhihai Xu via kasha)
-
- YARN-2713. "RM Home" link in NM should point to one of the RMs in an
- HA setup. (kasha)
-
- YARN-2857. ConcurrentModificationException in ContainerLogAppender
- (Mohammad Kamrul Islam via jlowe)
-
- YARN-2432. RMStateStore should process the pending events before close.
- (Varun Saxena via jianhe)
-
- YARN-1703. Fixed ResourceManager web-proxy to close connections correctly.
- (Rohith Sharma via vinodkv)
-
- YARN-2870. Updated the command to run the timeline server in the document.
- (Masatake Iwasaki via zjshen)
-
- YARN-2878. Fix DockerContainerExecutor.apt.vm formatting. (Abin Shahab via
- jianhe)
-
- YARN-2315. FairScheduler: Set current capacity in addition to capacity.
- (Zhihai Xu via kasha)
-
- YARN-2697. Remove useless RMAuthenticationHandler. (Haosong Huang via zjshen)
-
- YARN-2461. Fix PROCFS_USE_SMAPS_BASED_RSS_ENABLED property in
- YarnConfiguration. (rchiang via rkanter)
-
- YARN-2869. CapacityScheduler should trim sub queue names when parse
- configuration. (Wangda Tan via jianhe)
-
- YARN-2927. [YARN-1492] InMemorySCMStore properties are inconsistent.
- (Ray Chiang via kasha)
-
- YARN-2931. PublicLocalizer may fail until directory is initialized by
- LocalizeRunner. (Anubhav Dhoot via kasha)
-
- YARN-2930. Fixed TestRMRestart#testRMRestartRecoveringNodeLabelManager
- intermittent failure. (Wangda Tan via jianhe)
-
- YARN-2924. Fixed RMAdminCLI to not convert node labels to lower case.
- (Wangda Tan via jianhe)
-
- YARN-2243. Order of arguments for Preconditions.checkNotNull() is wrong in
- SchedulerApplicationAttempt ctor. (devaraj)
-
- YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel)
-
- YARN-2356. yarn status command for non-existent application/application
- attempt/container is too verbose. (Sunil G via devaraj)
-
- YARN-2914. [YARN-1492] Potential race condition in Singleton implementation of
- SharedCacheUploaderMetrics, CleanerMetrics, ClientSCMMetrics. (Varun Saxena via kasha)
-
- YARN-2945. FSLeafQueue#assignContainer - document the reason for using both write and
- read locks. (Tsuyoshi Ozawa via kasha)
-
- YARN-2944. InMemorySCMStore can not be instantiated with ReflectionUtils#newInstance.
- (Chris Trezzo via kasha)
-
- YARN-2675. containersKilled metrics is not updated when the container is killed
- during localization. (Zhihai Xu via kasha)
-
- YARN-2975. FSLeafQueue app lists are accessed without required locks. (kasha)
-
- YARN-2977. Fixed intermittent TestNMClient failure.
- (Junping Du via ozawa)
-
- YARN-2939. Fix new findbugs warnings in hadoop-yarn-common. (Li Lu via junping_du)
-
- YARN-2940. Fix new findbugs warnings in rest of the hadoop-yarn components. (Li Lu
- via junping_du)
-
- YARN-2937. Fixed new findbugs warnings in hadoop-yarn-nodemanager. (Varun Saxena
- via zjshen)
-
- YARN-2946. Fixed potential deadlock in RMStateStore. (Rohith Sharmaks via
- jianhe)
-
- YARN-2988. Graph#save() may leak file descriptors. (Ted Yu via ozawa)
-
- YARN-2938. Fixed new findbugs warnings in hadoop-yarn-resourcemanager and
- hadoop-yarn-applicationhistoryservice. (Varun Saxena via zjshen)
-
- YARN-2987. Fixed ClientRMService#getQueueInfo to check against queue and
- app ACLs. (Varun Saxena via jianhe)
-
- YARN-2991. Fixed DrainDispatcher to reuse the draining code path in
- AsyncDispatcher. (Rohith Sharmaks via zjshen)
-
- YARN-2958. Made RMStateStore not update the last sequence number when updating the
- delegation token. (Varun Saxena via zjshen)
-
- YARN-2230. Fixed few configs description in yarn-default.xml. (Vijay Bhat
- via jianhe)
-
- YARN-3010. Fixed findbugs warning in AbstractYarnScheduler. (Yi Liu via
- jianhe)
-
- YARN-2936. Changed YARNDelegationTokenIdentifier to set proto fields on
- getProto method. (Varun Saxena via jianhe)
-
- YARN-3014. Replaces labels on a host should update all NM's labels on that
- host. (Wangda Tan via jianhe)
-
- YARN-3027. Scheduler should use totalAvailable resource from node instead of
- availableResource for maxAllocation. (adhoot via rkanter)
-
- YARN-2861. Fixed Timeline DT secret manager to not reuse RM's configs.
- (Zhijie Shen via jianhe)
-
- YARN-3064. TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync
- failure with allocation timeout. (Jian He via junping_du)
-
- YARN-2815. Excluded transitive dependency of JLine in hadoop-yarn-server-common.
- (Ferdinand Xu via zjshen)
-
- YARN-3070. TestRMAdminCLI#testHelp fails for transitionToActive command.
- (Contributed by Junping Du)
-
- YARN-3015. yarn classpath command should support same options as hadoop
- classpath. (Contributed by Varun Saxena)
-
- YARN-2933. Capacity Scheduler preemption policy should only consider capacity
- without labels temporarily. (Mayank Bansal via wangda)
-
- YARN-2731. Fixed RegisterApplicationMasterResponsePBImpl
<TRUNCATED>
[2/5] hadoop git commit: HADOOP-11792. Remove all of the CHANGES.txt
files.
Posted by wa...@apache.org.
http://git-wip-us.apache.org/repos/asf/hadoop/blob/1bc45461/hadoop-mapreduce-project/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-mapreduce-project/CHANGES.txt b/hadoop-mapreduce-project/CHANGES.txt
deleted file mode 100644
index 8939007..0000000
--- a/hadoop-mapreduce-project/CHANGES.txt
+++ /dev/null
@@ -1,8906 +0,0 @@
-Hadoop MapReduce Change Log
-
-Trunk (Unreleased)
-
- INCOMPATIBLE CHANGES
-
- MAPREDUCE-5785. Derive heap size or mapreduce.*.memory.mb automatically.
- (Gera Shegalov and Karthik Kambatla via gera)
-
- MAPREDUCE-5653. DistCp does not honour config-overrides for
- mapreduce.[map,reduce].memory.mb (Ratandeep Ratti via aw)
-
- MAPREDUCE-4424. 'mapred job -list' command should show the job name
- as well. (Avinash Kujur via aajisaka)
-
- MAPREDUCE-6336. Enable v2 FileOutputCommitter by default.
- (Siqi Li via gera)
-
- MAPREDUCE-6613. Change mapreduce.jobhistory.jhist.format default from
- json to binary (rchiang via rkanter)
-
- NEW FEATURES
-
- MAPREDUCE-778. Rumen Anonymizer. (Amar Kamat and Chris Douglas via amarrk)
-
- MAPREDUCE-2669. Add new examples for Mean, Median, and Standard Deviation.
- (Plamen Jeliazkov via shv)
-
- MAPREDUCE-4887. Add RehashPartitioner, to smooth distributions
- with poor implementations of Object#hashCode(). (Radim Kolar via cutting)
-
- MAPREDUCE-5232. Add a configuration to be able to log classpath and other
- system properties on mapreduce JVMs startup. (Sangjin Lee via vinodkv)
-
- MAPREDUCE-5910. Make MR AM resync with RM in case of work-preserving
- RM-restart. (Rohith via jianhe)
-
- MAPREDUCE-2841. Add a native implementation of MapOutputCollector.
- (see section below for detailed breakdown)
-
- IMPROVEMENTS
-
- MAPREDUCE-3481. [Gridmix] Improve Gridmix STRESS mode. (amarrk)
-
- MAPREDUCE-3597. [Rumen] Rumen should provide APIs to access all the
- job-history related information.
-
- MAPREDUCE-3375. [Gridmix] Memory Emulation system tests.
- (Vinay Thota via amarrk)
-
- MAPREDUCE-2733. [Gridmix] Gridmix3 cpu emulation system tests.
- (Vinay Thota via amarrk)
-
- MAPREDUCE-2836. Provide option to fail jobs when submitted to non-existent
- fair scheduler pools. (Ahmed Radwan via todd)
-
- MAPREDUCE-3171. normalize nodemanager native code compilation with common/hdfs
- native. (tucu)
-
- MAPREDUCE-3149. Add a test to verify that TokenCache handles file system
- uri with no authority. (John George via jitendra)
-
- MAPREDUCE-3169. Create a new MiniMRCluster equivalent which only provides
- client APIs cross MR1 and MR2 (Ahmed via tucu)
-
- MAPREDUCE-2944. Improve checking of input for JobClient.displayTasks()
- (XieXianshan via harsh)
-
- MAPREDUCE-3956. Remove the use of the deprecated Syncable.sync() method from
- TeraOutputFormat in the terasort example. (szetszwo)
-
- MAPREDUCE-3935. Annotate Counters.Counter and Counters.Group as @Public.
- (tomwhite)
-
- HADOOP-8285 MR changes for Use ProtoBuf for RpcPayLoadHeader (sanjay radia)
-
- MAPREDUCE-3302. Remove the last dependency call from
- org.apache.hadoop.record package in MR. (harsh)
-
- MAPREDUCE-2384. The job submitter should make sure to validate
- jobs before creation of necessary files. (harsh)
-
- MAPREDUCE-4371. Check for cyclic dependencies in Jobcontrol job DAG
- (madhukara phatak via bobby)
-
- MAPREDUCE-4686. hadoop-mapreduce-client-core fails compilation in Eclipse
- due to missing Avro-generated classes (Chris Nauroth via harsh)
-
- MAPREDUCE-4735. Make arguments in TestDFSIO case insensitive.
- (Brandon Li via suresh)
-
- MAPREDUCE-5197. Add a service for checkpointing task state.
- (Carlo Curino via cdouglas)
-
- MAPREDUCE-5189. Add policies and wiring to respond to preemption requests
- from YARN. (Carlo Curino via cdouglas)
-
- MAPREDUCE-5196. Add bookkeeping for managing checkpoints of task state.
- (Carlo Curino via cdouglas)
-
- MAPREDUCE-5912. Task.calculateOutputSize does not handle Windows files after
- MAPREDUCE-5196. (Remus Rusanu via cnauroth)
-
- MAPREDUCE-6019. MapReduce changes for exposing YARN/MR endpoints on multiple
- interfaces. (Craig Welch, Milan Potocnik, Arpit Agarwal via xgong)
-
- MAPREDUCE-6013. [post-HADOOP-9902] mapred version is missing (Akira AJISAKA
- via aw)
-
- MAPREDUCE-6250. deprecate sbin/mr-jobhistory-daemon.sh (aw)
-
- MAPREDUCE-6260. Convert site documentation to markdown (Masatake Iwasaki
- via aw)
-
- MAPREDUCE-6057. Remove obsolete entries from mapred-default.xml
- (Ray Chiang via aw)
-
- MAPREDUCE-2632. Avoid calling the partitioner when the numReduceTasks is 1.
- (Ravi Teja Ch N V and Sunil G via kasha)
-
- MAPREDUCE-6407. Migrate MAPREDUCE nativetask build to new CMake framework
- (Alan Burlison via Colin P. McCabe)
-
- MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API
- (Junping Du via jianhe)
-
- BUG FIXES
-
- MAPREDUCE-5714. Removed forceful JVM exit in shutDownJob.
- (Jinghui Wang via Eric Yang)
-
- MAPREDUCE-3194. "mapred mradmin" command is broken in mrv2
- (Jason Lowe via bobby)
-
- MAPREDUCE-3462. Fix Gridmix JUnit testcase failures.
- (Ravi Prakash and Ravi Gummadi via amarrk)
-
- MAPREDUCE-3349. Log rack-name in JobHistory for unsuccessful tasks.
- (Devaraj K and Amar Kamat via amarrk)
-
- MAPREDUCE-3412. Fix 'ant docs'. (amarrk)
-
- MAPREDUCE-3346. [Rumen] LoggedTaskAttempt#getHostName() returns null.
- (amarrk)
-
- MAPREDUCE-2950. [Gridmix] TestUserResolve fails in trunk.
- (Ravi Gummadi via amarrk)
-
- MAPREDUCE-2784. [Gridmix] Bug fixes in ExecutionSummarizer and
- ResourceUsageMatcher. (amarrk)
-
- MAPREDUCE-2978. Fixed test-patch to make Jenkins report correct number of
- findBugs, correct links to findBugs artifacts and no links to the
- artifacts when there are no warnings. (Tom White via vinodkv).
-
- MAPREDUCE-3664. Federation Documentation has incorrect configuration example.
- (Brandon Li via jitendra)
-
- MAPREDUCE-1740. NPE in getMatchingLevelForNodes when node locations are
- variable depth (ahmed via tucu) [IMPORTANT: this is dead code in trunk]
-
- MAPREDUCE-3990. MRBench allows Long-sized input-lines value
- but parses CLI argument as an Integer. (harsh)
-
- MAPREDUCE-3868. Make Raid Compile. (Weiyan Wang via schen)
-
- MAPREDUCE-4685. DBCount should not use ACCESS. (Viji via harsh)
-
- MAPREDUCE-3223. Remove MR1 configs from mapred-default.xml (tlipcon via harsh)
-
- MAPREDUCE-4695. Fix LocalRunner on trunk after MAPREDUCE-3223 broke it
- (harsh)
-
- MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes
- (Joe Mudd via aw)
-
- MAPREDUCE-1125. SerialUtils.cc: deserializeFloat is out of sync with
- SerialUtils.hh (Simone Leo via aw)
-
- MAPREDUCE-4574. Fix TotalOrderParitioner to work with
- non-WritableComparable key types. (harsh)
-
- MAPREDUCE-5012. Typo in javadoc for IdentityMapper class. (Adam Monsen
- via suresh)
-
- MAPREDUCE-4987. TestMRJobs#testDistributedCache fails on Windows due to
- classpath problems and unexpected behavior of symlinks (Chris Nauroth via
- bikas)
-
- MAPREDUCE-5191. TestQueue#testQueue fails with timeout on Windows. (Ivan
- Mitic via hitesh)
-
- MAPREDUCE-5717. Task pings are interpreted as task progress (jlowe)
-
- MAPREDUCE-5867. Fix NPE in KillAMPreemptionPolicy related to
- ProportionalCapacityPreemptionPolicy (Sunil G via devaraj)
-
- MAPREDUCE-5972. Fix typo 'programatically' in job.xml (and a few other
- places) (Akira AJISAKA via aw)
-
- MAPREDUCE-6161. mapred hsadmin command missing from trunk (Allen Wittenauer
- via jlowe)
-
- MAPREDUCE-4413. MR lib dir contains jdiff (which is gpl) (Nemon Lou via aw)
-
- MAPREDUCE-6234. TestHighRamJob fails due to the change in MAPREDUCE-5785.
- (Masatake Iwasaki via kasha)
-
- MAPREDUCE-6343. JobConf.parseMaximumHeapSizeMB() fails to parse value
- greater than 2GB expressed in bytes. (Hao Xia via kasha)
-
- MAPREDUCE-6396. TestPipeApplication fails by NullPointerException.
- (Brahma Reddy Battula via aajisaka)
-
- MAPREDUCE-6406. Update FileOutputCommitter.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT
- to match mapred-default.xml. (Ray Chiang via devaraj)
-
- MAPREDUCE-6257. Document encrypted spills (Bibin A Chundatt via aw)
-
- MAPREDUCE-6391. util/Timer.cc completely misunderstands _POSIX_CPUTIME
- (Alan Burlison via aw)
-
- MAPREDUCE-6412. Make hadoop-mapreduce-client Native code -Wall-clean
- (Alan Burlison via aw)
-
- MAPREDUCE-6416. Not all platforms have d_type in struct dirent
- (Alan Burlison via aw)
-
- MAPREDUCE-6435. MapReduce client assumes the world is x86
- (Alan Burlison via aw)
-
- MAPREDUCE-6540. TestMRTimelineEventHandling fails (sjlee)
-
- MAPREDUCE-6555. TestMRAppMaster fails on trunk. (Junping Du via ozawa)
-
- BREAKDOWN OF MAPREDUCE-2841 (NATIVE TASK) SUBTASKS
-
- MAPREDUCE-5985. native-task: Fix build on macosx. Contributed by
- Binglin Chang
-
- MAPREDUCE-5994. Simplify ByteUtils and fix failing test. (todd)
-
- MAPREDUCE-5996. native-task: Rename system tests into standard directory
- layout (todd)
-
- MAPREDUCE-5997. native-task: Use DirectBufferPool from Hadoop Common (todd)
-
- MAPREDUCE-6000. native-task: Simplify ByteBufferDataReader/Writer (todd)
-
- MAPREDUCE-5991. native-task should not run unit tests if native profile is
- not enabled. (Binglin Chang)
-
- MAPREDUCE-5995. native-task: Revert changes to Text internals (todd)
-
- MAPREDUCE-6005. native-task: Fix some valgrind errors (Binglin Chang)
-
- MAPREDUCE-5984. native-task: Reuse lz4 sources in hadoop-common (Binglin
- Chang)
-
- MAPREDUCE-5976. native-task: should not fail to build if snappy is missing
- (Manu Zhang)
-
- MAPREDUCE-5978. native-task: remove test case for not supported codec
- Bzip2Codec and DefaultCodec (Manu Zhang)
-
- MAPREDUCE-6006. native-task: add native tests to maven and fix bug in
- pom.xml (Binglin Chang via todd)
-
- MAPREDUCE-6026. native-task: fix logging (Manu Zhang via todd)
-
- MAPREDUCE-6035. native-task: sources/test-sources jar distribution (Manu
- Zhang via todd)
-
- MAPREDUCE-5977. Fix or suppress native-task gcc warnings (Manu Zhang via
- todd)
-
- MAPREDUCE-6054. native-task: Speed up tests (todd)
-
- MAPREDUCE-6058. native-task: KVTest and LargeKVTest should check mr job is
- sucessful (Binglin Chang)
-
- MAPREDUCE-6056. native-task: move system test working dir to target dir and
- cleanup test config xml files (Manu Zhang via bchang)
-
- MAPREDUCE-6055. native-task: findbugs, interface annotations, and other misc
- cleanup (todd)
-
- MAPREDUCE-6067. native-task: fix some counter issues (Binglin Chang)
-
- MAPREDUCE-6069. native-task: Lint/style fixes and removal of unused code
- (todd)
-
- MAPREDUCE-6074. native-task: fix release audit, javadoc, javac warnings
- (todd)
-
- MAPREDUCE-6077. native-task: Remove CustomModule examples in nativetask (seanzhong)
-
- MAPREDUCE-6078. native-task: fix gtest build on macosx (Binglin Chang)
-
- MAPREDUCE-5801. Uber mode's log message is missing a vcore reason
- (Steven Wong via aw)
-
- MAPREDUCE-6525. Fix test failure of TestMiniMRClientCluster.testRestart.
- (Masatake Iwasaki via aajisaka)
-
-Release 2.9.0 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- MAPREDUCE-6431. JobClient should be an AutoClosable (haibochen via rkanter)
-
- MAPREDUCE-6634. Log uncaught exceptions/errors in various thread pools in
- mapreduce. (Sidharta Seethana via vvasudev)
-
- MAPREDUCE-6627. Add machine-readable output to mapred job -history
- command (rkanter)
-
- MAPREDUCE-6640. mapred job -history command should be able to take
- Job ID (rkanter)
-
- MAPREDUCE-6622. Add capability to set JHS job cache to a
- task-based limit (rchiang via rkanter)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-6620. Jobs that did not start are shown as starting in 1969 in
- the JHS web UI (haibochen via rkanter)
-
- MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter)
-
-Release 2.8.0 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application
- Master REST API. (Ryu Kobayashi via ozawa)
-
- MAPREDUCE-6364. Add a "Kill" link to Task Attempts page. (Ryu Kobayashi
- via ozawa)
-
- MAPREDUCE-6304. Specifying node labels when submitting MR jobs.
- (Naganarasimha G R via wangda)
-
- MAPREDUCE-6415. Create a tool to combine aggregated logs into HAR files.
- (Robert Kanter via kasha)
-
- IMPROVEMENTS
-
- MAPREDUCE-6471. Document distcp incremental copy
- (Neelesh Srinivas Salian via harsh)
-
- MAPREDUCE-5045. UtilTest#isCygwin method appears to be unused
- (Neelesh Srinivas Salian via harsh)
-
- MAPREDUCE-6291. Correct mapred queue usage command.
- (Brahma Reddu Battula via harsh)
-
- MAPREDUCE-579. Streaming "slowmatch" documentation. (harsh)
-
- MAPREDUCE-6287. Deprecated methods in org.apache.hadoop.examples.Sort
- (Chao Zhang via harsh)
-
- MAPREDUCE-5190. Unnecessary condition test in RandomSampler.
- (Jingguo Yao via harsh)
-
- MAPREDUCE-6239. Consolidate TestJobConf classes in
- hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core
- (Varun Saxena via harsh)
-
- MAPREDUCE-5807. Print usage by TeraSort job. (Rohith via harsh)
-
- MAPREDUCE-4653. TestRandomAlgorithm has an unused "import" statement.
- (Amir Sanjar via harsh)
-
- MAPREDUCE-6100. replace "mapreduce.job.credentials.binary" with
- MRJobConfig.MAPREDUCE_JOB_CREDENTIALS_BINARY for better readability.
- (Zhihai Xu via harsh)
-
- MAPREDUCE-6105. Inconsistent configuration in property
- mapreduce.reduce.shuffle.merge.percent. (Ray Chiang via harsh)
-
- MAPREDUCE-4414. Add main methods to JobConf and YarnConfiguration,
- for debug purposes. (Plamen Jeliazkov via harsh)
-
- MAPREDUCE-5755. MapTask.MapOutputBuffer#compare/swap should have
- @Override annotation. (ozawa)
-
- MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in
- CompletedJob#loadFullHistoryData for code optimization.
- (zxu via rkanter)
-
- MAPREDUCE-6307. Remove property
- mapreduce.tasktracker.taskmemorymanager.monitoringinterval.
- (J.Andreina via aajisaka)
-
- MAPREDUCE-6192. Create unit test to automatically compare
- MR related classes and mapred-default.xml (rchiang via rkanter)
-
- MAPREDUCE-6279. AM should explicity exit JVM after all services have
- stopped (Eric Payne via jlowe)
-
- MAPREDUCE-6079. Rename JobImpl#username to reporterUserName.
- (Tsuyoshi Ozawa via aajisaka)
-
- MAPREDUCE-5981. Log levels of certain MR logs can be changed to DEBUG.
- (Varun Saxena via devaraj)
-
- MAPREDUCE-5248. Let NNBenchWithoutMR specify the replication factor for
- its test (Erik Paulson via jlowe)
-
- MAPREDUCE-6174. Combine common stream code into parent class for
- InMemoryMapOutput and OnDiskMapOutput. (Eric Payne via gera)
-
- MAPREDUCE-6383. Pi job (QuasiMonteCarlo) should not try to read the
- results file if its job fails. (Harsh J via devaraj)
-
- MAPREDUCE-6354. ShuffleHandler should be able to log shuffle connections
- (Chang Li via jlowe)
-
- MAPREDUCE-6392. Document mapred class path options.
- (Brahma Reddy Battula via cnauroth)
-
- MAPREDUCE-6395. Improve the commit failure messages in MRAppMaster recovery.
- (Brahma Reddy Battula via gera)
-
- MAPREDUCE-6316. Task Attempt List entries should link to the task overview.
- (Siqi Li via gera)
-
- MAPREDUCE-6305. AM/Task log page should be able to link back to the job.
- (Siqi Li via gera)
-
- MAPREDUCE-6408. Queue name and user name should be printed on the job page.
- (Siqi Li via gera)
-
- MAPREDUCE-6384. Add the last reporting reducer info for too many fetch
- failure diagnostics (Chang Li via jlowe)
-
- MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. (Robert Kanter
- via junping_du)
-
- MAPREDUCE-6477. Replace usage of deprecated NameNode.DEFAULT_PORT in
- TestFileSystem. (Mingliang Liu via wheat9)
-
- MAPREDUCE-6483. Replace deprecated method NameNode.getUri() with
- DFSUtilClient.getNNUri() in TestMRCredentials. (Mingliang Liu via wheat9)
-
- HADOOP-12428. Fix inconsistency between log-level guards and statements.
- (Jagadesh Kiran N and Jackie Chang via ozawa)
-
- MAPREDUCE-6479. Add missing mapred job command options in mapreduce
- document. (nijel via aajisaka)
-
- MAPREDUCE-6489. Fail fast rogue tasks that write too much to local disk
- (Maysam Yabandeh via jlowe)
-
- MAPREDUCE-5763. Warn message about httpshuffle in NM logs.
- (Akira AJISAKA via ozawa)
-
- MAPREDUCE-6499. Add elapsed time for retired job in JobHistoryServer WebUI.
- (Lin Yiqun via aajisaka)
-
- MAPREDUCE-5870. Support for passing Job priority through Application
- Submission Context in Mapreduce Side (Sunil G via jlowe)
-
- MAPREDUCE-6566. Add retry support to mapreduce CLI tool.
- (Varun Vasudev via xgong)
-
- MAPREDUCE-6574. MR AM should print host of failed tasks.
- (Mohammad Shahid Khan via wangda)
-
- MAPREDUCE-6584. Remove trailing whitespaces from mapred-default.xml.
- (aajisaka)
-
- MAPREDUCE-6595. Fix findbugs warnings in OutputCommitter and
- FileOutputCommitter. (Akira AJISAKA via junping_du)
-
- OPTIMIZATIONS
-
- MAPREDUCE-6376. Add avro binary support for jhist files (Ray Chiang via
- jlowe)
-
- MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render()
- (Ray Chiang via jlowe)
-
- MAPREDUCE-6473. Job submission can take a long time during Cluster
- initialization (Kuhu Shukla via jlowe)
-
- MAPREDUCE-6626. Reuse ObjectMapper instance in MapReduce.
- (Lin Yiqun via aajisaka)
-
- BUG FIXES
-
- MAPREDUCE-6314. TestPipeApplication fails on trunk.
- (Varun Vasudev via harsh)
-
- MAPREDUCE-5448. MapFileOutputFormat#getReaders bug with hidden
- files/folders. (Maysam Yabandeh via harsh)
-
- MAPREDUCE-6213. NullPointerException caused by job history server addr not
- resolvable. (Peng Zhang via harsh)
-
- MAPREDUCE-6281. Fix javadoc in Terasort. (Albert Chu via ozawa)
-
- MAPREDUCE-6242. Progress report log is incredibly excessive in
- application master. (Varun Saxena via devaraj)
-
- MAPREDUCE-6294. Remove an extra parameter described in Javadoc of
- TockenCache. (Brahma Reddy Battula via ozawa)
-
- MAPREDUCE-5875. Make Counter limits consistent across JobClient,
- MRAppMaster, and YarnChild. (Gera Shegalov via kasha)
-
- MAPREDUCE-6199. AbstractCounters are not reset completely on
- deserialization (adhoot via rkanter)
-
- MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which
- causes counter limits are not reset correctly.
- (Zhihai Xu via harsh)
-
- MAPREDUCE-4844. Counters / AbstractCounters have constant references not
- declared final. (Brahma Reddy Battula via gera)
-
- MAPREDUCE-6266. Job#getTrackingURL should consistently return a proper URL
- (rchiang via rkanter)
-
- MAPREDUCE-6293. Set job classloader on uber-job's LocalContainerLauncher
- event thread. (Sangjin Lee via gera)
-
- MAPREDUCE-6297. Task Id of the failed task in diagnostics should link to
- the task page. (Siqi Li via gera)
-
- MAPREDUCE-6330. Fix typo in Task Attempt API's URL in documentations.
- (Ryu Kobayashi via ozawa)
-
- MAPREDUCE-6333. TestEvents,TestAMWebServicesTasks,TestAppController are
- broken due to MAPREDUCE-6297. (Siqi Li via gera)
-
- MAPREDUCE-6341. Fix typo in mapreduce tutorial. (John Michael Luy
- via ozawa)
-
- MAPREDUCE-6345. Documentation fix for when CRLA is enabled for MRAppMaster
- logs. (Rohit Agarwal via gera)
-
- MAPREDUCE-5905. CountersStrings.toEscapedCompactStrings outputs
- unnecessary "null" strings. (Akira AJISAKA via ozawa)
-
- MAPREDUCE-6349. Fix typo in property org.apache.hadoop.mapreduce.
- lib.chain.Chain.REDUCER_INPUT_VALUE_CLASS. (Ray Chiang via ozawa)
-
- MAPREDUCE-6165. [JDK8] TestCombineFileInputFormat failed on JDK8.
- (Akira AJISAKA via ozawa)
-
- MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests.
- (Brahma Reddy Battula via aajisaka)
-
- MAPREDUCE-6342. Make POM project names consistent. (Rohith via devaraj)
-
- MAPREDUCE-3383. Duplicate job.getOutputValueGroupingComparator() in
- ReduceTask (Binglin Chang via jlowe)
-
- MAPREDUCE-4750. Enable NNBenchWithoutMR in MapredTestDriver (Liang Xie
- and Jason Lowe via raviprak)
-
- MAPREDUCE-6359. In RM HA setup, "Cluster" tab links populated with AM
- hostname instead of RM. (zhaoyunjiong via junping_du)
-
- MAPREDUCE-6353. Divide by zero error in MR AM when calculating available
- containers. (Anubhav Dhoot via kasha)
-
- MAPREDUCE-5465. Tasks are often killed before they exit on their own
- (Ming Ma via jlowe)
-
- MAPREDUCE-6360. TestMapreduceConfigFields is placed in wrong dir,
- introducing compile error (Arshad Mohammad via vinayakumarb)
-
- MAPREDUCE-6366. mapreduce.terasort.final.sync configuration in TeraSort
- doesn't work. (Takuya Fukudome via ozawa)
-
- MAPREDUCE-5708. Duplicate String.format in YarnOutputFiles.getSpillFileForWrite.
- (Konstantin Weitz via devaraj)
-
- MAPREDUCE-6204. TestJobCounters should use new properties instead of
- JobConf.MAPRED_TASK_JAVA_OPTS. (Sam Liu via ozawa)
-
- MAPREDUCE-6374. Distributed Cache File visibility should check permission
- of full path (Chang Li via jlowe)
-
- MAPREDUCE-5965. Hadoop streaming throws error if list of input files is
- high. Error is: "error=7, Argument list too long at if number of input
- file is high" (wilfreds via rkanter)
-
- MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview.
- (Siqi Li via gera)
-
- MAPREDUCE-6388. Remove deprecation warnings from JobHistoryServer classes
- (Ray Chiang via ozawa).
-
- MAPREDUCE-6350. JobHistory doesn't support fully-functional search.
- (Siqi Li via devaraj)
-
- MAPREDUCE-6389. Fix BaileyBorweinPlouffe CLI usage message. (Brahma Reddy Battula
- via devaraj)
-
- MAPREDUCE-6373. The logger reports total input paths but it is referring
- to input files. (Bibin A Chundatt via devaraj)
-
- MAPREDUCE-6405. NullPointerException in App Attempts page.
- (Siqi Li and Gera Shegalov via devaraj)
-
- MAPREDUCE-6403. Fix typo in the usage of NNBench.
- (Jagadesh Kiran N via aajisaka)
-
- MAPREDUCE-6400. Multiple shuffle transfer fails because input is closed
- too early (Brahma Reddy Battula, Akira AJISAKA, and Gera Shegalov via
- jlowe)
-
- MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host
- (zhihai xu via jlowe)
-
- MAPREDUCE-6121. JobResourceUpdater#compareFs() doesn't handle HA namespaces
- (rchiang via rkanter)
-
- MAPREDUCE-6420. Interrupted Exception in LocalContainerLauncher should be
- logged in warn/info level (Chang Li via jlowe)
-
- MAPREDUCE-6418. MRApp should not shutdown LogManager during shutdown
- (Chang Li via jlowe)
-
- MAPREDUCE-6038. A boolean may be set error in the Word Count v2.0 in
- MapReduce Tutorial. (Tsuyoshi Ozawa via cdouglas)
-
- MAPREDUCE-6421. Fix findbugs warning in
- RMContainerAllocator.reduceNodeLabelExpression.
- (Brahma Reddy Battula via aajisaka)
-
- MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. (Ray Chiang via cdouglas)
-
- MAPREDUCE-6433. launchTime may be negative. (Zhihai Xu)
-
- MAPREDUCE-5817. Mappers get rescheduled on node transition even after all
- reducers are completed. (Sangjin Lee via kasha)
-
- MAPREDUCE-6357. MultipleOutputs.write() API should document that output
- committing is not utilized when input path is absolute.
- (Dustin Cote via aajisaka)
-
- MAPREDUCE-6357. MultipleOutputs.write() API should document that output
- committing is not utilized when input path is absolute.
- (Dustin Cote via aajisaka)
-
- MAPREDUCE-6452. NPE when intermediate encrypt enabled for LocalRunner.
- (Zhihai Xu)
-
- MAPREDUCE-5002. AM could potentially allocate a reduce container to a map
- attempt (Chang Li via jlowe)
-
- MAPREDUCE-6460. TestRMContainerAllocator.
- testAttemptNotFoundCausesRMCommunicatorException fails. (Zhihai Xu)
-
- MAPREDUCE-6484. Yarn Client uses local address instead of RM address as
- token renewer in a secure cluster when RM HA is enabled. (Zhihai Xu)
-
- MAPREDUCE-6480. archive-logs tool may miss applications (rkanter)
-
- MAPREDUCE-6494. Permission issue when running archive-logs tool as
- different users (rkanter)
-
- MAPREDUCE-6485. Create a new task attempt with failed map task priority
- if in-progress attempts are unassigned. (Xianyin Xin via rohithsharmaks)
-
- MAPREDUCE-6503. archive-logs tool should use HADOOP_PREFIX instead
- of HADOOP_HOME (rkanter)
-
- MAPREDUCE-6302. Preempt reducers after a configurable timeout irrespective
- of headroom. (kasha)
-
- MAPREDUCE-6495. Docs for archive-logs tool (rkanter)
-
- MAPREDUCE-6508. TestNetworkedJob fails consistently due to delegation
- token changes on RM. (Akira AJISAKA via junping_du)
-
- MAPREDUCE-6515. Update Application priority in AM side from AM-RM heartbeat
- (Sunil G via jlowe)
-
- MAPREDUCE-6533. testDetermineCacheVisibilities of
- TestClientDistributedCacheManager is broken (Chang Li via jlowe)
-
- MAPREDUCE-6553. Replace '\u2b05' with '<-' in rendering job configuration.
- (Gabor Liptak via aajisaka)
-
- MAPREDUCE-6557. Tests in mapreduce-client-app are writing outside of
- target. (Akira AJISAKA via junping_du)
-
- MAPREDUCE-6550. archive-logs tool changes log ownership to the Yarn
- user when using DefaultContainerExecutor (rkanter)
-
- MAPREDUCE-6419. JobHistoryServer doesn't sort properly based on Job ID when Job id's
- exceed 9999. (Mohammad Shahid Khan via rohithsharmaks)
-
- MAPREDUCE-6589. TestTaskLog outputs a log under directory other than
- target/test-dir. (aajisaka)
-
- MAPREDUCE-6593. TestJobHistoryEventHandler.testTimelineEventHandling fails
- on trunk because of NPE. (Naganarasimha G R via aajisaka)
-
- MAPREDUCE-6068. Illegal progress value warnings in map tasks. (Binglin Chang
- via junping_du)
-
- MAPREDUCE-6601. Fix typo in Job#setUseNewAPI. (Kai Sasaki via aajisaka)
-
- MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and
- mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml.
- (Kai Sasaki via aajisaka)
-
- MAPREDUCE-6614. Remove unnecessary code in TestMapreduceConfigFields.
- (Kai Sasaki via aajisaka)
-
- MAPREDUCE-6610. JobHistoryEventHandler should not swallow timeline response
- (Li Lu via jianhe)
-
- MAPREDUCE-6563. Streaming documentation contains a stray '%' character.
- (cnauroth)
-
- MAPREDUCE-6616. Fail to create jobhistory file if there are some multibyte
- characters in the job name. (Kousuke Saruta via aajisaka)
-
- MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader
- and IndexOutOfBoundsException. (Junping Du via vvasudev)
-
-Release 2.7.3 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- MAPREDUCE-6637. Testcase Failure : TestFileInputFormat.testSplitLocationInfo.
- (Brahma Reddy Battula via wang)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-6377. JHS sorting on state column not working in webUi.
- (zhihai xu via devaraj)
-
- MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly
- misleading (Nathan Roberts via jlowe)
-
- MAPREDUCE-6583. Clarify confusing sentence in MapReduce tutorial document.
- (Kai Sasaki via aajisaka)
-
- MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in
- MRAppMaster#parsePreviousJobHistory (Bibin A Chundatt via jlowe)
-
- MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread.
- (Xuan Gong via jlowe)
-
- MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal() (Xuan Gong
- via jlowe)
-
- MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host
- (zhihai xu via jlowe)
-
- MAPREDUCE-6191. Improve clearing stale state of Java serialization
- testcase. (Sam Liu via Eric Yang)
-
- MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader
- and IndexOutOfBoundsException. (Junping Du via vvasudev)
-
-Release 2.7.2 - 2016-01-25
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to getMapOutputInfo
- if mapId is not in the cache. (zhihai xu via devaraj)
-
- MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing.
- (zhihai xu via devaraj)
-
- MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the
- allocate call. (Anubhav Dhoot via kasha)
-
- MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol
- provider's constructor (Chang Li via ozawa)
-
- MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file
- descriptors (Kuhu Shukla via jlowe)
-
- MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be
- consistent with tasks (Naganarasimha G R via jlowe)
-
- MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle
- multibyte record delimiters well (Vinayakumar B, Rushabh Shah, and Akira
- AJISAKA via jlowe)
-
- MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong
- position/key information for uncompressed input sometimes. (Zhihai Xu via
- jlowe)
-
- MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can
- disappear (Chang Li via jlowe)
-
- MAPREDUCE-6492. AsyncDispatcher exit with NPE on
- TaskAttemptImpl#sendJHStartEventForAssignedFailTask (Bibin A Chundatt via
- jlowe)
-
- MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections (Chang Li via
- jlowe)
-
- MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists to
- avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state.
- (zhihai xu via devaraj)
-
- MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary()
- (Junping Du via jlowe)
-
- MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs when
- strategy is dynamic (Kuhu Shukla via kihwal)
-
-Release 2.7.1 - 2015-07-06
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-6252. JobHistoryServer should not fail when encountering a
- missing directory. (Craig Welch via devaraj)
-
- MAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon
- IOException during InMemoryMapOutput shuffle handler (Eric Payne via jlowe)
-
- MAPREDUCE-6339. Job history file is not flushed correctly because isTimerActive
- flag is not set true when flushTimerTask is scheduled. (zhihai xu via devaraj)
-
- MAPREDUCE-6259. IllegalArgumentException due to missing job submit time
- (zhihai xu via jlowe)
-
- MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so
- that they don't fail on history-server backed by DFSes with not so strong
- guarantees. (Craig Welch via vinodkv)
-
- MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at
- the end. (Arun Suresh via kasha)
-
- MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login)
- UGI to refresh log and cleaner settings. (Varun Saxena via vinodkv)
-
-Release 2.7.0 - 2015-04-20
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- MAPREDUCE-6227. DFSIO for truncate. (shv via yliu)
-
- MAPREDUCE-6228. Add truncate operation to SLive. (Plamen Jeliazkov via shv)
-
- MAPREDUCE-5583. Ability to limit running map and reduce tasks.
- (Jason Lowe via junping_du)
-
- IMPROVEMENTS
-
- MAPREDUCE-6149. Document override log4j.properties in MR job.
- (Junping Du via harsh)
-
- MAPREDUCE-6194. Bubble up final exception in failures during creation
- of output collectors (Varun Saxena via harsh)
-
- MAPREDUCE-5420. Remove mapreduce.task.tmp.dir from mapred-default.xml
- (James Carman via harsh)
-
- MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle
- log (Gera Shegalov via jlowe)
-
- MAPREDUCE-6046. Change the class name for logs in RMCommunicator
- (Sahil Takiar via devaraj)
-
- HADOOP-11032. Replace use of Guava's Stopwatch with Hadoop's StopWatch
- (ozawa)
-
- MAPREDUCE-6173. Document the configuration of deploying MR over distributed
- cache with enabling wired encryption at the same time.
- (Junping Du via xgong)
-
- MAPREDUCE-6141. History server leveldb recovery store (jlowe)
-
- MAPREDUCE-6150. Update document of Rumen (Masatake Iwasaki via aw)
-
- MAPREDUCE-6151. Update document of GridMix (Masatake Iwasaki via aw)
-
- MAPREDUCE-6143. add configuration for mapreduce speculative execution in
- MR2 (zxu via rkanter)
-
- MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors
- (aajisaka)
-
- MAPREDUCE-6253. Update use of Iterator to Iterable. (Ray Chiang via devaraj)
-
- MAPREDUCE-5335. Rename Job Tracker terminology in ShuffleSchedulerImpl.
- (devaraj via ozawa)
-
- MAPREDUCE-4431. mapred command should print the reason on killing already
- completed jobs. (devaraj via ozawa)
-
- MAPREDUCE-6255. Fix JobCounter's format to use grouping separator.
- (Ryu Kobayashi via ozawa)
-
- MAPREDUCE-6256. Removed unused private methods in o.a.h.mapreduce.Job.java.
- (Naganarasimha G R via ozawa)
-
- MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core.
- (Varun Saxena via junping_du)
-
- MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client.
- (Brahma Reddy Battula via aajisaka)
-
- MAPREDUCE-5612. Add javadoc for TaskCompletionEvent.Status.
- (Chris Palmer via aajisaka)
-
- MAPREDUCE-6248. Exposed the internal MapReduce job's information as a public
- API in DistCp. (Jing Zhao via vinodkv)
-
- MAPREDUCE-6263. Configurable timeout between YARNRunner terminate the
- application and forcefully kill. (Eric Payne via junping_du)
-
- MAPREDUCE-6265. Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable
- to better control to launch/kill containers. (Zhihai Xu via ozawa)
-
- MAPREDUCE-6292. Use org.junit package instead of junit.framework in
- TestCombineFileInputFormat. (aajisaka)
-
- OPTIMIZATIONS
-
- MAPREDUCE-6169. MergeQueue should release reference to the current item
- from key and value at the end of the iteration to save memory.
- (Zhihai Xu via kasha)
-
- MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw)
-
- MAPREDUCE-4815. Speed up FileOutputCommitter#commitJob for many output
- files. (Siqi Li via gera)
-
- BUG FIXES
-
- MAPREDUCE-6210. Use getApplicationAttemptId() instead of getApplicationId()
- for logging AttemptId in RMContainerAllocator.java (Leitao Guo via aajisaka)
-
- MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about
- ssl-client.xml (Yangping Wu via harsh)
-
- MAPREDUCE-5918. LineRecordReader can return the same decompressor to
- CodecPool multiple times (Sergey Murylev via raviprak)
-
- MAPREDUCE-6162. mapred hsadmin fails on a secure cluster (jlowe)
-
- MAPREDUCE-6049. AM JVM does not exit if MRClientService graceful shutdown
- fails (Rohith via devaraj)
-
- MAPREDUCE-5568. Fixed CompletedJob in JHS to show progress percentage
- correctly in case the number of mappers or reducers is zero. (MinJi Kim via
- jianhe)
-
- MAPREDUCE-6172. TestDbClasses timeouts are too aggressive (Varun Saxena
- via jlowe)
-
- MAPREDUCE-6160. Potential NullPointerException in MRClientProtocol
- interface implementation. (Rohith via jlowe)
-
- MAPREDUCE-4879. TeraOutputFormat may overwrite an existing output
- directory. (gera)
-
- MAPREDUCE-6045. need close the DataInputStream after open it in
- TestMapReduce.java (zxu via rkanter)
-
- MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems (Jens
- Rabe via jlowe)
-
- MAPREDUCE-3283. mapred classpath CLI does not display the complete classpath
- (Varun Saxena via cnauroth)
-
- MAPREDUCE-6231. Grep example job is not working on a fully-distributed
- cluster. (aajisaka)
-
- MAPREDUCE-6243. Fix findbugs warnings in hadoop-rumen. (Masatake Iwasaki
- via aajisaka)
-
- MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project.
- (aajisaka)
-
- MAPREDUCE-6186. Redundant call to requireJob() while displaying the conf
- page (Rohit Agarwal via jlowe)
-
- MAPREDUCE-6233. org.apache.hadoop.mapreduce.TestLargeSort.testLargeSort
- failed in trunk (zxu via rkanter)
-
- MAPREDUCE-6221. Stringifier is left unclosed in Chain#getChainElementConf().
- (Ted Yu via ozawa)
-
- MAPREDUCE-4286. TestClientProtocolProviderImpls passes on failure
- conditions. (Devaraj K via ozawa)
-
- MAPREDUCE-6261. NullPointerException if MapOutputBuffer.flush invoked
- twice (Tsuyoshi OZAWA via jlowe)
-
- MAPREDUCE-2815. JavaDoc does not generate correctly for
- MultithreadedMapRunner. (Chris Palmer via aajisaka)
-
- MAPREDUCE-6223. TestJobConf#testNegativeValueForTaskVmem failures.
- (Varun Saxena via kasha)
-
- MAPREDUCE-6268. Fix typo in Task Attempt API's URL. (Ryu Kobayashi
- via ozawa)
-
- MAPREDUCE-6136. MRAppMaster doesn't shutdown file systems. (Brahma
- Reddy Battula via ozawa)
-
- MAPREDUCE-5657. Fix Javadoc errors caused by incorrect or illegal tags in doc
- comments. (Akira AJISAKA and Andrew Purtell via ozawa)
-
- MAPREDUCE-4742. Fix typo in nnbench#displayUsage. (Liang Xie via ozawa)
-
- MAPREDUCE-6277. Job can post multiple history files if attempt loses
- connection to the RM (Chang Li via jlowe)
-
- MAPREDUCE-6275. Race condition in FileOutputCommitter v2 for
- user-specified task output subdirs (Gera Shegalov and Siqi Li via jlowe)
-
- MAPREDUCE-6285. ClientServiceDelegate should not retry upon
- AuthenticationException. (Jonathan Eagles via ozawa)
-
-Release 2.6.5 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host
- (zhihai xu via jlowe)
-
- MAPREDUCE-6191. Improve clearing stale state of Java serialization
- testcase. (Sam Liu via Eric Yang)
-
- MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader
- and IndexOutOfBoundsException. (Junping Du via vvasudev)
-
-Release 2.6.4 - 2016-02-11
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- MAPREDUCE-6436. JobHistory cache issue. (Kai Sasaki via zxu)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-6577. MR AM unable to load native library without
- MR_AM_ADMIN_USER_ENV set (sjlee)
-
- MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can
- disappear (Chang Li via jlowe)
-
- MAPREDUCE-6492. AsyncDispatcher exit with NPE on
- TaskAttemptImpl#sendJHStartEventForAssignedFailTask (Bibin A Chundatt via
- jlowe)
-
- MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple
- mappers. (Vlad Sharanhovich and Bibin A Chundatt via aajisaka)
-
- MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in
- MRAppMaster#parsePreviousJobHistory (Bibin A Chundatt via jlowe)
-
- MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread.
- (Xuan Gong via jlowe)
-
- MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal() (Xuan Gong
- via jlowe)
-
-Release 2.6.3 - 2015-12-17
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists to
- avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state.
- (zhihai xu via devaraj)
-
- MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary()
- (Junping Du via jlowe)
-
- MAPREDUCE-6377. JHS sorting on state column not working in webUi.
- (zhihai xu via devaraj)
-
- MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle
- multibyte record delimiters well (Vinayakumar B, Rushabh Shah, and Akira
- AJISAKA via jlowe)
-
- MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong
- position/key information for uncompressed input sometimes. (Zhihai Xu via
- jlowe)
-
- MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly
- misleading (Nathan Roberts via jlowe)
-
- MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause
- duplicate records (wilfreds via rkanter)
-
-Release 2.6.2 - 2015-10-28
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed
- cache files so that child processes running hadoop scripts can access these
- files. (Junping Du via vinodkv)
-
- MAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon
- IOException during InMemoryMapOutput shuffle handler (Eric Payne via jlowe)
-
- MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in
- JobHistoryEventHandler. (Shinichi Yamashita via aajisaka)
-
-Release 2.6.1 - 2015-09-23
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of
- reusing conections. (Kannan Rajah via ozawa)
-
- MAPREDUCE-6166. Reducers do not validate checksum of map outputs when
- fetching directly to disk. (Eric Payne via gera)
-
- MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken
- service name properly. (Jason Lowe via jianhe)
-
- MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal
- to a reducer. (Jason Lowe via junping_du)
-
- MAPREDUCE-6300. Task list sort by task id broken. (Siqi Li via aajisaka)
-
- MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own
- class. (Chris Trezzo via kasha)
-
- MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options
- which is a regression from MR1 (zxu via rkanter)
-
- MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM
- tokens when they roll-over. (Jason Lowe via vinodkv)
-
- MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge
- (Gera Shegalov via jlowe)
-
- MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between
- copySucceeded() in one thread and copyFailed() in another thread on the
- same host. (Junping Du via ozawa)
-
-Release 2.6.0 - 2014-11-18
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- MAPREDUCE-5933. Enabled MR AM to post history events to the timeline server.
- (Robert Kanter via zjshen)
-
- IMPROVEMENTS
-
- MAPREDUCE-5971. Move the default options for distcp -p to
- DistCpOptionSwitch. (clamb via wang)
-
- MAPREDUCE-5963. ShuffleHandler DB schema should be versioned with
- compatible/incompatible changes (Junping Du via jlowe)
-
- MAPREDUCE-883. harchive: Document how to unarchive (Akira AJISAKA and
- Koji Noguchi via aw)
-
- MAPREDUCE-4791. Javadoc for KeyValueTextInputFormat should include default
- separator and how to change it (Akira AJISAKA via aw)
-
- MAPREDUCE-5906. Inconsistent configuration in property
- "mapreduce.reduce.shuffle.input.buffer.percent" (Akira AJISAKA via aw)
-
- MAPREDUCE-5974. Allow specifying multiple MapOutputCollectors with
- fallback. (Todd Lipcon via kasha)
-
- MAPREDUCE-5130. Add missing job config options to mapred-default.xml
- (Ray Chiang via Sandy Ryza)
-
- MAPREDUCE-5891. Improved shuffle error handling across NM restarts
- (Junping Du via jlowe)
-
- MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN
- scheduler resource type is memory plus cpu. (Peng Zhang and Varun Vasudev
- via zjshen)
-
- MAPREDUCE-6072. Remove INSTALL document (Akira AJISAKA via aw)
-
- MAPREDUCE-5970. Provide a boolean switch to enable MR-AM profiling (Gera
- Shegalov via jlowe)
-
- MAPREDUCE-6018. Added an MR specific config to enable emitting job history
- data to the timeline server. (Robert Kanter via zjshen)
-
- MAPREDUCE-6052. Supported overriding the default container-log4j.properties
- file per job. (Junping Du via zjshen)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-5866. TestFixedLengthInputFormat fails in windows.
- (Varun Vasudev via cnauroth)
-
- MAPREDUCE-5956. Made MR AM not use maxAttempts to determine if the current
- attempt is the last retry. (Wangda Tan via zjshen)
-
- MAPREDUCE-5957. AM throws ClassNotFoundException with job classloader
- enabled if custom output format/committer is used (Sangjin Lee via jlowe)
-
- MAPREDUCE-5756. CombineFileInputFormat.getSplits() including directories
- in its results (Jason Dere via jlowe)
-
- MAPREDUCE-6014. New task status field in task attempts table can lead to
- an empty web page (Mit Desai via jlowe)
-
- MAPREDUCE-6021. MR AM should have working directory in LD_LIBRARY_PATH
- (jlowe)
-
- MAPREDUCE-6010. HistoryServerFileSystemStateStore fails to update tokens
- (jlowe)
-
- MAPREDUCE-5878. some standard JDK APIs are not part of system classes
- defaults (Sangjin Lee via jlowe)
-
- MAPREDUCE-5944. Remove MRv1 commands from CommandsManual.apt.vm
- (Akira AJISAKA via aw)
-
- MAPREDUCE-5943. Separate mapred commands from CommandManual.apt.vm
- (Akira AJISAKA via aw)
-
- MAPREDUCE-5363. Fix doc and spelling for TaskCompletionEvent#getTaskStatus
- and getStatus (Akira AJISAKA via aw)
-
- MAPREDUCE-5595. Typo in MergeManagerImpl.java (Akira AJISAKA via aw)
-
- MAPREDUCE-5597. Missing alternatives in javadocs for deprecated constructors
- in mapreduce.Job (Akira AJISAKA via aw)
-
- MAPREDUCE-5950. incorrect description in distcp2 document (Akira AJISAKA
- via aw)
-
- MAPREDUCE-5998. CompositeInputFormat javadoc is broken (Akira AJISAKA via
- aw)
-
- MAPREDUCE-5999. Fix dead link in InputFormat javadoc (Akira AJISAKA via aw)
-
- MAPREDUCE-6032. Made MR jobs write job history files on the default FS when
- the current context's FS is different. (Benjamin Zhitomirsky via zjshen)
-
- MAPREDUCE-6024. Shortened the time when Fetcher is stuck in retrying before
- concluding the failure by configuration. (Yunjiong Zhao via zjshen)
-
- MAPREDUCE-6036. TestJobEndNotifier fails intermittently in branch-2 (chang
- li via jlowe)
-
- MAPREDUCE-6012. DBInputSplit creates invalid ranges on Oracle.
- (Wei Yan via kasha)
-
- MAPREDUCE-6044. Fully qualified intermediate done dir path breaks per-user dir
- creation on Windows. (zjshen)
-
- MAPREDUCE-5885. build/test/test.mapred.spill causes release audit warnings
- (Chen He via jlowe)
-
- BREAKDOWN OF HDFS-6134 AND HADOOP-10150 SUBTASKS AND RELATED JIRAS
-
- MAPREDUCE-5890. Support for encrypting Intermediate
- data and spills in local filesystem. (asuresh via tucu)
-
- MAPREDUCE-6007. Add support to distcp to preserve raw.* namespace
- extended attributes. (clamb)
-
- MAPREDUCE-6041. Fix TestOptionsParser. (clamb)
- --
-
- MAPREDUCE-6051. Fix typos in log messages. (Ray Chiang via cdouglas)
-
- MAPREDUCE-5931. Validate SleepJob command line parameters (Gera Shegalov
- via jlowe)
-
- MAPREDUCE-6063. Correct spill size calculation for spills wrapping the
- circular buffer. (zhihai xu via cdouglas)
-
- MAPREDUCE-6071. JobImpl#makeUberDecision doesn't log that Uber mode is
- disabled because of too much CPUs (Tsuyoshi OZAWA via jlowe)
-
- MAPREDUCE-6075. HistoryServerFileSystemStateStore can create zero-length
- files (jlowe)
-
- MAPREDUCE-6070. yarn.app.am.resource.mb/cpu-vcores affects uber mode but
- is not documented (Tsuyoshi OZAWA via jlowe)
-
- MAPREDUCE-6090. mapred hsadmin getGroups fails to connect in some cases
- (Robert Kanter via jlowe)
-
- MAPREDUCE-6086. mapreduce.job.credentials.binary should allow all URIs.
- (Zhihai Xu via kasha)
-
- MAPREDUCE-6091. YARNRunner.getJobStatus() fails with
- ApplicationNotFoundException if the job rolled off the RM view (Sangjin
- Lee via jlowe)
-
- MAPREDUCE-6095. Enable DistributedCache for uber-mode Jobs (Gera Shegalov
- via jlowe)
-
- MAPREDUCE-6104. TestJobHistoryParsing.testPartialJob fails in branch-2
- (Mit Desai via jlowe)
-
- MAPREDUCE-6109. Fix minor typo in distcp -p usage text (Charles Lamb
- via aw)
-
- MAPREDUCE-6093. minor distcp doc edits (Charles Lamb via aw)
-
- MAPREDUCE-5831. Make MR client ignore unknown counters received from AM.
- (Junping Du via zjshen)
-
- MAPREDUCE-6073. Description of mapreduce.job.speculative.slowtaskthreshold
- in mapred-default should be moved into description tags (Tsuyoshi OZAWA
- via aw)
-
- MAPREDUCE-5796. Use current version of the archive name in
- DistributedCacheDeploy document (Akira AJISAKA via aw)
-
- MAPREDUCE-5945. Update the description of GenericOptionsParser -jt
- option (Akira AJISAKA via aw)
-
- MAPREDUCE-6087. Fixed wrong config name of
- MRJobConfig#MR_CLIENT_TO_AM_IPC_MAX_RETRIES_ON_TIMEOUTS. Contributed by
- Akira AJISAKA. (Akira AJISAKA via jianhe)
-
- MAPREDUCE-6094. TestMRCJCFileInputFormat.testAddInputPath() fails on trunk
- (Akira AJISAKA via jlowe)
-
- MAPREDUCE-6029. TestCommitterEventHandler fails in trunk (Mit Desai via
- jlowe)
-
- MAPREDUCE-6122. TestLineRecordReader may fail due to test data files checked
- out of git with incorrect line endings. (cnauroth)
-
- MAPREDUCE-6123. TestCombineFileInputFormat incorrectly starts 2
- MiniDFSCluster instances. (cnauroth)
-
- MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails (Mit Desai via
- jlowe)
-
- MAPREDUCE-6115. TestPipeApplication#testSubmitter fails in trunk (Binglin
- Chang via jlowe)
-
- MAPREDUCE-5873. Shuffle bandwidth computation includes time spent waiting
- for maps (Siqi Li via jlowe)
-
- MAPREDUCE-5542. Killing a job just as it finishes can generate an NPE in
- client (Rohith via jlowe)
-
- MAPREDUCE-6126. Fixed Rumen JobBuilder to ignore NormalizedResourceEvent.
- (Junping Du via jianhe)
-
- MAPREDUCE-6142. Fixed test failures in TestJobHistoryEventHandler and
- TestMRTimelineEventHandling. (Zhijie Shen via vinodkv)
-
- MAPREDUCE-6022. map_input_file is missing from streaming job environment.
- (jlowe via kihwal)
-
- MAPREDUCE-6048. Fixed TestJavaSerialization failure. (Varun Vasudev via
- jianhe)
-
- MAPREDUCE-5960. JobSubmitter's check whether job.jar is local is incorrect
- with no authority in job jar path. (Gera Shegalov via jlowe)
-
- MAPREDUCE-5958. Wrong reduce task progress if map output is compressed
- (Emilio Coppa and jlowe via kihwal)
-
- MAPREDUCE-6156. Fetcher - connect() doesn't handle connection refused
- correctly (Junping Du via jlowe)
-
-Release 2.5.2 - 2014-11-19
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
-
-Release 2.5.1 - 2014-09-05
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-6033. Updated access check for displaying job information
- (Yu Gao via Eric Yang)
-
-Release 2.5.0 - 2014-08-11
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- MAPREDUCE-5671. NaN can be created by client and assign to Progress (Chen
- He via jeagles)
-
- MAPREDUCE-5665. Add audience annotations to MiniMRYarnCluster and
- MiniMRCluster. (Anubhav Dhoot via kasha)
-
- MAPREDUCE-5765. Update hadoop-pipes examples README (Mit Desai via jeagles)
-
- MAPREDUCE-5713. InputFormat and JobConf JavaDoc Fixes (Chen He via jeagles)
-
- MAPREDUCE-5456. TestFetcher.testCopyFromHostExtraBytes is missing (Jason
- Lowe via jeagles)
-
- MAPREDUCE-5804. TestMRJobsWithProfiler#testProfiler timesout (Mit Desai
- via kihwal)
-
- MAPREDUCE-5825. Provide diagnostics for reducers killed during ramp down
- (Gera Shegalov via jlowe)
-
- MAPREDUCE-5836. Fix typo in RandomTextWriter (Akira AJISAKA via jeagles)
-
- MAPREDUCE-5852. Prepare MapReduce codebase for JUnit 4.11. (cnauroth)
-
- MAPREDUCE-5639. Port DistCp2 document to trunk (Akira AJISAKA via jeagles)
-
- MAPREDUCE-5812. Make job context available to
- OutputCommitter.isRecoverySupported() (Mohammad Kamrul Islam via jlowe)
-
- MAPREDUCE-5638. Port Hadoop Archives document to trunk (Akira AJISAKA via
- jeagles)
-
- MAPREDUCE-5402. In DynamicInputFormat, change MAX_CHUNKS_TOLERABLE,
- MAX_CHUNKS_IDEAL, MIN_RECORDS_PER_CHUNK and SPLIT_RATIO to be configurable.
- (Tsuyoshi OZAWA via szetszwo)
-
- MAPREDUCE-5637. Convert Hadoop Streaming document to APT (Akira AJISAKA via
- jeagles)
-
- MAPREDUCE-5636. Convert MapReduce Tutorial document to APT (Akira AJISAKA
- via jeagles)
-
- MAPREDUCE-5774. Job overview in History UI should list reducer phases in
- chronological order. (Gera Shegalov via kasha)
-
- MAPREDUCE-5652. NM Recovery. ShuffleHandler should handle NM restarts.
- (Jason Lowe via kasha)
-
- MAPREDUCE-5861. finishedSubMaps field in LocalContainerLauncher does not
- need to be volatile. (Tsuyoshi OZAWA via junping_du)
-
- MAPREDUCE-5809. Enhance distcp to support preserving HDFS ACLs. (cnauroth)
-
- MAPREDUCE-5899. Support incremental data copy in DistCp. (jing9)
-
- MAPREDUCE-5886. Allow wordcount example job to accept multiple input paths.
- (cnauroth)
-
- MAPREDUCE-5834. Increased test-timeouts in TestGridMixClasses to avoid
- occassional failures. (Mit Desai via vinodkv)
-
- MAPREDUCE-5896. InputSplits should indicate which locations have the block
- cached in memory. (Sandy Ryza via kasha)
-
- MAPREDUCE-5844. Add a configurable delay to reducer-preemption.
- (Maysam Yabandeh via kasha)
-
- MAPREDUCE-5790. Made it easier to enable hprof profile options by default.
- (Gera Shegalov via vinodkv)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-5759. Remove unnecessary conf load in Limits (Sandy Ryza)
-
- MAPREDUCE-5014. Extend Distcp to accept a custom CopyListing.
- (Srikanth Sundarrajan via amareshwari)
-
- MAPREDUCE-5775. Remove unnecessary job.setNumReduceTasks in SleepJob.createJob
- (jhanver chand sharma via devaraj)
-
- MAPREDUCE-4937. MR AM handles an oversized split metainfo file poorly
- (Eric Payne via jlowe)
-
- MAPREDUCE-5642. TestMiniMRChildTask fails on Windows.
- (Chuan Liu via cnauroth)
-
- MAPREDUCE-5846. Rumen doesn't understand JobQueueChangedEvent (Nathan Roberts via raviprak)
-
- MAPREDUCE-5837. MRAppMaster fails when checking on uber mode. (wheat9)
-
- MAPREDUCE-5749. TestRMContainerAllocator#testReportedAppProgress Failed
- (jlowe)
-
- MAPREDUCE-5884. History server uses short user name when canceling tokens
- (Mohammad Kamrul Islam via jlowe)
-
- MAPREDUCE-5888. Failed job leaves hung AM after it unregisters (Jason Lowe
- via jeagles)
-
- MAPREDUCE-5814. fat jar with *-default.xml may fail when
- mapreduce.job.classloader=true. (Gera Shegalov via jlowe)
-
- MAPREDUCE-5309. 2.0.4 JobHistoryParser can't parse certain failed job
- history files generated by 2.0.3 history server (Rushabh S Shah via jlowe)
-
- MAPREDUCE-5862. Line records longer than 2x split size aren't handled
- correctly (bc Wong via jlowe)
-
- MAPREDUCE-5895. Close streams properly to avoid leakage in TaskLog.
- (Kousuke Saruta via devaraj)
-
- MAPREDUCE-5777. Support utf-8 text with Byte Order Marker.
- (Zhihai Xu via kasha)
-
- MAPREDUCE-5898. distcp to support preserving HDFS extended attributes(XAttrs)
- (Yi Liu via umamahesh)
-
- MAPREDUCE-5920. Add Xattr option in DistCp docs. (Yi Liu via cnauroth)
-
- MAPREDUCE-5924. Changed TaskAttemptImpl to ignore TA_COMMIT_PENDING event
- at COMMIT_PENDING state. (Zhijie Shen via jianhe)
-
- MAPREDUCE-5939. StartTime showing up as the epoch time in JHS UI after
- upgrade (Chen He via jlowe)
-
- MAPREDUCE-5900. Changed to the interpret container preemption exit code as a
- task attempt killing event. (Mayank Bansal via zjshen)
-
- MAPREDUCE-5868. Fixed an issue with TestPipeApplication that was causing the
- nightly builds to fail. (Akira Ajisaka via vinodkv)
-
- MAPREDUCE-5517. Fixed MapReduce ApplicationMaster to not validate reduce side
- resource configuration for deciding uber-mode on map-only jobs. (Siqi Li via
- vinodkv)
-
- MAPREDUCE-5952. LocalContainerLauncher#renameMapOutputForReduce incorrectly
- assumes a single dir for mapOutIndex. (Gera Shegalov via kasha)
-
- MAPREDUCE-6002. Made MR task avoid reporting error to AM when the task process
- is shutting down. (Wangda Tan via zjshen)
-
-Release 2.4.1 - 2014-06-23
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- MAPREDUCE-5830. Added back the private API HostUtil.getTaskLogUrl(..) for
- binary compatibility with older clients like Hive 0.13. (Akira Ajisaka via
- vinodkv)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-5818. Added "hsadmin" command into mapred.cmd. (Jian He via zjshen)
-
- MAPREDUCE-5824. Fixed test-failure of TestPipesNonJavaInputFormat in
- Windows. (Xuan Gong via vinodkv)
-
- MAPREDUCE-5815. Fixed test-failure of TestMRAppMaster by making MRAppMaster
- gracefully handle empty-queue names. (Akira Ajisaka via vinodkv)
-
- MAPREDUCE-5826. Fixed HistoryServerFileSystemStore to use right permissions
- on Windows for temporary files and thus also fix the test-issue with
- TestHistoryServerFileSystemStateStoreService. (Varun Vasudev via vinodkv)
-
- MAPREDUCE-5828. Fixed a test issue with TestMapReduceJobControl that was
- causing it to fail on Windows. (vinodkv)
-
- MAPREDUCE-5827. TestSpeculativeExecutionWithMRApp fails.
- (Zhijie Shen via cnauroth)
-
- MAPREDUCE-5833. TestRMContainerAllocator fails ocassionally.
- (Zhijie Shen via cnauroth)
-
- MAPREDUCE-5832. Fixed TestJobClient to not fail on JDK7 or on Windows. (Jian
- He and Vinod Kumar Vavilapalli via vinodkv)
-
- MAPREDUCE-5841. uber job doesn't terminate on getting mapred job kill
- (Sangjin Lee via jlowe)
-
- MAPREDUCE-5843. Fixed TestMRKeyValueTextInputFormat to not leak files and
- thus avoid failing on Windows. (Varun Vasudev via vinodkv)
-
- MAPREDUCE-5835. Killing Task might cause the job to go to ERROR state
- (Ming Ma via jlowe)
-
- MAPREDUCE-5821. Avoid unintentional reallocation of byte arrays in segments
- during merge. (Todd Lipcon via cdouglas)
-
-Release 2.4.0 - 2014-04-07
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- MAPREDUCE-5787. Added the ability to keep alive shuffle connections in the
- MapReduce shuffle-handler. (Rajesh Balamohan via vinodkv)
-
- IMPROVEMENTS
-
- MAPREDUCE-5464. Add analogs of the SLOTS_MILLIS counters that jive with the
- YARN resource model (Sandy Ryza)
-
- MAPREDUCE-5732. Report proper queue when job has been automatically placed
- (Sandy Ryza)
-
- MAPREDUCE-5699. Allow setting tags on MR jobs (kasha)
-
- MAPREDUCE-5761. Added a simple log message to denote when encrypted shuffle
- is on in the shuffle-handler. (Jian He via vinodkv)
-
- MAPREDUCE-5754. Preserve Job diagnostics in history (Gera Shegalov via
- jlowe)
-
- MAPREDUCE-5766. Moved ping messages from TaskAttempts to be at DEBUG level
- inside the ApplicationMaster log. (Jian He via vinodkv)
-
- MAPREDUCE-5773. Provide dedicated MRAppMaster syslog length limit (Gera
- Shegalov via jlowe)
-
- MAPREDUCE-5553. Allow users to easily access
- completed/pending/successful/failed tasks on MR AM web-ui. (Paul Han via
- acmurthy)
-
- MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to
- handle cross platform application submissions. (Jian He via vinodkv)
-
- MAPREDUCE-2349. Modified FileInputFormat to be able to issue file and block
- location calls in parallel. (Siddharth Seth via vinodkv)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-5746. Job diagnostics can implicate wrong task for a failed job.
- (Jason Lowe via kasha)
-
- MAPREDUCE-5670. CombineFileRecordReader should report progress when moving
- to the next file (Chen He via jlowe)
-
- MAPREDUCE-5757. ConcurrentModificationException in JobControl.toList
- (jlowe)
-
- MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect
- to the YARN's web-app proxy with the correct scheme prefix. (Jian He via
- vinodkv)
-
- MAPREDUCE-5768. TestMRJobs.testContainerRollingLog fails on trunk (Gera
- Shegalov via jlowe)
-
- MAPREDUCE-5780. SliveTest should use the specified path to get the
- particular FileSystem instead of using the default FileSystem. (szetszwo)
-
- MAPREDUCE-5028. Fixed a bug in MapTask that was causing mappers to fail
- when a large value of io.sort.mb is set. (Karthik Kambatla via vinodkv)
-
- MAPREDUCE-5778. JobSummary does not escape newlines in the job name (Akira
- AJISAKA via jlowe)
-
- MAPREDUCE-5789. Average Reduce time is incorrect on Job Overview page
- (Rushabh S Shah via jlowe)
-
- MAPREDUCE-5794. SliveMapper always uses default FileSystem. (szetszwo via
- Arpit Agarwal)
-
- MAPREDUCE-5751. MR app master fails to start in some cases if
- mapreduce.job.classloader is true (Sangjin Lee via jlowe)
-
- MAPREDUCE-5688. TestStagingCleanup fails intermittently with JDK7 (Mit
- Desai via jeagles)
-
- MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed
- before registering with RM (Rohith via jlowe)
-
- MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt
- finish time (Rushabh S Shah via jlowe)
-
- MAPREDUCE-5806. Fixed a bug in MRAppMaster so as to enable users to properly
- override HADOOP_ROOT_LOGGER or HADOOP_CLIENT_OPTS. (Varun Vasudev via
- vinodkv)
-
- MAPREDUCE-5791. Shuffle phase is slow in Windows -
- FadviseFileRegion::transferTo does not read disks efficiently.
- (Nikola Vujic via cnauroth)
-
- MAPREDUCE-5795. Fixed MRAppMaster to record the correct job-state after it
- recovers from a commit during a previous attempt. (Xuan Gong via vinodkv)
-
- MAPREDUCE-5805. Fixed MapReduce JobHistory encoding of queue-name to escape
- hyphens and thus avoid parsing errors. (Akira AJISAKA via vinodkv)
-
- MAPREDUCE-5810. Removed the faulty and failing streaming test
- TestStreamingTaskLog. (Akira Ajisaka via vinodkv)
-
- MAPREDUCE-5813. Fix YarnChild to explicitly load job.xml from the
- local-filesystem, rather than rely on the classpath. (Gera Shegalov via
- acmurthy)
-
-Release 2.3.1 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
-Release 2.3.0 - 2014-02-18
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- MAPREDUCE-5265. History server admin service to refresh user and superuser
- group mappings (Ashwin Shankar via jlowe)
-
- MAPREDUCE-5356. Ability to refresh aggregated log retention period and
- check interval (Ashwin Shankar via jlowe)
-
- MAPREDUCE-5386. Ability to refresh history server job retention and job
- cleaner settings (Ashwin Shankar via jlowe)
-
- MAPREDUCE-5411. Refresh size of loaded job cache on history server (Ashwin
- Shankar via jlowe)
-
- MAPREDUCE-5332. Support token-preserving restart of history server (jlowe)
-
- IMPROVEMENTS
-
- MAPREDUCE-5329. Allow MR applications to use additional AuxServices,
- which are compatible with the default MapReduce shuffle.
- (Avner BenHanoch via sseth)
-
- MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters (Tzuyoshi Ozawa via Sandy
- Ryza)
-
- MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write
- out text files without separators (Sandy Ryza)
-
- MAPREDUCE-5596. Allow configuring the number of threads used to serve
- shuffle connections (Sandy Ryza via jlowe)
-
- MAPREDUCE-434. LocalJobRunner limited to single reducer (Sandy Ryza and
- Aaron Kimball via Sandy Ryza)
-
- MAPREDUCE-4421. Run MapReduce framework via the distributed cache (jlowe)
-
- MAPREDUCE-1176. FixedLengthInputFormat and FixedLengthRecordReader
- (Mariappan Asokan and BitsOfInfo via Sandy Ryza)
-
- MAPREDUCE-5613. DefaultSpeculator holds and checks hashmap that is always
- empty (Gera Shegalov via Sandy Ryza)
-
- MAPREDUCE-5431. Missing pom dependency in MR-client (Timothy St. Clair
- via stevel)
-
- MAPREDUCE-5624 Move grizzly-test and junit dependencies to test scope
- (Ted Yu via stevel)
-
- MAPREDUCE-5481. Enable uber jobs to have multiple reducers (Sandy Ryza)
-
- MAPREDUCE-5052. Job History UI and web services confusing job start time and
- job submit time (Chen He via jeagles)
-
- MAPREDUCE-5692. Add explicit diagnostics when a task attempt is killed due
- to speculative execution (Gera Shegalov via Sandy Ryza)
-
- MAPREDUCE-5550. Task Status message (reporter.setStatus) not shown in UI
- with Hadoop 2.0 (Gera Shegalov via Sandy Ryza)
-
- MAPREDUCE-3310. Custom grouping comparator cannot be set for Combiners (tucu)
-
- MAPREDUCE-5672. Provide optional RollingFileAppender for container log4j
- (syslog) (Gera Shegalov via jlowe)
-
- MAPREDUCE-5725. Make explicit that TestNetworkedJob relies on the Capacity
- Scheduler (Sandy Ryza)
-
- MAPREDUCE-5744. Job hangs because
- RMContainerAllocator$AssignedRequests.preemptReduce() violates the
- comparator contract (Gera Shegalov via kasha)
-
- OPTIMIZATIONS
-
- MAPREDUCE-4680. Job history cleaner should only check timestamps of files in
- old enough directories (Robert Kanter via Sandy Ryza)
-
- MAPREDUCE-5484. YarnChild unnecessarily loads job conf twice (Sandy Ryza)
-
- MAPREDUCE-5487. In task processes, JobConf is unnecessarily loaded again
- in Limits (Sandy Ryza)
-
- MAPREDUCE-5601. ShuffleHandler fadvises file regions as DONTNEED even when
- fetch fails (Sandy Ryza)
-
- BUG FIXES
-
- MAPREDUCE-5569. FloatSplitter is not generating correct splits (Nathan
- Roberts via jlowe)
-
- MAPREDUCE-5546. mapred.cmd on Windows set HADOOP_OPTS incorrectly (Chuan Liu
- via cnauroth)
-
- MAPREDUCE-5518. Fixed typo "can't read paritions file". (Albert Chu
- via devaraj)
-
- MAPREDUCE-5561. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl
- testcase failing on trunk (Karthik Kambatla via jlowe)
-
- MAPREDUCE-5598. TestUserDefinedCounters.testMapReduceJob is flakey
- (Robert Kanter via jlowe)
-
- MAPREDUCE-5604. TestMRAMWithNonNormalizedCapabilities fails on Windows due to
- exceeding max path length. (cnauroth)
-
- MAPREDUCE-5451. MR uses LD_LIBRARY_PATH which doesn't mean anything in
- Windows. (Yingda Chen via cnauroth)
-
- MAPREDUCE-5409. MRAppMaster throws InvalidStateTransitonException: Invalid
- event: TA_TOO_MANY_FETCH_FAILURE at KILLED for TaskAttemptImpl (Gera
- Shegalov via jlowe)
-
- MAPREDUCE-5674. Missing start and finish time in mapred.JobStatus.
- (Chuan Liu via cnauroth)
-
- MAPREDUCE-5650. Job fails when hprof mapreduce.task.profile.map/reduce.params
- is specified (Gera Shegalov via Sandy Ryza)
-
- MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal
- task-state (Ashwin Shankar via jlowe)
-
- MAPREDUCE-5380. Invalid mapred command should return non-zero exit code
- (Stephen Chu via jlowe)
-
- MAPREDUCE-5404. HSAdminServer does not use ephemeral ports in minicluster
- mode (Ted Yu via jlowe)
-
- MAPREDUCE-5522. Incorrect oreder expected from JobQueueInfo (Jinghui Wang
- via bobby)
-
- MAPREDUCE-5514. Fix TestRMContainerAllocator. (Zhijie Shen via acmurthy)
-
- MAPREDUCE-5102. fix coverage org.apache.hadoop.mapreduce.lib.db and
- org.apache.hadoop.mapred.lib.db (Aleksey Gorshkov, Andrey Klochkov, and
- Nathan Roberts via jlowe)
-
- MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7
- (jeagles)
-
- MAPREDUCE-5186. mapreduce.job.max.split.locations causes some splits
- created by CombineFileInputFormat to fail (Robert Parker and Jason Lowe
- via jlowe)
-
- MAPREDUCE-5610. TestSleepJob fails in jdk7 (Jonathan Eagles via jlowe)
-
- MAPREDUCE-5616. MR Client-AppMaster RPC max retries on socket timeout is too
- high. (cnauroth)
-
- MAPREDUCE-5625. TestFixedLengthInputFormat fails in jdk7 environment
- (Mariappan Asokan via jeagles)
-
- MAPREDUCE-5631. TestJobEndNotifier.testNotifyRetries fails with Should
- have taken more than 5 seconds in jdk7 (Jonathan Eagles via jlowe)
-
- MAPREDUCE-5645. TestFixedLengthInputFormat fails with native libs (Mit
- Desai via jeagles)
-
- MAPREDUCE-5632. TestRMContainerAllocator#testUpdatedNodes fails (jeagles)
-
- MAPREDUCE-5656. bzip2 codec can drop records when reading data in splits
- (jlowe)
-
- MAPREDUCE-5623. TestJobCleanup fails because of RejectedExecutionException
- and NPE. (jlowe)
-
- MAPREDUCE-5679. TestJobHistoryParsing has race condition (Liyin Liang via
- jlowe)
-
- MAPREDUCE-5687. Fixed failure in TestYARNRunner caused by YARN-1446. (Jian He
- via vinodkv)
-
- MAPREDUCE-5694. Fixed MR AppMaster to shutdown the LogManager so as to avoid
- losing syslog in some conditions. (Mohammad Kamrul Islam via vinodkv)
-
- MAPREDUCE-5685. Fixed a bug with JobContext getCacheFiles API inside the
- WrappedReducer class. (Yi Song via vinodkv)
-
- MAPREDUCE-5689. MRAppMaster does not preempt reducers when scheduled maps
- cannot be fulfilled. (lohit via kasha)
-
- MAPREDUCE-5724. JobHistoryServer does not start if HDFS is not running.
- (tucu)
-
- MAPREDUCE-5729. mapred job -list throws NPE (kasha)
-
- MAPREDUCE-5693. Restore MRv1 behavior for log flush (Gera Shegalov via
- jlowe)
-
- MAPREDUCE-5723. MR AM container log can be truncated or empty.
- (Mohammad Kamrul Islam via kasha)
-
- MAPREDUCE-5743. Fixed the test failure in TestRMContainerAllocator.
- (Ted Yu and Vinod Kumar Vavilapalli via zjshen)
-
-Release 2.2.0 - 2013-10-13
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- MAPREDUCE-5504. mapred queue -info inconsistent with types (Kousuke Saruta
- via tgraves)
-
- MAPREDUCE-5488. Changed MR client to keep trying to reach the application
- when it sees that on attempt's AM is down. (Jian He via vinodkv)
-
- MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config
- mapreduce.ssl.enabled to enable https and disabling it by default as MR AM
- needs to set up its own certificates etc and not depend on clusters'.
- (Omkar Vinit Joshi via vinodkv)
-
- MAPREDUCE-5505. Clients should be notified job finished only after job
- successfully unregistered (Zhijie Shen via bikas)
-
- MAPREDUCE-5503. Fixed a test issue in TestMRJobClient. (Jian He via vinodkv)
-
- MAPREDUCE-5170. Fixed a wrong log message in CombineFileInputFormat class.
- (Sangjin Lee via vinodkv)
-
- MAPREDUCE-5525. Increase timeout of TestDFSIO.testAppend and
- TestMRJobsWithHistoryService.testJobHistoryData. (Chuan Liu via cnauroth)
-
- MAPREDUCE-5513. ConcurrentModificationException in JobControl (Robert
- Parker via jlowe)
-
- MAPREDUCE-5531. Fix compat with hadoop-1 in mapreduce.(TaskID,
- TaskAttemptID) by re-introducing missing constructors. (Robert Kanter via
- acmurthy)
-
- MAPREDUCE-5545. org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.testCommitWindow
- times out (Robert Kanter via jlowe)
-
- MAPREDUCE-5529. Fix compat with hadoop-1 in mapred.TotalOrderPartitioner
- by re-introducing (get,set)PartitionFile which takes in JobConf. (Robert
- Kanter via acmurthy)
-
- MAPREDUCE-5538. Fixed MR AppMaster to send job-notification URL only after
- the job is really done - a bug caused by MAPREDUCE-5505. (Zhijie Shen via
- vinodkv)
-
- MAPREDUCE-5551. Fix compat with hadoop-1 in
- SequenceFileAsBinaryOutputFormat.WritableValueBytes by re-introducing
- missing constructors. (Zhijie Shen via acmurthy)
-
- MAPREDUCE-5544. JobClient#getJob loads job conf twice. (Sandy Ryza)
-
- MAPREDUCE-5536. Fixed MR AM and JHS to respect
- mapreduce.jobhistory.webapp.https.address. (Omkar Vinit Joshi via vinodkv)
-
- MAPREDUCE-5530. Fix compat with hadoop-1 in
- mapred.lib.CombinFileInputFormat by re-introducing
- isSplittable(FileSystem, Path) api and ensuring semantic compatibility.
- (Robert Kanter via acmurthy)
-
- MAPREDUCE-5459. Update documentation on how to run MRv1 examples on YARN.
- (Zhijie Shen via acmurthy)
-
- MAPREDUCE-5554. hdfs-site.xml included in hadoop-mapreduce-client-jobclient
- tests jar is breaking tests for downstream components (Robert Kanter via
- Sandy Ryza)
-
- MAPREDUCE-5489. MR jobs hangs as it does not use the node-blacklisting
- feature in RM requests (Zhijie Shen via bikas)
-
- MAPREDUCE-5442. $HADOOP_MAPRED_HOME/$HADOOP_CONF_DIR setting not working on
- Windows. (Yingda Chen via cnauroth)
-
- MAPREDUCE-5533. Fixed MR speculation code to track any TaskAttempts that
- aren't heart-beating for a while, so that we can aggressively speculate
- instead of waiting for task-timeout (Xuan Gong via vinodkv)
-
- MAPREDUCE-5562. Fixed MR App Master to perform pending tasks like staging-dir
- cleanup, sending job-end notification correctly when unregister with RM
- fails. (Zhijie Shen via vinodkv)
-
-Release 2.1.1-beta - 2013-09-23
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- MAPREDUCE-5478. TeraInputFormat unnecessarily defines its own FileSplit
- subclass (Sandy Ryza)
-
- MAPREDUCE-5497. Changed MRAppMaster to sleep only after doing everything else
- but just before ClientService to avoid race conditions during RM restart.
- (Jian He via vinodkv)
-
- MAPREDUCE-5379. Include token tracking ids in jobconf. (kkambatl via tucu)
-
- MAPREDUCE-5523. Added separate configuration properties for https for JHS
- without which even when https is enabled, it starts on http port itself.
- (Omkar Vinit Joshi via vinodkv)
-
- OPTIMIZATIONS
-
- MAPREDUCE-5446. TestJobHistoryEvents and TestJobHistoryParsing have race
- conditions (jlowe via kihwal)
-
- MAPREDUCE-5462. In map-side sort, swap entire meta entries instead of
- indexes for better cache performance. (Sandy Ryza)
-
- MAPREDUCE-1981. Improve getSplits performance by using listLocatedStatus
- (Hairong Kuang and Jason Lowe via jlowe)
-
- BUG FIXES
-
- MAPREDUCE-5385. Fixed a bug with JobContext getCacheFiles API. (Omkar Vinit
- Joshi via vinodkv)
-
- MAPREDUCE-5428. HistoryFileManager doesn't stop threads when service is
- stopped (Karthik Kambatla via jlowe)
-
- MAPREDUCE-5251. Reducer should not implicate map attempt if it has
- insufficient space to fetch map output (Ashwin Shankar via jlowe)
-
- MAPREDUCE-5317. Stale files left behind for failed jobs (Ravi Prakash via
- jlowe)
-
- MAPREDUCE-5358. MRAppMaster throws invalid transitions for JobImpl
- (Devaraj K via jlowe)
-
- MAPREDUCE-3193. FileInputFormat doesn't read files recursively in the
- input path dir (Devaraj K via jlowe)
-
- MAPREDUCE-5440. TestCopyCommitter Fails on JDK7 (Robert Parker via jlowe)
-
- MAPREDUCE-5367. Local jobs all use same local working directory
- (Sandy Ryza)
-
- MAPREDUCE-5425. Junit in TestJobHistoryServer failing in jdk 7 (Robert
- Parker via jlowe)
-
- MAPREDUCE-5454. TestDFSIO fails intermittently on JDK7 (Karthik Kambatla
- via Sandy Ryza)
-
- MAPREDUCE-5001. LocalJobRunner has race condition resulting in job
- failures (Sandy Ryza via jlowe)
-
- MAPREDUCE-5466. Changed MR AM to not promote history files of intermediate
- AMs in case they are exiting because of errors and thus help history-server
- pick up the right history file for the last successful AM. (Jian He via
- vinodkv)
-
- MAPREDUCE-5468. Fix MR AM recovery for map-only jobs. (vinodkv via
- acmurthy)
-
- MAPREDUCE-5470. LocalJobRunner does not work on Windows. (Sandy Ryza via
- cnauroth)
-
- MAPREDUCE-5476. Changed MR AM recovery code to cleanup staging-directory
- only after unregistering from the RM. (Jian He via vinodkv)
-
- MAPREDUCE-5483. revert MAPREDUCE-5357. (rkanter via tucu)
-
- MAPREDUCE-5441. Changed MR AM to return RUNNING state if exiting when RM
- commands to reboot, so that client can continue to track the overall job.
- (Jian He via vinodkv)
-
- MAPREDUCE-5475. MRClientService does not verify ACLs properly (jlowe)
-
- MAPREDUCE-5414. TestTaskAttempt fails in JDK7 with NPE (Nemon Lou via
- devaraj)
-
- MAPREDUCE-5020. Compile failure with JDK8 (Trevor Robinson via tgraves)
-
- MAPREDUCE-5164. mapred job and queue commands omit HADOOP_CLIENT_OPTS
- (Nemon Lou via devaraj)
-
- MAPREDUCE-5493. Cleanup in-memory & on-disk segments to prevent leak on
- shuffle completion. (jlowe via acmurthy)
-
-Release 2.1.0-beta - 2013-08-22
-
- INCOMPATIBLE CHANGES
-
- MAPREDUCE-4067. Changed MRClientProtocol api to throw IOException only (Xuan
- Gong via vinodkv)
-
- MAPREDUCE-5234. Change mapred.TaskReport and mapreduce.TaskReport for binary
- compatibility with mapred in 1.x but incompatible with 0.23.x. (Mayank Bansal
- via vinodkv)
-
- MAPREDUCE-5156. Change hadoop examples ProgramDriver to be able to run
- 1.x examples jar on top of YARN. This change breaks 0.23.x direct usages of
- ProgramDriver. (Zhijie Shen via vinodkv)
-
- MAPREDUCE-5233. Add methods that are changed or removed from JobControl.Job
- when compared to 1.x. This breaks 0.23.x users of one API in Job. (Mayank
- Bansal via vinodkv)
-
- MAPREDUCE-5237. Add methods that were removed from ClusterStatus back into
- 2.x so as to be compatible with 1.x. Incompatible as
- ClusterStatus.UNINITIALIZED_MEMORY_VALUE is a long now and so breaks 0.23.x
- but it shouldn't be a big deal in reality. (Zhijie Shen via vinodkv)
-
- NEW FEATURES
-
- HADOOP-8562. Enhancements to support Hadoop on Windows Server and Windows
- Azure environments. (See breakdown of tasks below for subtasks and
- contributors)
-
- IMPROVEMENTS
-
- MAPREDUCE-3008. Improvements to cumulative CPU emulation for short running
- tasks in Gridmix. (amarrk via tgraves)
-
- MAPREDUCE-5033. mapred shell script should respect usage flags
- (--help -help -h). (Andrew Wang via atm)
-
- MAPREDUCE-4892. Modify CombineFileInputFormat to not skew input slits'
- allocation on small clusters. (Bikas Saha via vinodkv)
-
- MAPREDUCE-4990. Construct debug strings conditionally in
- ShuffleHandler.Shuffle#sendMapOutput(). (kkambatl via tucu)
-
- MAPREDUCE-4875. coverage fixing for org.apache.hadoop.mapred
- (Aleksey Gorshkov via bobby)
-
- MAPREDUCE-5129. Allow tags to JobHistory for deeper analytics. (billie via
- acmurthy)
-
- MAPREDUCE-3787. [Gridmix] Optimize job monitoring and STRESS mode for
- faster job submission. (amarrk via tgraves)
-
- MAPREDUCE-5079. Changes job recovery to restore state directly from job
- history, instaed of simulating state machine events.
- (Jason Lowe and Robert Parker via sseth)
-
- MAPREDUCE-4981. Add WordMean, WordMedian, WordStandardDeviation
- to ExamplesDriver. (Plamen Jeliazkov via shv)
-
- MAPREUDUCE-5059. Change average merge time on Job overview page to be the
- time delta between the end of the shuffle and the start of the reduce.
- (Omkar Vinit Joshi via vinodkv)
-
- MAPREDUCE-4985. Add compression option to TestDFSIO usage.
- (Plamen Jeliazkov via shv)
-
- MAPREDUCE-5152. Make MR App to simply pass through the container from RM
- instead of extracting and populating information itself to start any
- container. (vinodkv)
-
- MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs
- anyways after YARN-561. (Xuan Gong via vinodkv)
-
- MAPREDUCE-5069. add concrete common implementations of
- CombineFileInputFormat (Sangjin Lee via bobby)
-
- MAPREDUCE-5145. Changed default max-attempts to be more than one for MR jobs
- inline with YARN. (Zhijie Shen via vinodkv)
-
- MAPREDUCE-5036. Default shuffle handler port should not be 8080.
- (Sandy Ryza via tomwhite)
-
- MAPREDUCE-5159. Change ValueAggregatorJob to add APIs which can support
- binary compatibility with hadoop-1 examples. (Zhijie Shen via vinodkv)
-
- MAPREDUCE-5157. Bring back old sampler related code so that we can support
- binary compatibility with hadoop-1 sorter example. (Zhijie Shen via vinodkv)
-
- MAPREDUCE-5222. Bring back some methods and constants in Jobclient for
- binary compatibility with mapred in 1.x. (Karthik Kambatla via vinodkv)
-
- MAPREDUCE-5235. Bring back old fields and exceptions in Counters for
- binary compatibility with mapred in 1.x. (Mayank Bansal via vinodkv)
-
- MAPREDUCE-5246. Specify application-type at the time of job submission after
- YARN-563. (Mayank Bansal via vinodkv)
-
- MAPREDUCE-5230. Bring back NLineInputFormat.createFileSplit for binary
- compatibility with mapred in 1.x (Mayank Bansal via vinodkv)
-
- MAPREDUCE-5270. Migrated MR app from using BuilderUtil factory methods to
- individual record factory methods. (Jian He via vinodkv)
-
- MAPREDUCE-5263. Bring back old methods and fields in
- filecache.DistributedCache for binary compatibility with mapred in 1.x.
- (Zhijie Shen via vinodkv)
-
- MAPREDUCE-5228. Bring back FileInputFormat.Counter and
- FileOuputFormat.Counter for binary compatibility with 1.x mapred APIs.
- (Mayank Bansal via vinodkv)
-
- MAPREDUCE-5176. Add annotation for tagging tasks as responsive to
- preemption. (Carlo Curino, cdouglas)
-
- MAPREDUCE-5275. Bring back a couple of APIs in mapreduce.security.TokenCache
- for binary compatibility with 1.x mapreduce APIs. (Mayank Bansal via vinodkv)
-
- MAPREDUCE-5231. Bring back a constructor in mapred's
- DBInputFormat.DBRecordReader for binary compatibility with 1.x mapred APIs.
- (Zhijie Shen via vinodkv)
-
- MAPREDUCE-5273. Bring back a couple of protected variables in mapred and
- mapreduce CombineFileRecordReader for binary compatibility with 1.x APIs.
- (Mayank Bansal via vinodkv)
-
- MAPREDUCE-5280. Bring back removed constructor and a method in mapreduce
- ClusterMetrics for binary compatibility with 1.x APIs. (Mayank Bansal via
- vinodkv)
-
- MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. (Jian He
- via vinodkv)
-
- MAPREDUCE-5229. Put back FileOutputCommitter.TEMP_DIR_NAME in mapreduce for
- binary compatibility with 1.x APIs. (Zhijie Shen via vinodkv)
-
- MAPREDUCE-5274. Bring back SecureShuffleUtils.toHex in mapreduce for binary
- compatibility with 1.x APIs. (Mayank Bansal via vinodkv)
-
- MAPREDUCE-5300. Fix backward incompatibility for
- o.a.h.mapreduce.filecache.DistributedCache. (Zhijie Shen via acmurthy)
-
- MAPREDUCE-5283. Over 10 different tests have near identical
- implementations of AppContext (Sandy Ryza via jlowe)
-
- MAPREDUCE-5199. Removing ApplicationTokens file as it is no longer needed.
- (Daryn Sharp via vinodkv)
-
- MAPREDUCE-5192. Allow for alternate resolutions of TaskCompletionEvents.
- (cdouglas via acmurthy)
-
- MAPREDUCE-5184. Document compatibility for MapReduce applications in
- hadoop-2 vis-a-vis hadoop-1. (Zhijie Shen via acmurthy)
-
- MAPREDUCE-5194. Heed interrupts during Fetcher shutdown. (cdouglas)
-
- MAPREDUCE-5326. Added version to shuffle header. (Zhijie Shen via
- acmurthy)
-
- MAPREDUCE-5333. Add test that verifies MRAM works correctly when sending
- requests with non-normalized capabilities. (ywskycn via tucu)
-
- MAPREDUCE-5398. MR changes for YARN-513 (Jian He via bikas)
-
- OPTIMIZATIONS
-
- MAPREDUCE-4974. Optimising the LineRecordReader initialize() method
- (Gelesh via bobby)
-
- MAPREDUCE-5268. Improve history server startup performance (Karthik
- Kambatla via jlowe)
-
- MAPREDUCE-5352. Optimize node local splits generated by
- CombineFileInputFormat. (sseth)
-
- BUG FIXES
-
- MAPREDUCE-4671. AM does not tell the RM about container requests which are
- no longer needed. (Bikas Saha via sseth)
-
- MAPREDUCE-4994. -jt generic command line option does not work. (sandyr via tucu)
-
- MAPREDUCE-5000. Fixes getCounters when speculating by fixing the selection
- of the best attempt for a task. (Jason Lowe via sseth)
-
- MAPREDUCE-4994. Addendum fixing testcases failures. (sandyr via tucu)
-
- MAPREDUCE-4846. Some JobQueueInfo methods are public in MR1 but protected
- in MR2. (Sandy Ryza via tomwhite)
-
- MAPREDUCE-5013. mapred.JobStatus compatibility: MR2 missing constructors
- from MR1. (Sandy Ryza via tomwhite)
-
- MAPREDUCE-4951. Container preemption interpreted as task failure.
- (Sandy Ryza via tomwhite)
-
- MAPREDUCE-5008. Merger progress miscounts with respect to EOF_MARKER.
- (Sandy Ryza via tomwhite)
-
- MAPREDUCE-4693. History server should include counters for failed tasks.
- (Xuan Gong via sseth)
-
- MAPREDUCE-4896. mapred queue -info spits out ugly exception when queue does
- not exist. (sandyr via tucu)
-
- MAPREDUCE-3685. Fix bugs in MergeManager to ensure compression codec is
- appropriately used and that on-disk segments are correctly sorted on
- file-size. (Anty Rao and Ravi Prakash via acmurthy)
-
- MAPREDUCE-4571. TestHsWebServicesJobs fails on jdk7. (tgraves via tucu)
-
- MAPREDUCE-4716. TestHsWebServicesJobsQuery.testJobsQueryStateInvalid
- fails with jdk7. (tgraves via tucu)
-
- MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream
- does not close the wrapped InputStream. (Chris Nauroth via szetszwo)
-
- MAPREDUCE-3872. Fix an event handling races in ContainerLauncherImpl.
- (Robert Kanter via sseth)
-
- MAPREDUCE-5062. Fix MR AM to read max-retries from the RM. (Zhijie Shen via
- vinodkv)
-
- MAPREDUCE-3829. [Gridmix] Gridmix should give better error message when
- input data directory already exists and -generate opton is
- given.(ravigummadi via tgraves)
-
- MAPREDUCE-2722. [Gridmix] Gridmix simulated job's map's hdfsBytesRead
- counter is wrong when compressed input is used.(ravigummadi via tgraves)
-
- MAPREDUCE-3953. [Gridmix] Gridmix throws NPE and does not simulate a
- job if the trace contains null taskStatus for a task. (ravigummadi via
- tgraves)
-
- MAPREDUCE-4087. [Gridmix] GenerateDistCacheData job of Gridmix can
- become slow in some cases (ravigummadi via tgraves).
-
- MAPREDUCE-5077. Remove mapreduce.util.ResourceCalculatorPlugin and related
- code. (Karthik Kambatla via sseth)
-
- MAPREDUCE-4083. [Gridmix] NPE in cpu emulation. (amarrk via tgraves)
-
- MAPREDUCE-4100. [Gridmix] Bug fixed in compression emulation feature for
- map only jobs. (amarrk via tgraves)
-
- MAPREDUCE-4356. [Rumen] Provide access to the method
- ParsedTask.obtainTaskAttempts(). (ravigummadi via tgraves)
-
- MAPREDUCE-4149. [Rumen] Rumen fails to parse certain counter
- strings. (ravigummadi via tgraves)
-
- MAPREDUCE-3757. [Rumen] Fixed Rumen Folder to adjust shuffleFinished and
- sortFinished times when needed. (Ravi Gummadi via tgraves)
-
- MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. (Omkar Vinit
- Joshi via vinodkv)
-
- MAPREDUCE-5086. MR app master deletes staging dir when sent a reboot
- command from the RM. (Jian He via jlowe)
-
- MAPREDUCE-5113. Streaming input/output types are ignored with java
- mapper/reducer. (sandyr via tucu)
-
- MAPREDUCE-5098. Fix findbugs w
<TRUNCATED>
[3/5] hadoop git commit: HADOOP-11792. Remove all of the CHANGES.txt
files.
Posted by wa...@apache.org.
http://git-wip-us.apache.org/repos/asf/hadoop/blob/1bc45461/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
deleted file mode 100644
index 0ddecaf..0000000
--- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
+++ /dev/null
@@ -1,14611 +0,0 @@
-Hadoop HDFS Change Log
-
-Trunk (Unreleased)
-
- INCOMPATIBLE CHANGES
-
- HDFS-3034. Remove the deprecated DFSOutputStream.sync() method. (szetszwo)
-
- HDFS-5079. Cleaning up NNHAStatusHeartbeat.State from
- DatanodeProtocolProtos. (Tao Luo via shv)
-
- HDFS-5570. Deprecate hftp / hsftp and replace them with webhdfs / swebhdfs.
- (wheat9)
-
- HDFS-2538. option to disable fsck dots (Mohammad Kamrul Islam via aw)
-
- HDFS-7302. Remove "downgrade" from "namenode -rollingUpgrade" startup
- option since it may incorrectly finalize an ongoing rolling upgrade.
- (Kai Sasaki via szetszwo)
-
- HDFS-7985. WebHDFS should be always enabled. (Li Lu via wheat9)
-
- HDFS-7281. Missing block is marked as corrupted block (Ming Ma via
- Yongjun Zhang)
-
- HDFS-8349. Remove .xml and documentation references to dfs.webhdfs.enabled.
- (Ray Chiang via aajisaka)
-
- HDFS-8241. Remove unused NameNode startup option -finalize.
- (Brahma Reddy Battula via aajisaka)
-
- HDFS-8135. Remove the deprecated FSConstants class. (Li Lu via wheat9)
-
- HDFS-8591. Remove support for deprecated configuration key
- dfs.namenode.decommission.nodes.per.interval. (wang)
-
- HDFS-8895. Remove deprecated BlockStorageLocation APIs. (wang)
-
- HDFS-8981. Adding revision to data node jmx getVersion() method. (Siqi Li
- via mingma)
-
- HDFS-9085. Show renewer information in DelegationTokenIdentifier#toString.
- (Zhihai Xu via cnauroth)
-
- HDFS-9278. Fix preferredBlockSize typo in OIV XML output. (Nicole Pazmany via wang)
-
- HDFS-5165. Remove the TotalFiles metrics. (Akira Ajisaka via wheat9)
-
- NEW FEATURES
-
- HDFS-3125. Add JournalService to enable Journal Daemon. (suresh)
-
- HDFS-6440. Support more than 2 NameNodes. (Jesse Yates via atm)
-
- HDFS-9057. allow/disallow snapshots via webhdfs
- (Bramma Reddy Battula via vinayakumarb)
-
- HDFS-9525. hadoop utilities need to support provided delegation
- tokens (HeeSoo Kim via aw)
-
- HDFS-9804. Allow long-running Balancer to login with keytab.
- (Xiao Chen via zhz)
-
- IMPROVEMENTS
-
- HDFS-9653. Added blocks pending deletion report to dfsadmin.
- (Weiwei Yang via eyang)
-
- HDFS-4665. Move TestNetworkTopologyWithNodeGroup to common.
- (Junping Du via llu)
-
- HDFS-1620. Rename HdfsConstants -> HdfsServerConstants, FSConstants ->
- HdfsConstants. (Harsh J Chouraria via atm)
-
- HDFS-2197. Refactor RPC call implementations out of NameNode class (todd)
-
- HDFS-2572. Remove unnecessary double-check in DN#getHostName. (harsh)
-
- HDFS-2857. Cleanup BlockInfo class. (suresh)
-
- HDFS-2786. Fix host-based token incompatibilities in DFSUtil. (Kihwal Lee
- via jitendra)
-
- HDFS-2878. Fix TestBlockRecovery and move it back into main test directory.
- (todd)
-
- HDFS-2655. BlockReaderLocal#skip performs unnecessary IO.
- (Brandon Li via jitendra)
-
- HDFS-3009. Remove duplicate code in DFSClient#isLocalAddress by using
- NetUtils. (Hari Mankude via suresh)
-
- HDFS-3002. TestNameNodeMetrics need not wait for metrics update.
- (suresh)
-
- HDFS-3016. Security in unit tests. (Jaimin Jetly via jitendra)
-
- HDFS-3030. Remove getProtocolVersion and getProtocolSignature from translators.
- (jitendra)
-
- HDFS-3111. Missing license headers in trunk. (umamahesh)
-
- HDFS-3091. Update the usage limitations of ReplaceDatanodeOnFailure policy in
- the config description for the smaller clusters. (szetszwo via umamahesh)
-
- HDFS-309. FSEditLog should log progress during replay. (Sho Shimauchi
- via todd)
-
- HDFS-3131. Improve TestStorageRestore. (Brandon Li via atm)
-
- HDFS-3178. Add states and state handler for journal synchronization in
- JournalService. (szetszwo)
-
- HDFS-3273. Refactor BackupImage and FSEditLog, and rename
- JournalListener.rollLogs(..) to startLogSegment(..). (szetszwo)
-
- HDFS-3292. Remove the deprecated DiskStatus, getDiskStatus(), getRawUsed()
- and getRawCapacity() from DistributedFileSystem. (Arpit Gupta via szetszwo)
-
- HADOOP-8285. HDFS changes for Use ProtoBuf for RpcPayLoadHeader. (sanjay
- radia)
-
- HDFS-2743. Streamline usage of bookkeeper journal manager.
- (Ivan Kelly via umamahesh)
-
- HDFS-3293. Add toString(), equals(..) and hashCode() to JournalInfo.
- (Hari Mankude via szetszwo)
-
- HDFS-3197. Incorrect class comments in a few tests. (Andy Isaacson via eli)
-
- HDFS-3476. Correct the default used in TestDFSClientRetries.busyTest()
- after HDFS-3462 (harsh)
-
- HDFS-3478. Test quotas with Long.Max_Value. (Sujay Rau via eli)
-
- HDFS-3630 Modify TestPersistBlocks to use both flush and hflush (sanjay)
-
- HDFS-3768. Exception in TestJettyHelper is incorrect.
- (Eli Reisman via jghoman)
-
- HDFS-2580. NameNode#main(...) can make use of GenericOptionsParser. (harsh)
-
- HDFS-2127. Add a test that ensure AccessControlExceptions contain
- a full path. (Stephen Chu via eli)
-
- HDFS-3995. Use DFSTestUtil.createFile() for file creation and
- writing in test cases. (Jing Zhao via suresh)
-
- HDFS-3735. NameNode WebUI should allow sorting live datanode list by fields
- Block Pool Used, Block Pool Used(%) and Failed Volumes.
- (Brahma Reddy Battula via suresh)
-
- HDFS-4052. BlockManager#invalidateWork should print log outside the lock.
- (Jing Zhao via suresh)
-
- HDFS-3358. Specify explicitly that the NN UI status total is talking
- of persistent objects on heap. (harsh)
-
- HDFS-4904. Remove JournalService. (Arpit Agarwal via cnauroth)
-
- HDFS-5041. Add the time of last heartbeat to dead server Web UI (Shinichi
- Yamashita via brandonli)
-
- HDFS-5721. sharedEditsImage in Namenode#initializeSharedEdits() should be
- closed before method returns. (Ted Yu via junping_du)
-
- HDFS-6228. comments typo fix for FsDatasetImpl.java (zhaoyunjiong via umamahesh)
-
- HDFS-6246. Remove 'dfs.support.append' flag from trunk code. (umamahesh)
-
- HDFS-6609. Use DirectorySnapshottableFeature to represent a snapshottable
- directory. (Jing Zhao via wheat9)
-
- HDFS-7591. hdfs classpath command should support same options as hadoop
- classpath (Varun Saxena via Arpit Agarwal)
-
- HDFS-316. Balancer should run for a configurable # of iterations (Xiaoyu
- Yao via aw)
-
- HDFS-7322. deprecate sbin/hadoop-daemon.sh (aw)
-
- HDFS-7668. Convert site documentation from apt to markdown (Masatake
- Iwasaki via aw)
-
- HDFS-7460. Rewrite httpfs to use new shell framework (John Smith via aw)
-
- HDFS-6353. Check and make checkpoint before stopping the NameNode. (jing9)
-
- HDFS-5033. Bad error message for fs -put/copyFromLocal if user
- doesn't have permissions to read the source (Darrell Taylor via aw)
-
- HDFS-9182. Cleanup the findbugs and other issues after HDFS EC merged to trunk.
- (umamahesh)
-
- HDFS-8632. Add InterfaceAudience annotation to the erasure coding classes.
- (Rakesh R via wang)
-
- HDFS-9070. Allow fsck display pending replica location information for
- being-written blocks. (GAO Rui via jing9)
-
- HDFS-9261. Erasure Coding: Skip encoding the data cells if all the parity data
- streamers are failed for the current block group. (Rakesh R via umamahesh)
-
- HDFS-9323. Randomize the DFSStripedOutputStreamWithFailure tests. (szetszwo)
-
- HDFS-8777. Erasure Coding: add tests for taking snapshots on EC files.
- (Rakesh R via zhz)
-
- HADOOP-11684. S3a to use thread pool that blocks clients.
- (Thomas Demoor and Aaron Fabbri via lei)
-
- HDFS-9234. WebHdfs: getContentSummary() should give quota for storage types.
- (Surendra Singh Lilhore via xyao)
-
- HDFS-8968. Erasure coding: a comprehensive I/O throughput benchmark tool.
- (Rui Li via zhz)
-
- HDFS-8831. Trash Support for deletion in HDFS encryption zone. (xyao)
-
- HDFS-9582. TestLeaseRecoveryStriped file missing Apache License header
- and not well formatted. (umamahesh)
-
- HDFS-9494. Parallel optimization of DFSStripedOutputStream#flushAllInternals.
- (Gao Rui via jing9)
-
- HDFS-9260. Improve the performance and GC friendliness of NameNode startup
- and full block reports (Staffan Friberg via cmccabe)
-
- HDFS-9795. OIV Delimited should show which files are ACL-enabled (lei)
-
- HDFS-9829. Erasure Coding: Improve few exception handling logic of
- ErasureCodingWorker. (Rakesh R via jing9)
-
- HDFS-9838. Refactor the excessReplicateMap to a class. (szetszwo)
-
- HDFS-9733. Refactor DFSClient#getFileChecksum and DataXceiver#blockChecksum
- (Kai Zheng via umamahesh)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- HADOOP-9635 Fix potential Stack Overflow in DomainSocket.c (V. Karthik Kumar
- via cmccabe)
-
- HDFS-2299. TestOfflineEditsViewer is failing on trunk. (Uma Maheswara Rao G
- via atm)
- HDFS-2310. TestBackupNode fails since HADOOP-7524 went in.
- (Ivan Kelly via todd)
-
- HDFS-2313. Rat excludes has a typo for excluding editsStored files. (atm)
-
- HDFS-2314. MRV1 test compilation broken after HDFS-2197 (todd)
-
- HDFS-46. Change default namespace quota of root directory from
- Integer.MAX_VALUE to Long.MAX_VALUE. (Uma Maheswara Rao G via szetszwo)
-
- HDFS-2373. Commands using WebHDFS and hftp print unnecessary debug
- info on the console with security enabled. (Arpit Gupta via suresh)
-
- HDFS-2776. Missing interface annotation on JournalSet.
- (Brandon Li via jitendra)
-
- HDFS-3037. TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is
- racy. (atm)
-
- HDFS-3116. Typo in fetchdt error message. (AOE Takashi via atm)
-
- HDFS-3126. Journal stream from Namenode to BackupNode needs to have
- timeout. (Hari Mankude via suresh)
-
- HDFS-3121. Add HDFS tests for HADOOP-8014 change. (John George via
- suresh)
-
- HDFS-3119. Overreplicated block is not deleted even after the replication
- factor is reduced after sync follwed by closing that file. (Ashish Singhi
- via umamahesh)
-
- HDFS-3265. PowerPc Build error. (Kumar Ravi via mattf)
-
- HDFS-2312. FSNamesystem javadoc incorrectly says its for DNs. (harsh)
-
- HDFS-3163. TestHDFSCLI.testAll fails if the user name is not all lowercase.
- (Brandon Li via atm)
-
- HDFS-3462. TestDFSClientRetries.busyTest() should restore default
- xceiver count in the config. (Madhukara Phatak via harsh)
-
- HDFS-3550. Fix raid javadoc warnings. (Jason Lowe via daryn)
-
- HDFS-3549. Fix dist tar build fails in hadoop-hdfs-raid project. (Jason Lowe via daryn)
-
- HDFS-3614. Revert unused MiniDFSCluster constructor from HDFS-3049.
- (acmurthy via eli)
-
- HDFS-3792. Fix two findbugs introduced by HDFS-3695 (todd)
-
- HDFS-3827. TestHASafeMode#assertSafemode method should be made static.
- (Jing Zhao via suresh)
-
- HDFS-3834. Remove unused static fields NAME, DESCRIPTION and Usage from
- Command. (Jing Zhao via suresh)
-
- HDFS-2434. TestNameNodeMetrics.testCorruptBlock fails intermittently.
- (Jing Zhao via suresh)
-
- HDFS-4067. TestUnderReplicatedBlocks intermittently fails due to
- ReplicaAlreadyExistsException. (Jing Zhao via suresh)
-
- HDFS-4115. TestHDFSCLI.testAll fails one test due to number format.
- (Trevor Robinson via suresh)
-
- HDFS-4105. The SPNEGO user for secondary namenode should use the web
- keytab. (Arpit Gupta via jitendra)
-
- HDFS-4003. test-patch should build the common native libs before
- running hdfs tests. (Colin Patrick McCabe via eli)
-
- HDFS-4260 Fix HDFS tests to set test dir to a valid HDFS path as opposed
- to the local build path (Chri Nauroth via Sanjay)
-
- HDFS-4310. fix test org.apache.hadoop.hdfs.server.datanode.
- TestStartSecureDataNode (Ivan A. Veselovsky via atm)
-
- HDFS-4275. MiniDFSCluster-based tests fail on Windows due to failure
- to delete test namenode directory. (Chris Nauroth via suresh)
-
- HDFS-4338. TestNameNodeMetrics#testCorruptBlock is flaky. (Andrew Wang via
- atm)
-
- HDFS-4391. TestDataTransferKeepalive fails when tests are executed in a
- certain order. (Andrew Wang via atm)
-
- HDFS-4757. Update FSDirectory#inodeMap when replacing an INodeDirectory
- while setting quota. (Jing Zhao via szetszwo)
-
- HDFS-4761. When resetting FSDirectory, the inodeMap should also be reset.
- (Jing Zhao via szetszwo)
-
- HDFS-4687. TestDelegationTokenForProxyUser#testWebHdfsDoAs is flaky with
- JDK7. (Andrew Wang via atm)
-
- HDFS-3934. duplicative dfs_hosts entries handled wrong. (Colin Patrick
- McCabe)
-
- HDFS-5719. FSImage#doRollback() should close prevState before return
- (Ted Yu via brandonli)
-
- HDFS-5794. Fix the inconsistency of layout version number of
- ADD_DATANODE_AND_STORAGE_UUIDS between trunk and branch-2. (jing9)
-
- HDFS-6893. crypto subcommand is not sorted properly in hdfs's hadoop_usage
- (David Luo via aw)
-
- HDFS-6981. Fix DN upgrade with layout version change. (Arpit Agarwal)
-
- HDFS-7204. balancer doesn't run as a daemon (aw)
-
- HDFS-7319. Remove dead link to HFTP documentation from index.xml.
- (Masatake Iwasaki via wheat9)
-
- HDFS-7407. Minor typo in privileged pid/out/log names (aw)
-
- HDFS-7581. HDFS documentation needs updating post-shell rewrite (aw)
-
- HADOOP-11484. hadoop-mapreduce-client-nativetask fails to build on ARM
- AARCH64 due to x86 asm statements (Edward Nevill via Colin P. McCabe)
-
- HDFS-7667. Various typos and improvements to HDFS Federation doc
- (Charles Lamb via aw)
-
- HDFS-3750. API docs don't include HDFS (Jolly Chen via aw)
-
- HDFS-7320. The appearance of hadoop-hdfs-httpfs site docs is inconsistent
- (Masatake Iwasaki via aw)
-
- HDFS-7670. HDFS Quota guide has typos, incomplete command lines
- (Brahma Reddy Battula via aw)
-
- HDFS-7669. HDFS Design Doc references commands that no longer exist.
- (Brahma Reddy Battula via aw)
-
- HDFS-7755. httpfs shell code has hardcoded path to bash (Dmitry
- Sivachenko via aw)
-
- HDFS-7711. Fix various typos in ClusterSetup.md.
- (Brahma Reddy Battula via wheat9)
-
- HDFS-7791. dfs count -v should be added to quota documentation (Akira
- AJISAKA via aw)
-
- HDFS-7803. Wrong command mentioned in HDFSHighAvailabilityWithQJM
- documentation (Arshad Mohammad via aw)
-
- HDFS-4681. TestBlocksWithNotEnoughRacks#testCorruptBlockRereplicatedAcrossRacks
- fails using IBM java (Ayappan via aw)
-
- HDFS-8110. Remove unsupported 'hdfs namenode -rollingUpgrade downgrade'
- from document. (J.Andreina via aajisaka)
-
- HDFS-7673. synthetic load generator docs give incorrect/incomplete commands
- (Brahma Reddy Battula via aw)
-
- HDFS-8332. DFS client API calls should check filesystem closed
- (Rakesh R via umamahesh)
-
- HDFS-8412. Fix the test failures in HTTPFS. (umamahesh)
-
- HDFS-8627. NPE thrown if unable to fetch token from Namenode
- (J.Andreina via vinayakumarb)
-
- HDFS-8657. Update docs for mSNN. (Jesse Yates via atm)
-
- HDFS-8800. hdfs --daemon stop namenode corrupts logs (John Smith via aw)
-
- HDFS-9204. DatanodeDescriptor#PendingReplicationWithoutTargets is wrongly
- calculated. (Mingliang Liu via jing9)
-
- HDFS-9217. Fix broken findbugsExcludeFile.xml for hadoop-hdfs-client module.
- (Mingliang Liu via aajisaka)
-
- HDFS-9423. Fix intermittent failure of TestEditLogTailer.
- (Masatake Iwasaki via waltersu4549)
-
- HDFS-9580. TestComputeInvalidateWork#testDatanodeReRegistration failed
- due to unexpected number of invalidate blocks. (Wei-Chiu Chuang via zhz)
-
- HDFS-9376. TestSeveralNameNodes fails occasionally.
- (Masatake Iwasaki via cnauroth)
-
- HDFS-9615. Fix variable name typo in DFSConfigKeys. (Ray Chiang via
- Arpit Agarwal)
-
- HDFS-9646. ErasureCodingWorker may fail when recovering data blocks with
- length less than the first internal block. (jing9)
-
- HDFS-9789. Correctly update DataNode's scheduled block size when writing
- small EC file. (jing9)
-
- HDFS-9794. Streamer threads may leak if failure happens when closing the
- striped outputstream. (jing9)
-
- HDFS-9818. Correctly handle EC reconstruction work caused by not enough
- racks. (jing9)
-
- HDFS-9837. BlockManager#countNodes should be able to detect duplicated
- internal blocks. (jing9)
-
- HDFS-9734. Refactoring of checksum failure report related codes.
- (Kai Zheng via zhz)
-
- HDFS-9866. BlockManager#chooseExcessReplicasStriped may weaken rack fault
- tolerance. (jing9)
-
- HDFS-9867. Missing block exception should carry locatedBlocks information.
- (Mingliang Liu via jing9)
-
- HDFS-9876. shouldProcessOverReplicated should not count number of pending
- replicas. (jing9)
-
- BREAKDOWN OF HDFS-7285 SUBTASKS AND RELATED JIRAS
-
- HDFS-7347. Configurable erasure coding policy for individual files and
- directories ( Zhe Zhang via vinayakumarb )
-
- HDFS-7339. Representing striped block groups in NameNode with hierarchical
- naming protocol ( zhz )
-
- HDFS-7652. Process block reports for erasure coded blocks (Zhe Zhang)
-
- HDFS-7716. Erasure Coding: extend BlockInfo to handle EC info (Jing Zhao)
-
- HDFS-7749. Erasure Coding: Add striped block support in INodeFile (Jing Zhao)
-
- HDFS-7837. Erasure Coding: allocate and persist striped blocks in NameNode
- (Jing Zhao via Zhe Zhang)
-
- HDFS-7872. Erasure Coding: INodeFile.dumpTreeRecursively() supports to print
- striped blocks (Takuya Fukudome via jing9)
-
- HDFS-7853. Erasure coding: extend LocatedBlocks to support reading from
- striped files (Jing Zhao)
-
- HDFS-7826. Erasure Coding: Update INodeFile quota computation for striped
- blocks ( Kai Sasaki via jing9 )
-
- HDFS-7912. Erasure Coding: track BlockInfo instead of Block in
- UnderReplicatedBlocks and PendingReplicationBlocks (Jing Zhao)
-
- HDFS-7369. Erasure coding: distribute recovery work for striped blocks to
- DataNode (Zhe Zhang)
-
- HDFS-7864. Erasure Coding: Update safemode calculation for striped blocks
- (GAO Rui via jing9)
-
- HDFS-7827. Erasure Coding: support striped blocks in non-protobuf fsimage
- ( Hui Zheng via jing9 )
-
- HDFS-7616. Add a test for BlockGroup support in FSImage.
- (Takuya Fukudome via szetszwo)
-
- HDFS-7907. Erasure Coding: track invalid, corrupt, and under-recovery striped
- blocks in NameNode (Jing Zhao)
-
- HDFS-8005. Erasure Coding: simplify striped block recovery work computation
- and add tests (Jing Zhao)
-
- HDFS-7617. Add unit tests for editlog transactions for EC
- (Hui Zheng via Zhe Zhang)
-
- HDFS-7839. Erasure coding: implement facilities in NameNode to create and
- manage EC zones (Zhe Zhang)
-
- HDFS-7969. Erasure coding: NameNode support for lease recovery of striped
- block groups. (Zhe Zhang)
-
- HDFS-7782. Erasure coding: pread from files in striped layout.
- (Zhe Zhang and Jing Zhao via Zhe Zhang)
-
- HDFS-8023. Erasure Coding: retrieve eraure coding schema for a file from
- NameNode (vinayakumarb)
-
- HDFS-8074. Define a system-wide default EC schema. (Kai Zheng)
-
- HDFS-8077. Erasure coding: fix bugs in EC zone and symlinks.
- (Jing Zhao and Zhe Zhang via Jing Zhao)
-
- HDFS-8104. Make hard-coded values consistent with the system default schema first before remove them. (Kai Zheng)
-
- HDFS-7889. Subclass DFSOutputStream to support writing striping layout files. (Li Bo via Kai Zheng)
-
- HDFS-8090. Erasure Coding: Add RPC to client-namenode to list all
- ECSchemas loaded in Namenode. (vinayakumarb)
-
- HDFS-8122. Erasure Coding: Support specifying ECSchema during creation of ECZone.
- (Vinayakumar B via Zhe Zhang)
-
- HDFS-8114. Erasure coding: Add auditlog FSNamesystem#createErasureCodingZone if this
- operation fails. (Rakesh R via Zhe Zhang)
-
- HDFS-8123. Erasure Coding: Better to move EC related proto messages to a
- separate erasurecoding proto file (Rakesh R via vinayakumarb)
-
- HDFS-7349. Support DFS command for the EC encoding (vinayakumarb)
-
- HDFS-8120. Erasure coding: created util class to analyze striped block groups.
- (Contributed by Zhe Zhang and Li Bo via Jing Zhao)
-
- HDFS-7994. Detect if resevered EC Block ID is already used during namenode
- startup. (Hui Zheng via szetszwo)
-
- HDFS-8167. BlockManager.addBlockCollectionWithCheck should check if the block is a striped block. (Hui Zheng via zhz).
-
- HDFS-8166. DFSStripedOutputStream should not create empty blocks. (Jing Zhao)
-
- HDFS-7937. Erasure Coding: INodeFile quota computation unit tests.
- (Kai Sasaki via Jing Zhao)
-
- HDFS-8145. Fix the editlog corruption exposed by failed TestAddStripedBlocks.
- (Jing Zhao)
-
- HDFS-8146. Protobuf changes for BlockECRecoveryCommand and its fields for
- making it ready for transfer to DN (Uma Maheswara Rao G via vinayakumarb)
-
- HDFS-8181. createErasureCodingZone sets retryCache state as false always
- (Uma Maheswara Rao G via vinayakumarb)
-
- HDFS-8190. StripedBlockUtil.getInternalBlockLength may have overflow error.
- (szetszwo)
-
- HDFS-8216. TestDFSStripedOutputStream should use BlockReaderTestUtil to
- create BlockReader. (szetszwo via Zhe Zhang)
-
- HDFS-8212. DistributedFileSystem.createErasureCodingZone should pass schema
- in FileSystemLinkResolver. (szetszwo via Zhe Zhang)
-
- HDFS-8024. Erasure Coding: ECworker frame, basics, bootstraping and configuration.
- (umamahesh)
-
- HDFS-8156. Add/implement necessary APIs even we just have the system default
- schema. (Kai Zheng via Zhe Zhang)
-
- HDFS-8136. Client gets and uses EC schema when reads and writes a stripping
- file. (Kai Sasaki via Kai Zheng)
-
- HDFS-8233. Fix DFSStripedOutputStream#getCurrentBlockGroupBytes when the last
- stripe is at the block group boundary. (jing9)
-
- HDFS-8223. Should calculate checksum for parity blocks in DFSStripedOutputStream.
- (Yi Liu via jing9)
-
- HDFS-8228. Erasure Coding: SequentialBlockGroupIdGenerator#nextValue may cause
- block id conflicts (Jing Zhao via Zhe Zhang)
-
- HDFS-8033. Erasure coding: stateful (non-positional) read from files in
- striped layout (Zhe Zhang)
-
- HDFS-8230. Erasure Coding: Ignore DatanodeProtocol#DNA_ERASURE_CODING_RECOVERY
- commands from standbynode if any (vinayakumarb)
-
- HDFS-8189. ClientProtocol#createErasureCodingZone API was wrongly annotated
- as Idempotent (vinayakumarb)
-
- HDFS-8235. Erasure Coding: Create DFSStripedInputStream in DFSClient#open.
- (Kai Sasaki via jing9)
-
- HDFS-8272. Erasure Coding: simplify the retry logic in DFSStripedInputStream
- (stateful read). (Jing Zhao via Zhe Zhang)
-
- HDFS-8282. Erasure coding: move striped reading logic to StripedBlockUtil.
- (Zhe Zhang)
-
- HDFS-8183. Erasure Coding: Improve DFSStripedOutputStream closing of
- datastreamer threads. (Rakesh R via Zhe Zhang)
-
- HDFS-8308. Erasure Coding: NameNode may get blocked in waitForLoadingFSImage()
- when loading editlog. (jing9)
-
- HDFS-7949. WebImageViewer need support file size calculation with striped
- blocks. (Rakesh R via Zhe Zhang)
-
- HDFS-8316. Erasure coding: refactor EC constants to be consistent with HDFS-8249.
- (Zhe Zhang via jing9)
-
- HDFS-8281. Erasure Coding: implement parallel stateful reading for striped layout.
- (jing9)
-
- HDFS-8137. Send the EC schema to DataNode via EC encoding/recovering command(umamahesh)
-
- HDFS-8242. Erasure Coding: XML based end-to-end test for ECCli commands
- (Rakesh R via vinayakumarb)
-
- HDFS-8324. Add trace info to DFSClient#getErasureCodingZoneInfo(..) (vinayakumarb via
- umamahesh)
-
- HDFS-7672. Handle write failure for stripping blocks and refactor the
- existing code in DFSStripedOutputStream and StripedDataStreamer. (szetszwo)
-
- HDFS-7348. Erasure Coding: DataNode reconstruct striped blocks.
- (Yi Liu via Zhe Zhang)
-
- HDFS-8334. Erasure coding: rename DFSStripedInputStream related test
- classes. (Zhe Zhang)
-
- HDFS-8129. Erasure Coding: Maintain consistent naming for Erasure Coding related classes - EC/ErasureCoding
- (umamahesh)
-
- HDFS-8203. Erasure Coding: Seek and other Ops in DFSStripedInputStream.
- (Yi Liu via jing9)
-
- HDFS-8289. Erasure Coding: add ECSchema to HdfsFileStatus. (Yong Zhang via
- jing9)
-
- HDFS-8355. Erasure Coding: Refactor BlockInfo and BlockInfoUnderConstruction.
- (Tsz Wo Nicholas Sze via jing9)
-
- HDFS-7678. Erasure coding: DFSInputStream with decode functionality (pread).
- (Zhe Zhang)
-
- HDFS-8372. Erasure coding: compute storage type quotas for striped files,
- to be consistent with HDFS-8327. (Zhe Zhang via jing9)
-
- HDFS-8368. Erasure Coding: DFS opening a non-existent file need to be
- handled properly (Rakesh R via zhz)
-
- HDFS-8363. Erasure Coding: DFSStripedInputStream#seekToNewSource. (yliu)
-
- HDFS-8195. Erasure coding: Fix file quota change when we complete/commit
- the striped blocks. (Takuya Fukudome via zhz)
-
- HDFS-8364. Erasure coding: fix some minor bugs in EC CLI
- (Walter Su via vinayakumarb)
-
- HDFS-8391. NN should consider current EC tasks handling count from DN while
- assigning new tasks. (umamahesh)
-
- HDFS-8367. BlockInfoStriped uses EC schema. (Kai Sasaki via Kai Zheng)
-
- HDFS-8352. Erasure Coding: test webhdfs read write stripe file. (waltersu4549)
-
- HDFS-8417. Erasure Coding: Pread failed to read data starting from not-first stripe.
- (Walter Su via jing9)
-
- HDFS-8418. Fix the isNeededReplication calculation for Striped block in NN.
- (Yi Liu via jing9)
-
- HDFS-8320. Erasure coding: consolidate striping-related terminologies. (zhz)
-
- HDFS-8366. Erasure Coding: Make the timeout parameter of polling blocking queue
- configurable in DFSStripedOutputStream. (Li Bo)
-
- HDFS-8378. Erasure Coding: Few improvements for the erasure coding worker.
- (Rakesh R via waltersu4549)
-
- HDFS-8375. Add cellSize as an XAttr to ECZone. ( Vinayakumar B via zhz).
-
- HDFS-8428. Erasure Coding: Fix the NullPointerException when deleting file.
- (Yi Liu via zhz).
-
- HDFS-8323. Bump GenerationStamp for write faliure in DFSStripedOutputStream.
- (Tsz Wo Nicholas Sze via jing9)
-
- HDFS-8427. Remove dataBlockNum and parityBlockNum from BlockInfoStriped.
- (Kai Sasaki via jing9)
-
- HDFS-8186. Erasure coding: Make block placement policy for EC file configurable.
- (Walter Su via zhz)
-
- HDFS-8294. Erasure Coding: Fix Findbug warnings present in erasure coding.
- (Rakesh R via zhz)
-
- HDFS-8441. Erasure Coding: make condition check earlier for setReplication.
- (waltersu4549)
-
- HDFS-7768. Change fsck to support EC files. (Takanobu Asanuma via szetszwo)
-
- HDFS-8382. Remove chunkSize and initialize from erasure coder. (Kai Zheng)
-
- HDFS-8408. Revisit and refactor ErasureCodingInfo (vinayakumarb)
-
- HDFS-8479. Erasure coding: fix striping related logic in FSDirWriteFileOp to
- sync with HDFS-8421. (Zhe Zhang via jing9)
-
- HDFS-8481. Erasure coding: remove workarounds in client side stripped blocks
- recovering. (zhz)
-
- HDFS-8336. Expose some administrative erasure coding operations to HdfsAdmin
- (Uma Maheswara Rao G via vinayakumarb)
-
- HDFS-8444. Erasure Coding: fix cannot rename a zone dir
- (Walter Su via vinayakumarb)
-
- HDFS-8517. Fix a decoding issue in stripped block recovering in client side.
- (Kai Zheng via jing9)
-
- HDFS-8453. Erasure coding: properly handle start offset for internal blocks
- in a block group. (Zhe Zhang via jing9)
-
- HDFS-7621. Erasure Coding: update the Balancer/Mover data migration logic.
- (Walter Su via zhz)
-
- HDFS-8328. Follow-on to update decode for DataNode striped blocks
- reconstruction. (yliu)
-
- HDFS-8319. Erasure Coding: support decoding for stateful read.
- (Jing Zhao via zhz)
-
- HDFS-8460. Erasure Coding: stateful read result doesn't match data
- occasionally because of flawed test. (Walter Su via zhz)
-
- HDFS-8556. Erasure Coding: Fix usage of 'createZone' (vinayakumarb)
-
- HDFS-8571. Fix TestErasureCodingCli test (Vinayakumar B via waltersu4549)
-
- HDFS-8450. Erasure Coding: Consolidate erasure coding zone related
- implementation into a single class (Rakesh R via vinayakumarb)
-
- HDFS-8585. Erasure Coding: Remove dataBlockNum and parityBlockNum from
- StripedBlockProto. (Yi Liu via jing9)
-
- HDFS-8559. Erasure Coding: fix non-protobuf fsimage for striped blocks.
- (Jing Zhao via yliu)
-
- HDFS-8580. Erasure coding: Persist cellSize in BlockInfoStriped and
- StripedBlocksFeature. (Walter Su via jing9)
-
- HDFS-8466. Refactor BlockInfoContiguous and fix NPE in
- TestBlockInfo#testCopyConstructor() (vinayakumarb)
-
- HDFS-8254. Avoid assigning a leading streamer in StripedDataStreamer to
- tolerate datanode failure. (Tsz Wo Nicholas Sze via jing9)
-
- HDFS-8543. Erasure Coding: processOverReplicatedBlock() handles striped block.
- (Walter Su via jing9)
-
- HDFS-8602. Erasure Coding: Client can't read(decode) the EC files which have
- corrupt blocks. (jing9 and Kai Sasaki)
-
- HDFS-8567. Erasure Coding: SafeMode handles file smaller than a full stripe.
- (Walter Su via jing9)
-
- HDFS-8253. DFSStripedOutputStream.closeThreads releases cellBuffers
- multiple times. (Kai Sasaki via szetszwo)
-
- HDFS-8468. 2 RPC calls for every file read in DFSClient#open(..) resulting in
- double Audit log entries (vinayakumarb)
-
- HDFS-8684. Erasure Coding: fix some block number calculation for striped
- block. (yliu)
-
- HDFS-8461. Erasure coding: fix priority level of UnderReplicatedBlocks for
- striped block. (Walter Su via jing9)
-
- HDFS-8719. Erasure Coding: client generates too many small packets when
- writing parity data. (Li Bo via waltersu4549)
-
- HDFS-8563. Erasure Coding: fsck handles file smaller than a full stripe.
- (Walter Su via jing9)
-
- HDFS-8484. Erasure coding: Two contiguous blocks occupy IDs belong to same
- striped group. (Walter Su via jing9)
-
- HDFS-8744. Erasure Coding: the number of chunks in packet is not updated
- when writing parity data. (Li Bo)
-
- HDFS-8669. Erasure Coding: handle missing internal block locations in
- DFSStripedInputStream. (jing9)
-
- HDFS-8702. Erasure coding: update BlockManager.blockHasEnoughRacks(..) logic
- for striped block. (Kai Sasaki via jing9)
-
- HDFS-8734. Erasure Coding: fix one cell need two packets. (Walter Su via
- jing9)
-
- HDFS-8619. Erasure Coding: revisit replica counting for striped blocks.
- (Jing Zhao via yliu)
-
- HDFS-8058. Erasure coding: use BlockInfo[] for both striped and contiguous
- blocks in INodeFile. (Zhe Zhang and Yi Liu via zhz)
-
- HDFS-8787. Erasure coding: rename BlockInfoContiguousUC and BlockInfoStripedUC
- to be consistent with trunk. (zhz)
-
- HDFS-8433. Erasure coding: set blockToken in LocatedStripedBlock.(waltersu4549)
-
- HDFS-8760. Erasure Coding: reuse BlockReader when reading the same block in pread.
- (jing9)
-
- HDFS-8781. Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for
- striped block. (Yi Liu via jing9)
-
- HDFS-8813. Erasure Coding: Client no need to decode missing parity blocks.
- (Walter Su via jing9)
-
- HDFS-8798. Erasure Coding: fix DFSStripedInputStream/DFSStripedOutputStream
- re-fetch token when expired. (Walter Su via jing9)
-
- HDFS-8769. Erasure coding: unit test for SequentialBlockGroupIdGenerator.
- (Rakesh R via waltersu4549)
-
- HDFS-8202. Improve end to end stirpping file test to add erasure recovering
- test. (Xinwei Qin via zhz)
-
- HDFS-8804. Erasure Coding: use DirectBufferPool in DFSStripedInputStream for
- buffer allocation. (jing9)
-
- HDFS-8399. Erasure Coding: unit test the behaviour of BlockManager recovery
- work for the deleted blocks. (Rakesh R via zhz)
-
- HDFS-8857. Erasure Coding: Fix ArrayIndexOutOfBoundsException in
- TestWriteStripedFileWithFailure. (Li Bo)
-
- HDFS-8827. Erasure Coding: Fix NPE when NameNode processes over-replicated
- striped blocks. (Walter Su and Takuya Fukudome via jing9)
-
- HDFS-8854. Erasure coding: add ECPolicy to replace schema+cellSize in
- hadoop-hdfs. (Walter Su via zhz)
-
- HDFS-8220. Erasure Coding: StripedDataStreamer fails to handle the
- blocklocations which doesn't satisfy BlockGroupSize. (Rakesh R via zhz)
-
- HDFS-8838. Erasure Coding: Tolerate datanode failures in DFSStripedOutputStream
- when the data length is small. (szetszwo via waltersu4549)
-
- HDFS-8909. Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC
- to use BlockUnderConstructionFeature. (Jing Zhao via waltersu4549)
-
- HDFS-8937. Erasure coding: do not throw exception when setting replication on
- EC file. (Gao Rui via jing9)
-
- HDFS-8978. Erasure coding: fix 2 failed tests of DFSStripedOutputStream.
- (Walter Su via jing9)
-
- HDFS-8833. Erasure coding: store EC schema and cell size in INodeFile and
- eliminate notion of EC zones. (zhz)
-
- HDFS-8853. Erasure Coding: Provide ECSchema validation when setting EC
- policy. (andreina via zhz)
-
- HDFS-8975. Erasure coding : Fix random failure in TestSafeModeWithStripedFile
- (J.Andreina via vinayakumarb)
-
- HDFS-7351. Document the HDFS Erasure Coding feature.
- (umamahesh and Zhe Zhang via wang)
-
- HDFS-8899. Erasure Coding: use threadpool for EC recovery tasks on DataNode.
- (Rakesh R via zhz)
-
- HDFS-9088. Cleanup erasure coding documentation. (wang via zhz)
-
- HDFS-9097. Erasure coding: update EC command "-s" flag to "-p" when
- specifying policy. (zhz)
-
- HDFS-9086. Rename dfs.datanode.stripedread.threshold.millis to
- dfs.datanode.stripedread.timeout.millis. (wang via zhz)
-
- HDFS-8550. Erasure Coding: Fix FindBugs Multithreaded correctness Warning.
- (Rakesh R via zhz)
-
- HDFS-9113. ErasureCodingWorker#processErasureCodingTasks should not fail to process
- remaining tasks due to one invalid ECTask (umamahesh)
-
- HDFS-9091. Erasure Coding: Provide DistributedFilesystem API to
- getAllErasureCodingPolicies. (Rakesh R via zhz)
-
- HDFS-8920. Erasure Coding: when recovering lost blocks, logs can be too
- verbose and hurt performance. (Rui Li via Kai Zheng)
-
- HDFS-8882. Erasure Coding: Use datablocks, parityblocks and cell size from
- ErasureCodingPolicy (Vinayakumar B via zhz)
-
- HDFS-9040. Erasure coding: coordinate data streamers in
- DFSStripedOutputStream. (jing9 and Walter Su)
-
- HDFS-9185. Fix null tracer in ErasureCodingWorker. (Rakesh R via jing9)
-
- HDFS-9180. Update excluded DataNodes in DFSStripedOutputStream based on failures
- in data streamers. (jing9)
-
- HDFS-8557. Allow to configure RS and XOR raw coders (Kai Zheng)
-
- HDFS-9206. Inconsistent default value of dfs.datanode.stripedread.buffer.size.
- (Walter Su via jing9)
-
- HDFS-9209. Erasure coding: Add apache license header in
- TestFileStatusWithECPolicy.java. (Surendra Singh Lilhore via jing9)
-
- HDFS-8438. Erasure Coding: Allow concat striped files if they have the same
- ErasureCodingPolicy. (Walter Su via jing9)
-
- HDFS-9275. Wait previous ErasureCodingWork to finish before schedule
- another one. (Walter Su via yliu)
-
- HDFS-9348. Erasure Coding: DFS GetErasureCodingPolicy API on a non-existent
- file should be handled properly. (Rakesh R via umamahesh)
-
- HDFS-9373. Erasure coding: friendly log information for write operations
- with some failed streamers. (Li Bo via zhz)
-
- HDFS-9173. Erasure Coding: Lease recovery for striped file. (Walter Su and
- Jing Zhao via zhz)
-
- HDFS-9451. Clean up depreated umasks and related unit tests.
- (Wei-Chiu Chuang via wheat9)
-
- HDFS-9575. Use byte array for internal block indices in a striped block.
- (jing9 via szetszwo)
-
- HDFS-9659. EditLogTailerThread to Active Namenode RPC should timeout
- (surendra singh lilhore via vinayakumarb)
-
- HDFS-9731. Erasure Coding: Rename BlockECRecoveryCommand to
- BlockECReconstructionCommand. (Rakesh R via zhz)
-
- HDFS-9403. Erasure coding: some EC tests are missing timeout.
- (Gao Rui via zhz)
-
- HDFS-9658. Erasure Coding: allow to use multiple EC policies in striping
- related tests. (Rui Li via zhz)
-
- HDFS-9775. Erasure Coding : Rename BlockRecoveryWork to
- BlockReconstructionWork. (Rakesh R via zhz)
-
- HDFS-9755. Erasure Coding: allow to use multiple EC policies in striping
- related tests [Part 2]. (Rui Li via zhz)
-
- HDFS-9787. SNNs stop uploading FSImage to ANN once isPrimaryCheckPointer
- changed to false. (Guocui Mi via vinayakumarb)
-
- HDFS-9830. Remove references to hftp in ViewFs documentation in trunk.
- (Wei-Chiu Chuang via aajisaka)
-
- HDFS-9816. Erasure Coding: allow to use multiple EC policies in striping
- related tests [Part 3]. (Rui Li via zhz)
-
-Release 2.9.0 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- HDFS-9047. Retire libwebhdfs. (wheat9)
-
- NEW FEATURES
- HDFS-9835. OIV: add ReverseXML processor which reconstructs an fsimage from
- an XML file (cmccabe)
-
- IMPROVEMENTS
-
- HDFS-9267. TestDiskError should get stored replicas through
- FsDatasetTestUtils. (Lei (Eddy) Xu via Colin P. McCabe)
-
- HDFS-9491. Tests should get the number of pending async delets via
- FsDatasetTestUtils. (Tony Wu via lei)
-
- HDFS-9414. Refactor reconfiguration of ClientDatanodeProtocol for
- reusability. (Xiaobing Zhou via Arpit Agarwal)
-
- HDFS-8947. NameNode, DataNode and NFS gateway to support JvmPauseMonitor as
- a service. (Sunil G via Stevel)
-
- HDFS-9129. Move the safemode block count into BlockManager. (Mingliang Liu
- via jing9)
-
- HDFS-9281. Change TestDeleteBlockPool to not explicitly use File to check
- block pool existence. (lei)
-
- HDFS-9371. Code cleanup for DatanodeManager. (jing9)
-
- HDFS-9300. TestDirectoryScanner.testThrottle() is still a little flakey.
- (Daniel Templeton via wang)
-
- HDFS-8477. describe dfs.ha.zkfc.port in hdfs-default.xml.
- (Kanaka Kumar Avvaru via wang)
-
- HDFS-9350. Avoid creating temprorary strings in Block.toString() and
- getBlockName() (Staffan Friberg via cmccabe)
-
- HDFS-9624. DataNode start slowly due to the initial DU command operations.
- (Lin Yiqun via wang)
-
- HDFS-9576: HTrace: collect position/length information on read operations
- (zhz via cmccabe)
-
- HDFS-9542. Move BlockIdManager from FSNamesystem to BlockManager. (jing9)
-
- HDFS-9674. The HTrace span for OpWriteBlock should record the maxWriteToDisk
- time. (cmccabe via zhz)
-
- HDFS-9094. Add command line option to ask NameNode reload
- configuration. (Xiaobing Zhou via Arpit Agarwal)
-
- HDFS-9541. Add hdfsStreamBuilder API to libhdfs to support defaultBlockSizes
- greater than 2 GB. (cmccabe via zhz)
-
- HDFS-9677. Rename generationStampV1/generationStampV2 to
- legacyGenerationStamp/generationStamp. (Mingliang Liu via jing9)
-
- HDFS-7764. DirectoryScanner shouldn't abort the scan if one directory had
- an error (Rakesh R via cmccabe)
-
- HDFS-9637. Tests for RollingFileSystemSink. (Daniel Templeton via kasha)
-
- HDFS-9780. RollingFileSystemSink doesn't work on secure clusters.
- (Daniel Templeton via kasha)
-
- HDFS-9754. Avoid unnecessary getBlockCollection calls in BlockManager.
- (jing9)
-
- HDFS-7964. Add support for async edit logging. (Daryn Sharp)
-
- HDFS-9534. Add CLI command to clear storage policy from a path.
- (Xiaobing Zhou via Arpit Agarwal)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- HDFS-9618. Fix mismatch between log level and guard in
- BlockManager#computeRecoveryWorkForBlocks (iwasakims)
-
- HDFS-9621. getListing wrongly associates Erasure Coding policy to pre-existing
- replicated files under an EC directory. (jing9)
-
- HDFS-9691. TestBlockManagerSafeMode#testCheckSafeMode fails intermittently.
- (Mingliang Liu via aajisaka)
-
- HDFS-9608. Disk IO imbalance in HDFS with heterogeneous storages.
- (Wei Zhou via wang)
-
- HDFS-9858. RollingFileSystemSink can throw an NPE on non-secure clusters.
- (Daniel Templeton via kasha)
-
-Release 2.8.0 - UNRELEASED
-
- NEW FEATURES
-
- HDFS-7891. A block placement policy with best rack failure tolerance.
- (Walter Su via szetszwo)
-
- HDFS-8131. Implement a space balanced block placement policy (Liu Shaohui
- via kihwal)
-
- HDFS-8155. Support OAuth2 in WebHDFS. (jghoman)
-
- HDFS-9184. Logging HDFS operation's caller context into audit logs.
- (Mingliang Liu via jitendra)
-
- HDFS-9244. Support nested encryption zones. (zhz)
-
- HDFS-9711. Integrate CSRF prevention filter in WebHDFS. (cnauroth)
-
- IMPROVEMENTS
-
- HDFS-9257. improve error message for "Absolute path required" in INode.java
- to contain the rejected path (Marcell Szabo via harsh)
-
- HDFS-2390. dfsadmin -setBalancerBandwidth does not validate -ve value
- (Gautam Gopalakrishnan via harsh)
-
- HDFS-8821. Explain message "Operation category X is not supported
- in state standby" (Gautam Gopalakrishnan via harsh)
-
- HDFS-3918. EditLogTailer shouldn't log WARN when other node
- is in standby mode (todd via harsh)
-
- HDFS-4396. Add START_MSG/SHUTDOWN_MSG for ZKFC
- (Liang Xie via harsh)
-
- HDFS-7875. Improve log message when wrong value configured for
- dfs.datanode.failed.volumes.tolerated.
- (nijel via harsh)
-
- HDFS-2360. Ugly stacktrace when quota exceeds. (harsh)
-
- HDFS-7835. make initial sleeptime in locateFollowingBlock configurable for
- DFSClient. (Zhihai Xu via Yongjun Zhang)
-
- HDFS-7829. Code clean up for LocatedBlock. (Takanobu Asanuma via jing9)
-
- HDFS-7854. Separate class DataStreamer out of DFSOutputStream. (Li Bo via
- jing9)
-
- HDFS-7713. Implement mkdirs in the HDFS Web UI. (Ravi Prakash via wheat9)
-
- HDFS-7928. Scanning blocks from disk during rolling upgrade startup takes
- a lot of time if disks are busy (Rushabh S Shah via kihwal)
-
- HDFS-7990. IBR delete ack should not be delayed. (daryn via kihwal)
-
- HDFS-8004. Use KeyProviderCryptoExtension#warmUpEncryptedKeys when creating
- an encryption zone. (awang via asuresh)
-
- HDFS-6263. Remove DRFA.MaxBackupIndex config from log4j.properties.
- (Abhiraj Butala via aajisaka)
-
- HDFS-6408. Remove redundant definitions in log4j.properties.
- (Abhiraj Butala via aajisaka)
-
- HDFS-7890. Improve information on Top users for metrics in
- RollingWindowsManager and lower log level (J.Andreina via vinayakumarb)
-
- HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times.
- (Vinayakumar B and Keisuke Ogiwara via Arpit Agarwal)
-
- HDFS-7944. Minor cleanup of BlockPoolManager#getAllNamenodeThreads.
- (Arpit Agarwal)
-
- HDFS-7671. hdfs user guide should point to the common rack awareness doc.
- (Kai Sasaki via aajisaka)
-
- HDFS-8009. Signal congestion on the DataNode. (wheat9)
-
- HDFS-7978. Add LOG.isDebugEnabled() guard for some LOG.debug(..).
- (Walter Su via wang)
-
- HDFS-7888. Change DFSOutputStream and DataStreamer for convenience of
- subclassing. (Li Bo via szetszwo)
-
- HDFS-8049. Add @InterfaceAudience.Private annotation to hdfs client
- implementation. (Takuya Fukudome via szetszwo)
-
- HDFS-8079. Move dfs.client.retry.* confs from DFSConfigKeys to
- HdfsClientConfigKeys.Retry. (szetszwo)
-
- HDFS-8073. Split BlockPlacementPolicyDefault.chooseTarget(..) so it
- can be easily overrided. (Walter Su via vinayakumarb)
-
- HDFS-8080. Separate JSON related routines used by WebHdfsFileSystem to a
- package local class. (wheat9)
-
- HDFS-8085. Move CorruptFileBlockIterator to a new hdfs.client.impl package.
- (szetszwo)
-
- HDFS-8076. Code cleanup for DFSInputStream: use offset instead of
- LocatedBlock when possible. (Zhe Zhang via wang)
-
- HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can
- complete without blocks being replicated. (Ming Ma via wang)
-
- HDFS-8089. Move o.a.h.hdfs.web.resources.* to the client jars. (wheat9)
-
- HDFS-7979. Initialize block report IDs with a random number. (wang)
-
- HDFS-8101. DFSClient use of non-constant DFSConfigKeys pulls in WebHDFS
- classes at runtime. (Sean Busbey via atm)
-
- HDFS-8102. Separate webhdfs retry configuration keys from DFSConfigKeys.
- (wheat9)
-
- HDFS-8100. Refactor DFSClient.Conf to a standalone class and separates
- short-circuit related conf to ShortCircuitConf. (szetszwo)
-
- HDFS-8103. Move BlockTokenSecretManager.AccessMode into
- BlockTokenIdentifier. (wheat9)
-
- HDFS-8084. Move dfs.client.failover.* confs from DFSConfigKeys to
- HdfsClientConfigKeys.Failover and fix typos in the dfs.http.client.*
- configuration keys. (szetszwo)
-
- HDFS-7933. fsck should also report decommissioning replicas.
- (Xiaoyu Yao via cnauroth)
-
- HDFS-8083. Move dfs.client.write.* conf from DFSConfigKeys to
- HdfsClientConfigKeys.Write. (szetszwo)
-
- HDFS-8117. More accurate verification in SimulatedFSDataset: replace
- DEFAULT_DATABYTE with patterned data. (Zhe Zhang via wang)
-
- HDFS-8144. Split TestLazyPersistFiles into multiple tests. (Arpit Agarwal)
-
- HDFS-8082. Move dfs.client.read.*, dfs.client.short.circuit.*,
- dfs.client.mmap.* and dfs.client.hedged.read.* conf from DFSConfigKeys
- to HdfsClientConfigKeys. (szetszwo)
-
- HDFS-8165. Move GRANDFATHER_GENERATION_STAMP and GRANDFATER_INODE_ID to
- hdfs-client. (wheat9)
-
- HDFS-8152. Refactoring of lazy persist storage cases. (Arpit Agarwal)
-
- HDFS-8169. Move LocatedBlocks and related classes to hdfs-client. (wheat9)
-
- HDFS-8133. Improve readability of deleted block check (Daryn Sharp via
- Colin P. McCabe)
-
- HDFS-8185. Separate client related routines in HAUtil into a new class.
- (wheat9)
-
- HDFS-8218. Move classes that used by ClientProtocol into hdfs-client.
- (wheat9)
-
- HDFS-4448. Allow HA NN to start in secure mode with wildcard address
- configured (atm via asuresh)
-
- HDFS-8215. Refactor NamenodeFsck#check method. (Takanobu Asanuma
- via szetszwo)
-
- HDFS-8052. Move WebHdfsFileSystem into hadoop-hdfs-client. (wheat9)
-
- HDFS-8176. Record from/to snapshots in audit log for snapshot diff report.
- (J. Andreina via jing9)
-
- HDFS-8280. Code Cleanup in DFSInputStream. (Jing Zhao via wheat9)
-
- HDFS-8283. DataStreamer cleanup and some minor improvement. (szetszwo via
- jing9)
-
- HDFS-5574. Remove buffer copy in BlockReader.skip.
- (Binglin Chang via aajisaka)
-
- HDFS-8200. Refactor FSDirStatAndListingOp. (wheat9)
-
- HDFS-8292. Move conditional in fmt_time from dfs-dust.js to status.html.
- (Charles Lamb via wang)
-
- HDFS-8086. Move LeaseRenewer to the hdfs.client.impl package. (Takanobu
- Asanuma via szetszwo)
-
- HDFS-8249. Separate HdfsConstants into the client and the server side
- class. (wheat9)
-
- HDFS-7397. Add more detail to the documentation for the conf key
- "dfs.client.read.shortcircuit.streams.cache.size" (Brahma Reddy Battula via
- Colin P. McCabe)
-
- HDFS-8237. Move all protocol classes used by ClientProtocol to hdfs-client.
- (wheat9)
-
- HDFS-7758. Retire FsDatasetSpi#getVolumes() and use
- FsDatasetSpi#getVolumeRefs() instead (Lei (Eddy) Xu via Colin P. McCabe)
-
- HDFS-7847. Modify NNThroughputBenchmark to be able to operate on a remote
- NameNode (Charles Lamb via Colin P. McCabe)
-
- HDFS-8314. Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and
- SMALL_BUFFER_SIZE to the users. (Li Lu via wheat9)
-
- HDFS-8207. Improper log message when blockreport interval compared with
- initial delay. (Brahma Reddy Battula and Ashish Singhi via ozawa)
-
- HDFS-7559. Create unit test to automatically compare HDFS related classes
- and hdfs-default.xml. (Ray Chiang via asuresh)
-
- HDFS-5640. Add snapshot methods to FileContext. (Rakesh R via cnauroth)
-
- HDFS-8284. Update documentation about how to use HTrace with HDFS (Masatake
- Iwasaki via Colin P. McCabe)
-
- HDFS-8113. Add check for null BlockCollection pointers in
- BlockInfoContiguous structures (Chengbing Liu via Colin P. McCabe)
-
- HDFS-6757. Simplify lease manager with INodeID. (wheat9)
-
- HDFS-8327. Simplify quota calculations for snapshots and truncate. (wheat9)
-
- HDFS-8357. Consolidate parameters of INode.CleanSubtree() into a parameter
- objects. (Li Lu via wheat9)
-
- HDFS-8255. Rename getBlockReplication to getPreferredBlockReplication.
- (Contributed by Zhe Zhang)
-
- HDFS-6184. Capture NN's thread dump when it fails over.
- (Ming Ma via aajisaka)
-
- HDFS-8350. Remove old webhdfs.xml and other outdated documentation stuff.
- (Brahma Reddy Battula via aajisaka)
-
- HDFS-6888. Allow selectively audit logging ops (Chen He via vinayakumarb)
-
- HDFS-8397. Refactor the error handling code in DataStreamer.
- (Tsz Wo Nicholas Sze via jing9)
-
- HDFS-8394. Move getAdditionalBlock() and related functionalities into a
- separate class. (wheat9)
-
- HDFS-8157. Writes to RAM DISK reserve locked memory for block files.
- (Arpit Agarwal)
-
- HDFS-8345. Storage policy APIs must be exposed via the FileSystem
- interface. (Arpit Agarwal)
-
- HDFS-4185. Add a metric for number of active leases (Rakesh R via raviprak)
-
- HDFS-4383. Document the lease limits. (Arshad Mohammad via aajisaka)
-
- HDFS-8421. Move startFile() and related functions into FSDirWriteFileOp.
- (wheat9)
-
- HDFS-8377. Support HTTP/2 in datanode. (Duo Zhang via wheat9)
-
- HDFS-8482. Rename BlockInfoContiguous to BlockInfo. (Zhe Zhang via wang)
-
- HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml.
- (J.Andreina via aajisaka)
-
- HDFS-8489. Subclass BlockInfo to represent contiguous blocks.
- (Zhe Zhang via jing9)
-
- HDFS-8386. Improve synchronization of 'streamer' reference in
- DFSOutputStream. (Rakesh R via wang)
-
- HDFS-8513. Rename BlockPlacementPolicyRackFaultTolarent to
- BlockPlacementPolicyRackFaultTolerant. (wang)
-
- HDFS-8532. Make the visibility of DFSOutputStream#streamer member variable
- to private. (Rakesh R via wang)
-
- HDFS-8535. Clarify that dfs usage in dfsadmin -report output includes all
- block replicas. (Eddy Xu via wang)
-
- HDFS-8432. Introduce a minimum compatible layout version to allow downgrade
- in more rolling upgrade use cases. (cnauroth)
-
- HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from
- RollingWindowManager. (Brahma Reddy Battula via xyao)
-
- HDFS-8553. Document hdfs class path options.
- (Brahma Reddy Battula via cnauroth)
-
- HDFS-8552. Fix hdfs CLI usage message for namenode and zkfc.
- (Brahma Reddy Battula via xyao)
-
- HDFS-8568. TestClusterId#testFormatWithEmptyClusterIdOption is failing.
- (Rakesh R. via xyao)
-
- HDFS-8549. Abort the balancer if an upgrade is in progress. (wang)
-
- HDFS-8573. Move creation of restartMeta file logic from BlockReceiver to
- ReplicaInPipeline. (Eddy Xu via wang)
-
- HDFS-7923. The DataNodes should rate-limit their full block reports by
- asking the NN on heartbeat messages (cmccabe)
-
- HDFS-8540. Mover should exit with NO_MOVE_BLOCK if no block can be moved.
- (surendra singh lilhore via szetszwo)
-
- HDFS-8606. Cleanup DFSOutputStream by removing unwanted changes
- from HDFS-8386. (Rakesh R via szetszwo)
-
- HDFS-9608. Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead
- of Block in UnderReplicatedBlocks and PendingReplicationBlocks).
- (Zhe Zhang via wang)
-
- HDFS-8238. Move ClientProtocol to the hdfs-client.
- (Takanobu Asanuma via wheat9)
-
- HDFS-8446. Separate safemode related operations in GetBlockLocations().
- (wheat9)
-
- HDFS-8589. Fix unused imports in BPServiceActor and BlockReportLeaseManager
- (cmccabe)
-
- HDFS-6249. Output AclEntry in PBImageXmlWriter.
- (surendra singh lilhore via aajisaka)
-
- HDFS-8605. Merge Refactor of DFSOutputStream from HDFS-7285 branch.
- (vinayakumarb via wang)
-
- HDFS-8582. Support getting a list of reconfigurable config properties and
- do not generate spurious reconfig warnings (Lei (Eddy) Xu via Colin P.
- McCabe)
-
- HDFS-8192. Eviction should key off used locked memory instead of
- ram disk free space. (Arpit Agarwal)
-
- HDFS-6564. Use slf4j instead of common-logging in hdfs-client.
- (Rakesh R via wheat9)
-
- HDFS-8639. Add Option for NameNode HTTP port in MiniDFSClusterManager.
- (Kai Sasaki via jing9)
-
- HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer.
- (Jagadesh Kiran N via aajisaka)
-
- HDFS-8640. Make reserved RBW space visible through JMX. (kanaka kumar
- avvaru via Arpit Agarwal)
-
- HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang)
-
- HDFS-8546. Use try with resources in DataStorage and Storage. (wang)
-
- HDFS-8651. Make hadoop-hdfs-project Native code -Wall-clean (Alan Burlison
- via Colin P. McCabe)
-
- HDFS-8653. Code cleanup for DatanodeManager, DatanodeDescriptor and
- DatanodeStorageInfo. (Zhe Zhang via wang)
-
- HDFS-8493. Consolidate truncate() related implementation in a single class.
- (Rakesh R via wheat9)
-
- HDFS-8635. Migrate HDFS native build to new CMake framework (Alan Burlison
- via Colin P. McCabe)
-
- HDFS-8666. Speedup the TestMover tests. (Walter Su via jing9)
-
- HDFS-8703. Merge refactor of DFSInputStream from ErasureCoding branch
- (vinayakumarb)
-
- HDFS-8709. Clarify automatic sync in FSEditLog#logEdit. (wang)
-
- HDFS-8711. setSpaceQuota command should print the available storage type
- when input storage type is wrong. (Brahma Reddy Battula via xyao)
-
- HDFS-8620. Clean up the checkstyle warinings about ClientProtocol.
- (Takanobu Asanuma via wheat9)
-
- HDFS-8712. Remove 'public' and 'abstracta modifiers in FsVolumeSpi and
- FsDatasetSpi (Lei (Eddy) Xu via vinayakumarb)
-
- HDFS-8726. Move protobuf files that define the client-sever protocols to
- hdfs-client. (wheat9)
-
- HDFS-8751. Remove setBlocks API from INodeFile and misc code cleanup. (Zhe
- Zhang via jing9)
-
- HDFS-8541. Mover should exit with NO_MOVE_PROGRESS if there is no move
- progress. (Surendra Singh Lilhore via szetszwo)
-
- HDFS-8742. Inotify: Support event for OP_TRUNCATE.
- (Surendra Singh Lilhore via aajisaka)
-
- HDFS-8794. Improve CorruptReplicasMap#corruptReplicasMap. (yliu)
-
- HDFS-7483. Display information per tier on the Namenode UI.
- (Benoy Antony and wheat9 via wheat9)
-
- HDFS-8721. Add a metric for number of encryption zones.
- (Rakesh R via cnauroth)
-
- HDFS-8495. Consolidate append() related implementation into a single class.
- (Rakesh R via wheat9)
-
- HDFS-8795. Improve InvalidateBlocks#node2blocks. (yliu)
-
- HDFS-8797. WebHdfsFileSystem creates too many connections for pread. (jing9)
-
- HDFS-8730. Clean up the import statements in ClientProtocol.
- (Takanobu Asanuma via wheat9)
-
- HDFS-8735. Inotify: All events classes should implement toString() API.
- (Surendra Singh Lilhore via aajisaka)
-
- HDFS-7858. Improve HA Namenode Failover detection on the client. (asuresh)
-
- HDFS-8180. AbstractFileSystem Implementation for WebHdfs. (snayak via jghoman)
-
- HDFS-8811. Move BlockStoragePolicy name's constants from
- HdfsServerConstants.java to HdfsConstants.java (vinayakumarb)
-
- HDFS-8822. Add SSD storagepolicy tests in TestBlockStoragePolicy#
- testDefaultPolicies (vinayakumarb)
-
- HDFS-8816. Improve visualization for the Datanode tab in the NN UI. (wheat9)
-
- HDFS-7192. DN should ignore lazyPersist hint if the writer is
- not local. (Arpit Agarwal)
-
- HDFS-6860. BlockStateChange logs are too noisy. (Chang Li and xyao via xyao)
-
- HDFS-8815. DFS getStoragePolicy implementation using single RPC call
- (Surendra Singh Lilhore via vinayakumarb)
-
- HDFS-8856. Make LeaseManager#countPath O(1). (Arpit Agarwal)
-
- HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails.
- (Walter Su via wang)
-
- HDFS-8818. Changes the global moveExecutor to per datanode executors and
- changes MAX_SIZE_TO_MOVE to be configurable. (szetszwo)
-
- HDFS-8805. Archival Storage: getStoragePolicy should not need superuser privilege.
- (Brahma Reddy Battula via jing9)
-
- HDFS-8887. Expose storage type and storage ID in BlockLocation. (wang)
-
- HDFS-8622. Implement GETCONTENTSUMMARY operation for WebImageViewer.
- (Jagadesh Kiran N via aajisaka)
-
- HDFS-7649. Multihoming docs should emphasize using hostnames in
- configurations. (Brahma Reddy Battula via Arpit Agarwal)
-
- HDFS-8824. Do not use small blocks for balancing the cluster. (szetszwo)
-
- HDFS-8883. NameNode Metrics : Add FSNameSystem lock Queue Length.
- (Anu Engineer via xyao)
-
- HDFS-8713. Convert DatanodeDescriptor to use SLF4J logging. (wang)
-
- HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI.
- (wheat9)
-
- HDFS-8801. Convert BlockInfoUnderConstruction as a feature.
- (Jing Zhao via wheat9)
-
- HDFS-8880. NameNode metrics logging. (Arpit Agarwal)
-
- HDFS-8278. When computing max-size-to-move in Balancer, count only the
- storage with remaining >= default block size. (szetszwo)
-
- HDFS-8435. Support CreateFlag in WebHDFS. (Jakob Homan via cdouglas)
-
- HDFS-8826. In Balancer, add an option to specify the source node list
- so that balancer only selects blocks to move from those nodes. (szetszwo)
-
- HDFS-8911. NameNode Metric : Add Editlog counters as a JMX metric.
- (Anu Engineer via Arpit Agarwal)
-
- HDFS-8803. Move DfsClientConf to hdfs-client. (Mingliang Liu via wheat9)
-
- HDFS-8917. Cleanup BlockInfoUnderConstruction from comments and tests.
- (Zhe Zhang via jing9)
-
- HDFS-8884. Fail-fast check in BlockPlacementPolicyDefault#chooseTarget.
- (yliu)
-
- HDFS-8828. Utilize Snapshot diff report to build diff copy list in distcp.
- (Yufei Gu via Yongjun Zhang)
-
- HDFS-8823. Move replication factor into individual blocks. (wheat9)
-
- HDFS-8934. Move ShortCircuitShm to hdfs-client. (Mingliang Liu via wheat9)
-
- HDFS-8928. Improvements for BlockUnderConstructionFeature:
- ReplicaUnderConstruction as a separate class and replicas as an array.
- (jing9)
-
- HDFS-8900. Compact XAttrs to optimize memory footprint. (yliu)
-
- HDFS-8951. Move the shortcircuit package to hdfs-client.
- (Mingliang Liu via wheat9)
-
- HDFS-8896. DataNode object isn't GCed when shutdown, because it has GC
- root in ShutdownHookManager. (Walter Su via jing9)
-
- HDFS-8248. Store INodeId instead of the INodeFile object in
- BlockInfoContiguous. (wheat9)
-
- HDFS-8962. Clean up checkstyle warnings in o.a.h.hdfs.DfsClientConf.
- (Mingliang Liu via wheat9)
-
- HDFS-8865. Improve quota initialization performance. (kihwal)
-
- HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone
- classes from BlockManager. (Mingliang Liu via wheat9)
-
- HDFS-8925. Move BlockReaderLocal to hdfs-client.
- (Mingliang Liu via wheat9)
-
- HDFS-8983. NameNode support for protected directories. (Arpit Agarwal)
-
- HDFS-8980. Remove unnecessary block replacement in INodeFile. (jing9)
-
- HDFS-8990. Move RemoteBlockReader to hdfs-client module.
- (Mingliang via wheat9)
-
- HDFS-8946. Improve choosing datanode storage for block placement. (yliu)
-
- HDFS-8965. Harden edit log reading code against out of memory errors (cmccabe)
-
- HDFS-2070. Add more unit tests for FsShell getmerge (Daniel Templeton via
- Colin P. McCabe)
-
- HDFS-328. Improve fs -setrep error message for invalid replication factors.
- (Daniel Templeton via wang)
-
- HDFS-8890. Allow admin to specify which blockpools the balancer should run
- on. (Chris Trezzo via mingma)
-
- HDFS-9002. Move o.a.h.hdfs.net/*Peer classes to hdfs-client.
- (Mingliang Liu via wheat9)
-
- HDFS-9021. Use a yellow elephant rather than a blue one in diagram. (wang)
-
- HDFS-9012. Move o.a.h.hdfs.protocol.datatransfer.PipelineAck class to
- hadoop-hdfs-client module. (Mingliang Liu via wheat9)
-
- HDFS-8984. Move replication queues related methods in FSNamesystem to
- BlockManager. (wheat9)
-
- HDFS-9019. Adding informative message to sticky bit permission denied
- exception. (xyao)
-
- HDFS-8860. Remove unused Replica copyOnWrite code (Lei (Eddy) Xu via Colin P. McCabe)
-
- HDFS-8716. Introduce a new config specifically for safe mode block count
- (Chang Li via kihwal)
-
- HDFS-7116. Add a command to get the balancer bandwidth
- (Rakesh R via vinayakumarb)
-
- HDFS-8974. Convert docs in xdoc format to markdown.
- (Masatake Iwasaki via aajisaka)
-
- HDFS-6763. Initialize file system-wide quota once on transitioning to active
- (kihwal)
-
- HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method.
- (Mingliang Liu via Arpit Agarwal)
-
- HDFS-8996. Consolidate validateLog and scanLog in FJM#EditLogFile (Zhe
- Zhang via Colin P. McCabe)
-
- HDFS-9010. Replace NameNode.DEFAULT_PORT with HdfsClientConfigKeys.
- DFS_NAMENODE_RPC_PORT_DEFAULT config key. (Mingliang Liu via wheat9)
-
- HDFS-9065. Include commas on # of files, blocks, total filesystem objects
- in NN Web UI. (Daniel Templeton via wheat9)
-
- HDFS-9008. Balancer#Parameters class could use a builder pattern.
- (Chris Trezzo via mingma)
-
- HDFS-8953. DataNode Metrics logging (Kanaka Kumar Avvaru via vinayakumarb)
-
- HDFS-9082. Change the log level in WebHdfsFileSystem.initialize() from INFO
- to DEBUG. (Santhosh Nayak via cnauroth)
-
- HDFS-7986. Allow files / directories to be deleted from the NameNode UI.
- (Ravi Prakash via wheat9)
-
- HDFS-7995. Implement chmod in the HDFS Web UI.
- (Ravi Prakash and Haohui Mai via wheat9)
-
- HDFS-9022. Move NameNode.getAddress() and NameNode.getUri() to
- hadoop-hdfs-client. (Mingliang Liu via wheat9)
-
- HDFS-5802. NameNode does not check for inode type before traversing down a
- path. (Xiao Chen via Yongjun Zhang)
-
- HDFS-9101. Remove deprecated NameNode.getUri() static helper method.
- (Mingliang Liu via wheat9)
-
- HDFS-9111. Move hdfs-client protobuf convert methods from PBHelper to
- PBHelperClient. (Mingliang Liu via wheat9)
-
- HADOOP-12428. Fix inconsistency between log-level guards and statements.
- (Jagadesh Kiran N and Jackie Chang via ozawa)
-
- HDFS-9039. Separate client and server side methods of o.a.h.hdfs.
- NameNodeProxies. (Mingliang Liu via wheat9)
-
- HDFS-8733. Keep server related definition in hdfs.proto on server side.
- (Mingliang Liu via wheat9)
-
- HDFS-9130. Use GenericTestUtils#setLogLevel to the logging level.
- (Mingliang Liu via wheat9)
-
- HDFS-9131 Move config keys used by hdfs-client to HdfsClientConfigKeys.
- (Mingliang Liu via wheat9)
-
- HDFS-7529. Consolidate encryption zone related implementation into a single
- class. (Rakesh R via wheat9)
-
- HDFS-9134. Move LEASE_{SOFTLIMIT,HARDLIMIT}_PERIOD constants from
- HdfsServerConstants to HdfsConstants. (Mingliang Liu via wheat9)
-
- HDFS-5795. RemoteBlockReader2#checkSuccess() shoud print error status.
- (Xiao Chen via Yongjun Zhang)
-
- HDFS-9112. Improve error message for Haadmin when multiple name service IDs
- are configured. (Anu Engineer via jing9)
-
- HDFS-9132. Pass genstamp to ReplicaAccessorBuilder. (Colin Patrick McCabe via
- Lei (Eddy) Xu)
-
- HDFS-9133. ExternalBlockReader and ReplicaAccessor need to return -1 on read
- when at EOF. (Colin Patrick McCabe via Lei (Eddy) Xu)
-
- HDFS-8873. Allow the directoryScanner to be rate-limited (Daniel Templeton
- via Colin P. McCabe)
-
- HDFS-8053. Move DFSIn/OutputStream and related classes to
- hadoop-hdfs-client. (Mingliang Liu via wheat9)
-
- HDFS-9087. Add some jitter to DataNode.checkDiskErrorThread (Elliott Clark
- via Colin P. McCabe)
-
- HDFS-8740. Move DistributedFileSystem to hadoop-hdfs-client. (Mingliang Liu
- via wheat9)
-
- HDFS-9080. Update htrace version to 4.0.1 (cmccabe)
-
- HDFS-9148. Incorrect assert message in TestWriteToReplica#testWriteToTemporary
- (Tony Wu via lei)
-
- HDFS-8859. Improve DataNode ReplicaMap memory footprint to save about 45%.
- (yliu)
-
- HDFS-9165. Move entries in META-INF/services/o.a.h.fs.FileSystem to
- hdfs-client. (Mingliang Liu via wheat9)
-
- HDFS-8696. Make the lower and higher watermark in the DN Netty server
- configurable. (Xiaobing Zhou via wheat9)
-
- HDFS-8971. Remove guards when calling LOG.debug() and LOG.trace() in client
- package. (Mingliang Liu via wheat9)
-
- HDFS-9175. Change scope of 'AccessTokenProvider.getAccessToken()' and
- 'CredentialBasedAccessTokenProvider.getCredential()' abstract methods to
- public (Santhosh Nayak via cnauroth)
-
- HDFS-9015. Refactor TestReplicationPolicy to test different block placement
- policies. (Ming Ma via lei)
-
- HDFS-8979. Clean up checkstyle warnings in hadoop-hdfs-client module.
- (Mingliang Liu via wheat9)
-
- HDFS-9155. OEV should treat .XML files as XML even when the file name
- extension is uppercase (nijel via cmccabe)
-
- HDFS-9170. Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client. (wheat9)
-
- HDFS-8164. cTime is 0 in VERSION file for newly formatted NameNode.
- (Xiao Chen via Yongjun Zhang)
-
- HDFS-9181. Better handling of exceptions thrown during upgrade shutdown.
- (Wei-Chiu Chuang via Yongjun Zhang)
-
- HDFS-9110. Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for
- better efficiency. (Charlie Helin via wang)
-
- HDFS-8988. Use LightWeightHashSet instead of LightWeightLinkedSet in
- BlockManager#excessReplicateMap. (yliu)
-
- HDFS-9006. Provide BlockPlacementPolicy that supports upgrade domain.
- (Ming Ma via lei)
-
- HDFS-1172. Blocks in newly completed files are considered under-replicated
- too quickly. (Masatake Iwasaki via jing9)
-
- HDFS-9238. Update TestFileCreation.testLeaseExpireHardLimit() to avoid using
- DataNodeTestUtils.getFile(). (Tony Wu via lei)
-
- HDFS-9223. Code cleanup for DatanodeDescriptor and HeartbeatManager. (jing9)
-
- HDFS-9188. Make block corruption related tests FsDataset-agnostic. (lei)
-
- HDFS-9205. Do not schedule corrupt blocks for replication. (szetszwo)
-
- HDFS-9250. Add Precondition check to LocatedBlock#addCachedLoc.
- (Xiao Chen via wang)
-
- HDFS-9251. Refactor TestWriteToReplica and TestFsDatasetImpl to avoid
- explicitly creating Files in the tests code. (lei)
-
- HDFS-8647. Abstract BlockManager's rack policy into BlockPlacementPolicy.
- (Brahma Reddy Battula via mingma)
-
- HDFS-9225. Fix intermittent test failure of
- TestBlockManager.testBlocksAreNotUnderreplicatedInSingleRack.
- (Masatake Iwasaki via wang)
-
- HDFS-7087. Ability to list /.reserved. (Xiao Chen via wang)
-
- HDFS-9264. Minor cleanup of operations on FsVolumeList#volumes.
- (Walter Su via lei)
-
- HDFS-8808. dfs.image.transfer.bandwidthPerSec should not apply to
- -bootstrapStandby (zhz)
-
- HDFS-4015. Safemode should count and report orphaned blocks.
- (Anu Engineer via Arpit Agarwal)
-
- HDFS-7284. Add more debug info to
- BlockInfoUnderConstruction#setGenerationStampAndVerifyReplicas.
- (Wei-Chiu Chuang via Yongjun Zhang)
-
- HDFS-9284. fsck command should not print exception trace when file not
- found. (Jagadesh Kiran N via wang)
-
- HDFS-9291. Fix TestInterDatanodeProtocol to be FsDataset-agnostic. (lei)
-
- HDFS-8945. Update the description about replica placement in HDFS
- Architecture documentation. (Masatake Iwasaki via wang)
-
- HDFS-9292. Make TestFileConcorruption independent to underlying FsDataset
- Implementation. (lei)
-
- HDFS-9259. Make SO_SNDBUF size configurable at DFSClient side for hdfs
- write scenario. (Mingliang Liu via mingma)
-
- HDFS-9299. Give ReplicationMonitor a readable thread name (Staffan Friberg
- via Colin P. McCabe)
-
- HDFS-9307. fuseConnect should be private to fuse_connect.c (Mingliang Liu
- via Colin P. McCabe)
-
- HDFS-9311. Support optional offload of NameNode HA service health checks to
- a separate RPC server. (cnauroth)
-
- HDFS-9255. Consolidate block recovery related implementation into a single
- class. (Walter Su via zhz)
-
- HDFS-9295. Add a thorough test of the full KMS code path.
- (Daniel Templeton via zhz)
-
- HDFS-8545. Refactor FS#getUsed() to use ContentSummary and add an API to fetch
- the total file length from a specific path (J.Andreina via vinayakumarb)
-
- HDFS-9229. Expose size of NameNode directory as a metric.
- (Surendra Singh Lilhore via zhz)
-
- HDFS-9339. Extend full test of KMS ACLs. (Daniel Templeton via zhz)
-
- HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma via lei)
-
- HDFS-9331. Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for
- filesystem entirely allocated for DFS use. (Tony Wu via lei)
-
- HDFS-9363. Add fetchReplica() to FsDatasetTestUtils to return FsDataset-agnostic
- replica. (Tony Wu via lei)
-
- HDFS-9377. Fix findbugs warnings in FSDirSnapshotOp.
- (Mingliang Liu via Yongjun Zhang)
-
- HDFS-9236. Missing sanity check for block size during block recovery.
- (Tony Wu via Yongjun Zhang)
-
- HDFS-9379. Make NNThroughputBenchmark$BlockReportStats support more than 10
- datanodes. (Mingliang Liu via Arpit Agarwal)
-
- HDFS-9398. Make ByteArraryManager log message in one-line format.
- (Mingliang Liu via szetszwo)
-
- HDFS-2261. AOP unit tests are not getting compiled or run. (wheat9)
-
- HDFS-9369. Use ctest to run tests for hadoop-hdfs-native-client. (wheat9)
-
- HDFS-9252. Change TestFileTruncate to use FsDatasetTestUtils to get block
- file size and genstamp. (Lei (Eddy) Xu via cmccabe)
-
- HDFS-8056. Decommissioned dead nodes should continue to be counted as dead
- after NN restart. (mingma)
-
- HDFS-9439. Include status of closeAck into exception message in DataNode#run.
- (Xiao Chen via Yongjun Zhang)
-
- HDFS-9402. Switch DataNode.LOG to use slf4j. (Walter Su via wheat9)
-
- HDFS-9153. Pretty-format the output for DFSIO. (Kai Zheng via wheat9)
-
- HDFS-7988. Replace usage of ExactSizeInputStream with LimitInputStream.
- (Walter Su via wheat9)
-
- HDFS-9314. Improve BlockPlacementPolicyDefault's picking of excess
- replicas. (Xiao Chen via mingma)
-
- HDFS-8807. dfs.datanode.data.dir does not handle spaces between
- storageType and URI correctly. (Anu Engineer via szetszwo)
-
- HDFS-9438. TestPipelinesFailover assumes Linux ifconfig.
- (John Zhuge via Yongjun Zhang)
-
- HDFS-8512. WebHDFS : GETFILESTATUS should return LocatedBlock with storage
- type info. (xyao)
-
- HDFS-9269. Update the documentation and wrapper for fuse-dfs.
- (Wei-Chiu Chuang via zhz)
-
- HDFS-9485. Make BlockManager#removeFromExcessReplicateMap accept BlockInfo
- instead of Block. (Mingliang Liu via jing9)
-
- HDFS-9490. MiniDFSCluster should change block generation stamp via
- FsDatasetTestUtils. (Tony Wu via lei)
-
- HDFS-9474. TestPipelinesFailover should not fail when printing debug
- message. (John Zhuge via Yongjun Zhang)
-
- HDFS-9214. Support reconfiguring dfs.datanode.balance.max.concurrent.moves
- without DN restart. (Xiaobing Zhou via Arpit Agarwal)
-
- HDFS-9527. The return type of FSNamesystem.getBlockCollection should be
- changed to INodeFile. (szetszwo)
-
- HDFS-9472. concat() API does not give proper exception messages on ./reserved
- relative path (Rakesh R via umamahesh)
-
- HDFS-9528. Cleanup namenode audit/log/exception messages. (szetszwo via umamahesh)
-
- HDFS-9532. Detailed exception info is lost in reportTo methods of
- ErrorReportAction and ReportBadBlockAction. (Yongjun Zhang)
-
- HDFS-9519. Some coding improvement in SecondaryNameNode#main.
- (Xiao Chen via Yongjun Zhang)
-
- HDFS-9514. TestDistributedFileSystem.testDFSClientPeerWriteTimeout failing;
- exception being swallowed. (Wei-Chiu Chuang via Yongjun Zhang)
-
- HDFS-9535. Newly completed blocks in IBR should not be considered
- under-replicated too quickly. (Mingliang Liu via jing9)
-
- HDFS-8894. Set SO_KEEPALIVE on DN server sockets.
- (Kanaka Kumar Avvaru via wang)
-
- HDFS-9557. Reduce object allocation in PB conversion
- (Daryn Sharp via cnauroth)
-
- HDFS-9552. Document types of permission checks performed for HDFS
- operations. (cnauroth)
-
- HDFS-7779. Support changing ownership, group and replication in HDFS Web
- UI. (Ravi Prakash via wheat9)
-
- HDFS-9498. Move code that tracks blocks with future generation stamps
- to BlockManagerSafeMode. (Mingliang Liu via Arpit Agarwal)
-
- HDFS-9626. TestBlockReplacement#testBlockReplacement fails occasionally.
- (Xiao Chen via zhz)
-
- HDFS-9630. DistCp minor refactoring and clean up. (Kai Zheng via zhz)
-
- HDFS-9415. Document dfs.cluster.administrators and
- dfs.permissions.superusergroup. (Xiaobing Zhou via Arpit Agarwal)
-
- HDFS-9655. NN should start JVM pause monitor before loading fsimage.
- (John Zhuge via Lei (Eddy) Xu)
-
- HDFS-8898. Create API and command-line argument to get quota and quota
- usage without detailed content summary. (Ming Ma via kihwal)
-
- HDFS-9706. Log more details in debug logs in BlockReceiver's constructor.
- (Xiao Chen via Yongjun Zhang)
-
- HDFS-9638. Improve DistCp Help and documentation.
- (Wei-Chiu Chuang via Yongjun Zhang)
-
- HDFS-9721. Allow Delimited PB OIV tool to run upon fsimage that contains
- INodeReference. (Xiao Chen via lei)
-
- HDFS-9503. Use generic option -fs for NNThroughputBenchmark instead of
- -namenode. (Mingliang Liu via shv)
-
- HDFS-9777. Fix typos in DFSAdmin command line and documentation.
- (Wei-Chiu Chuang via umamahesh)
-
- HDFS-9700. DFSClient and DFSOutputStream should set TCP_NODELAY on sockets
- for DataTransferProtocol (Gary Helmling via iwasakims)
-
- HDFS-9644. Update encryption documentation to reflect nested EZs. (zhz)
-
- HDFS-9797. Log Standby exceptions thrown by RequestHedgingProxyProvider
- at DEBUG Level (Inigo Goiri via asuresh)
-
- HDFS-9773. Remove dead code related to SimulatedFSDataset in tests.
- (Brahma Reddy Battula via aajisaka)
-
- HDFS-8923. Add -source flag to balancer usage message. (Chris Trezzo
- via szetszwo)
-
- HDFS-9764. DistCp doesn't print value for several arguments including
- -numListstatusThreads. (Wei-Chiu Chuang via Yongjun Zhang)
-
- HDFS-9084. Pagination, sorting and filtering of files/directories in the
- HDFS Web UI. (Ravi Prakash via ozawa)
-
- HDFS-9844. Correct path creation in getTrashRoot to handle root dir. (zhz)
-
- HDFS-9854. Log cipher suite negotiation more verbosely
- (Wei-Chiu Chuang via cnauroth)
-
- HDFS-9843. Document distcp options required for copying between encrypted
- locations. (Xiaoyu Yao via cnauroth)
-
- HDFS-9831. Document webhdfs retry configuration keys introduced by
- HDFS-5219/HDFS-5122. (Xiaobing Zhou via xyao)
-
- HDFS-9887. WebHdfs socket timeouts should be configurable.
- (Austin Donnelly and Chris Douglas via xyao)
-
- OPTIMIZATIONS
-
- HDFS-8026. Trace FSOutputSummer#writeChecksumChunks rather than
- DFSOutputStream#writeChunk (cmccabe)
-
- HDFS-7433. Optimize performance of DatanodeManager's node map.
- (daryn via kihwal)
-
- HDFS-8792. BlockManager#postponedMisreplicatedBlocks should use a
- LightWeightHashSet to save memory (Yi Liu via Colin P. McCabe)
-
- HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via
- Colin P. McCabe)
-
- HDFS-8862. BlockManager#excessReplicateMap should use a HashMap. (yliu)
-
- HDFS-8929. Add a metric to expose the timestamp of the last journal
- (surendra singh lilhore via vinayakumarb)
-
- HDFS-8829. Make SO_RCVBUF and SO_SNDBUF size configurable for
- DataTransferProtocol sockets and allow configuring auto-tuning (He Tianyi
- via Colin P. McCabe)
-
- HDFS-9139. Enable parallel JUnit tests for HDFS Pre-commit
- (Chris Nauroth and Vinayakumar B via vinayakumarb)
-
- HDFS-9145. Tracking methods that hold FSNamesytemLock for too long.
- (Mingliang Liu via wheat9)
-
- HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead
- of hdfs. (Mingliang Liu via wheat9)
-
- HDFS-9253. Refactor tests of libhdfs into a directory. (wheat9)
-
- HDFS-9280. Document NFS gateway export point parameter. (Xiao Chen via zhz)
-
- HDFS-9297. Update TestBlockMissingException to use corruptBlockOnDataNodesByDeletingBlockFile().
- (Tony Wu via lei)
-
- HDFS-9168. Move client side unit test to hadoop-hdfs-client. (wheat9)
-
- HDFS-9312. Fix TestReplication to be FsDataset-agnostic. (lei)
-
- HDFS-9308. Add truncateMeta() and deleteMeta() to MiniDFSCluster. (Tony Wu via lei)
-
- HDFS-9372. Remove dead code in DataStorage.recoverTransitionRead.
- (Duo Zhang via wheat9)
-
- HDFS-9282. Make data directory count and storage raw capacity related tests
- FsDataset-agnostic. (Tony Wu via lei)
-
- HDFS-9318. considerLoad factor can be improved. (Kuhu Shukla via kihwal)
-
- HDFS-8335. FSNamesystem should construct FSPermissionChecker only if
- permission is enabled. (Gabor Liptak via wheat9)
-
- HDFS-9436. Make NNThroughputBenchmark$BlockReportStats run with 10
- datanodes by default. (Mingliang Liu via shv)
-
- HDFS-8999. Allow a file to be closed with COMMITTED but not yet COMPLETE
- blocks. (szetszwo)
-
- HDFS-9715. Check storage ID uniqueness on datanode startup
- (Lei (Eddy) Xu via vinayakumarb)
-
- HDFS-9726. Refactor IBR code to a new class. (szetszwo)
-
- HDFS-9686. Remove useless boxing/unboxing code.
- (Kousuke Saruta via aajisaka)
-
- HDFS-9768. Reuse ObjectMapper instance in HDFS to improve the performance.
- (Lin Yiqun via aajisaka)
-
- HDFS-9425. Expose number of blocks per volume as a metric
- (Brahma Reddy Battula via vinayakumarb)
-
- HDFS-9710. DN can be configured to send block receipt IBRs in batches.
- (szetszwo)
-
- BUG FIXES
-
- HDFS-7501. TransactionsSinceLastCheckpoint can be negative on SBNs.
- (Gautam Gopalakrishnan via harsh)
-
- HDFS-5356. MiniDFSCluster should close all open FileSystems when shutdown()
- (Rakesh R via vinayakumarb)
-
- HDFS-7867. Update action param from "start" to "prepare" in rolling upgrade
- javadoc (J.Andreina via vinayakumarb)
-
- HDFS-3325. When configuring "dfs.namenode.safemode.threshold-pct" to a value
- greater or equal to 1 there is mismatch in the UI report
- (J.Andreina via vinayakumarb)
-
- HDFS-8002. Website refers to /trash directory. (Brahma Reddy Battula via
- aajisaka)
-
- HDFS-7261. storageMap is accessed without synchronization in
- DatanodeDescriptor#updateHeartbeatState() (Brahma Reddy Battula via Colin
- P. McCabe)
-
- HDFS-7997. The first non-existing xattr should also throw IOException.
- (zhouyingchao via yliu)
-
- HDFS-7922. ShortCircuitCache#close is not releasing
- ScheduledThreadPoolExecutors (Rakesh R via Colin P. McCabe)
-
- HDFS-5215. dfs.datanode.du.reserved is not considered while computing
- available space ( Brahma Reddy Battula via Yongjun Zhang)
-
- HDFS-7725. Incorrect "nodes in service" metrics caused all writes to fail.
- (Ming Ma via wang)
-
- HDFS-8096. DatanodeMetrics#blocksReplicated will get incremented early and
- even for failed transfers (vinayakumarb)
-
- HDFS-7939. Two fsimage_rollback_* files are created which are not deleted
- after rollback. (J.Andreina via vinayakumarb)
-
- HDFS-8111. NPE thrown when invalid FSImage filename given for
- 'hdfs oiv_legacy' cmd ( surendra singh lilhore via vinayakumarb )
-
- HDFS-7701. Support reporting per storage type quota and usage
- with hadoop/hdfs shell. (Peter Shi via Arpit Agarwal)
-
- HDFS-6666. Abort NameNode and DataNode startup if security is enabled but
- block access token is not enabled. (Vijay Bhat via cnauroth)
-
- HDFS-8055. NullPointerException when topology script is missing.
- (Anu Engineer via cnauroth)
-
- HDFS-8142. DistributedFileSystem encryption zone commands should resolve
- relative paths. (Rakesh R via wang)
-
- HDFS-7863. Missing description of some methods and parameters in javadoc of
- FSDirDeleteOp. (Brahma Reddy Battula via ozawa)
-
- HDFS-8043. NPE in MiniDFSCluster teardown. (Brahma Reddy Battula via ozawa)
-
- HDFS-8173. NPE thrown at DataNode shutdown when HTTP server was not able to
- create (surendra singh lilhore via vinayakumarb)
-
- HDFS-7993. Provide each Replica details in fsck (J.Andreina via vinayakumarb)
-
- HDFS-8217. During block recovery for truncate Log new Block Id in case of
- copy-on-truncate is true. (vinayakumarb)
-
- HDFS-8231. StackTrace displayed at client while QuotaByStorageType exceeds
- (J.Andreina and Xiaoyu Yao via vinayakumarb)
-
- HDFS-8191. Fix byte to integer casting in SimulatedFSDataset#simulatedByte.
- (Zhe Zhang via wang)
-
- HDFS-8211. DataNode UUID is always null in the JMX counter. (Anu Engineer
- via Arpit Agarwal)
-
- HDFS-8247. TestDiskspaceQuotaUpdate#testAppendOverTypeQuota is failing.
- (Xiaoyu Yao via cnauroth)
-
- HDFS-8206. Fix the typos in hadoop-hdfs-httpfs. (Brahma Reddy Battula via xyao)
-
- HDFS-8205. CommandFormat#parse() should not parse option as
- value of option. (Peter Shi and Xiaoyu Yao via Arpit Agarwal)
-
- HDFS-8232. Missing datanode counters when using Metrics2 sink interface.
- (Anu Engineer via cnauroth)
-
- HDFS-8214. Secondary NN Web UI shows wrong date for Last Checkpoint. (clamb via wang)
-
- HDFS-8300. Fix unit test failures and findbugs warning caused by HDFS-8283.
- (jing9)
-
- HDFS-8276. LazyPersistFileScrubber should be disabled if scrubber interval
- configured zero. (Surendra Singh Lilhore via Arpit Agarwal)
-
- HDFS-8229. LAZY_PERSIST file gets deleted after NameNode restart.
- (Surendra Singh Lilhore via Arpit Agarwal)
-
- HDFS-8309. Skip unit test using DataNodeTestUtils#injectDataDirFailure() on Windows.
- (xyao)
-
- HDFS-8290. WebHDFS calls before namesystem initialization can cause
- NullPointerException. (cnauroth)
-
- HDFS-8310. Fix TestCLI.testAll "help: help for find" on Windows.
- (Kiran Kumar M R via Xiaoyu Yao)
-
- HDFS-2484. checkLease should throw FileNotFoundException when file does
- not exist. (Rakesh R via shv)
-
- HDFS-7833. DataNode reconfiguration does not recalculate valid volumes
- required, based on configured failed volumes tolerated.
- (Lei (Eddy) Xu via cnauroth)
-
- HDFS-8325. Misspelling of threshold in log4j.properties for tests.
- (Brahma Reddy Battula via aajisaka)
-
- HDFS-8321. CacheDirectives and CachePool operations should throw
- RetriableException in safemode. (wheat9)
-
- HDFS-8037. CheckAccess in WebHDFS silently accepts malformed FsActions
- parameters. (wheat9)
-
- HDFS-8257. Namenode rollingUpgrade option is incorrect in document
- (J.Andreina via vinayakumarb)
-
- HDFS-8067. haadmin prints out stale help messages (Ajith S via vinayakumarb)
-
- HDFS-8174. Update replication count to live rep count in fsck report. (J.Andreina)
-
- HDFS-6291. FSImage may be left unclosed in BootstrapStandby#doRun()
- (Sanghyun Yun via vinayakumarb)
-
- HDFS-7998. HDFS Federation : Command mentioned to add a NN to existing
- federated cluster is wrong (Ajith S via vinayakumarb)
-
- HDFS-8222. Remove usage of "dfsadmin -upgradeProgress" from document which
- is no longer supported. (J.Andreina via aajisaka)
-
- HDFS-8108. Fsck should provide the info on mandatory option to be used along with "-blocks ,
- -locations and -racks" (J.Andreina via umamahesh)
-
- HDFS-8187. Remove usage of "-setStoragePolicy" and "-getStoragePolicy" using
- dfsadmin cmd (as it is not been supported) (J.Andreina via vinayakumarb)
-
- HDFS-8175. Provide information on snapshotDiff for supporting the comparison
- between snapshot and current status (J.Andreina via vinayakumarb)
-
- HDFS-8209. Support different number of datanode directories in MiniDFSCluster.
- (surendra singh lilhore via vinayakumarb)
-
- HDFS-6576. Datanode log is generating at root directory in security mode
- (surendra singh lilhore via vinayakumarb)
-
- HDFS-3384. DataStreamer thread should be closed immediatly when failed to
- setup a PipelineForAppendOrRecovery (Uma Maheswara Rao G via vinayakumarb)
-
- HDFS-6285. tidy an error log inside BlockReceiver. (Liang Xie via umamahesh)
-
- HDFS-8346. libwebhdfs build fails during link due to unresolved external
- symbols. (Chris Nauroth via wheat9)
-
- HDFS-8274. NFS configuration nfs.dump.dir not working. (Ajith S via
- Arpit Agarwal)
-
- HDFS-8340. Fix NFS documentation of nfs.wtmax. (Ajith S via Arpit Agarwal)
-
- HDFS-8311. DataStreamer.transfer() should timeout the socket InputStream.
- (Esteban Gutierrez via Yongjun Zhang)
-
- HDFS-8326. Documentation about when checkpoints are run is out of date.
- (Misty Stanley-Jones via xyao)
-
- HDFS-8097. TestFileTruncate is failing intermittently. (Rakesh R via
- Arpit Agarwal)
-
- HDFS-8351. Remove namenode -finalize option from document. (aajisaka)
-
- HDFS-8362. Java Compilation Error in TestHdfsConfigFields.java
- (Arshad Mohammad via vinayakumarb)
-
- HDFS-8358. TestTraceAdmin fails (Masatake Iwasaki via kihwal)
-
- HDFS-8380. Always call addStoredBlock on blocks which have been shifted
- from one storage to another (cmccabe)
-
- HDFS-8243. Files written by TestHostsFiles and TestNameNodeMXBean are
- causing Release Audit Warnings. (Ruth Wisniewski via Arpit Agarwal)
-
- HDFS-7728. Avoid updating quota usage while loading edits.
- (Jing Zhao via wheat9)
-
- HDFS-8150. Make getFileChecksum fail for blocks under construction
- (J.Andreina via vinayakumarb)
-
- HDFS-8371. Fix test failure in TestHdfsConfigFields for spanreceiver
- properties. (Ray Chiang via aajisaka)
-
- HDFS-8403. Eliminate retries in TestFileCreation
- #testOverwriteOpenForWrite. (Arpit Agarwal via wheat9)
-
- HDFS-6348. SecondaryNameNode not terminating properly on runtime exceptions
- (Rakesh R via vinayakumarb)
-
- HDFS-8454. Remove unnecessary throttling in TestDatanodeDeath.
- (Arpit Agarwal)
-
- HDFS-8268. Port conflict log for data node server is not sufficient
- (Mohammad Shahid Khan via vinayakumarb)
-
- HDFS-8407. hdfsListDirectory must set errno to 0 on success (Masatake
- Iwasaki via Colin P. McCabe)
-
- HDFS-7401. Add block info to DFSInputStream' WARN message when it adds
- node
<TRUNCATED>
[4/5] hadoop git commit: HADOOP-11792. Remove all of the CHANGES.txt
files.
Posted by wa...@apache.org.
http://git-wip-us.apache.org/repos/asf/hadoop/blob/1bc45461/hadoop-common-project/hadoop-common/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/CHANGES.txt b/hadoop-common-project/hadoop-common/CHANGES.txt
deleted file mode 100644
index 78ac5d7..0000000
--- a/hadoop-common-project/hadoop-common/CHANGES.txt
+++ /dev/null
@@ -1,18364 +0,0 @@
-Hadoop Change Log
-
-Trunk (Unreleased)
-
- INCOMPATIBLE CHANGES
-
- HADOOP-8124. Remove the deprecated FSDataOutputStream constructor,
- FSDataOutputStream.sync() and Syncable.sync(). (szetszwo)
-
- HADOOP-10474 Move o.a.h.record to hadoop-streaming. (wheat9)
-
- HADOOP-9902. Shell script rewrite (aw)
-
- HADOOP-10950. rework heap management vars (John Smith via aw)
-
- HADOOP-6857. FsShell should report raw disk usage including replication
- factor. (Byron Wong via shv)
-
- HADOOP-11657. Align the output of `hadoop fs -du` to be more Unix-like.
- (aajisaka)
-
- HADOOP-11553. Formalize the shell API (aw)
-
- HADOOP-11731. Rework the changelog and releasenotes (aw)
-
- HADOOP-11627. Remove io.native.lib.available.
- (Brahma Reddy Battula via aajisaka)
-
- HADOOP-11698. Remove DistCpV1 and Logalyzer.
- (Brahma Reddy Battula via aajisaka)
-
- HADOOP-11356. Removed deprecated o.a.h.fs.permission.AccessControlException.
- (Li Lu via wheat9)
-
- HADOOP-10787 Rename/remove non-HADOOP_*, etc from the shell scripts.
- (aw via vvasudev)
-
- HADOOP-12294. Throw an Exception when fs.permissions.umask-mode is
- misconfigured (Chang Li vha wheat9)
-
- NEW FEATURES
-
- HADOOP-6590. Add a username check for hadoop sub-commands (John Smith via
- aw)
-
- HADOOP-11353. Add support for .hadooprc (aw)
-
- HADOOP-9044. add FindClass main class to provide classpath checking
- of installations (Steve Loughran via aw)
-
- HADOOP-11485. Pluggable shell integration (aw)
-
- HADOOP-11554. Expose HadoopKerberosName as a hadoop subcommand (aw)
-
- HADOOP-11565. Add --slaves shell option (aw)
-
- HADOOP-9642. Configuration to resolve environment variables via
- ${env.VARIABLE} references (Kengo Seki via aw)
-
- HADOOP-7947. Validate XMLs if a relevant tool is available, when using
- scripts (Kengo Seki via aw)
-
- HADOOP-10854. unit tests for the shell scripts (aw)
-
- HADOOP-10949. metrics2 sink plugin for Apache Kafka (Babak Behzad via aw)
-
- IMPROVEMENTS
-
- HADOOP-11203. Allow ditscp to accept bandwitdh in fraction MegaBytes
- (Raju Bairishetti via amareshwari)
-
- HADOOP-8017. Configure hadoop-main pom to get rid of M2E plugin execution
- not covered (Eric Charles via bobby)
-
- HADOOP-8015. ChRootFileSystem should extend FilterFileSystem
- (Daryn Sharp via bobby)
-
- HADOOP-7595. Upgrade dependency to Avro 1.5.3. (Alejandro Abdelnur via atm)
-
- HADOOP-8078. Add capability to turn on security in unit tests. (Jaimin
- Jetly via jitendra)
-
- HADOOP-7757. Test file reference count is at least 3x actual value (Jon
- Eagles via bobby)
-
- HADOOP-8147. test-patch should run tests with -fn to avoid masking test
- failures (Robert Evans via tgraves)
-
- HADOOP-8290. Remove remaining references to hadoop.native.lib (harsh)
-
- HADOOP-8308. Support cross-project Jenkins builds. (tomwhite)
-
- HADOOP-8297. Writable javadocs don't carry default constructor (harsh)
-
- HADOOP-8360. empty-configuration.xml fails xml validation
- (Radim Kolar via harsh)
-
- HADOOP-7659. fs -getmerge isn't guaranteed to work well over non-HDFS
- filesystems (harsh)
-
- HADOOP-8434. Add tests for Configuration setter methods.
- (Madhukara Phatak via suresh)
-
- HADOOP-8523. test-patch.sh doesn't validate patches before building
- (Jack Dintruff via jeagles)
-
- HADOOP-8719. Workaround for kerberos-related log errors upon running any
- hadoop command on OSX. (Jianbin Wei via harsh)
-
- HADOOP-8588. SerializationFactory shouldn't throw a
- NullPointerException if the serializations list is empty.
- (Sho Shimauchi via harsh)
-
- HADOOP-7930. Kerberos relogin interval in UserGroupInformation
- should be configurable (Robert Kanter via harsh)
-
- HADOOP-8838. Colorize the test-patch output sent to JIRA (Harsh J via
- bobby)
-
- HADOOP-8840. Fix the test-patch colorizer to cover all sorts of +1 lines.
- (Harsh J via bobby)
-
- HADOOP-8864. Addendum to HADOOP-8840: Add a coloring case for +0 results
- too. (harsh)
-
- HADOOP-8910. Add examples to GlobExpander#expand method. (suresh)
-
- HADOOP-8920. Add more javadoc to metrics2 related classes. (suresh)
-
- HADOOP-8776. Provide an option in test-patch that can enable/disable
- compiling native code. (Chris Nauroth via suresh)
-
- HADOOP-6616. Improve documentation for rack awareness. (Adam Faris via
- jghoman)
-
- HADOOP-9075. FileContext#FSLinkResolver should be made static.
- (Arpit Agarwal via suresh)
-
- HADOOP-9277. Improve javadoc for FileContext. (Andrew Wang via suresh)
-
- HADOOP-9258 Add stricter tests to FileSystemContractTestBase (stevel)
-
- HADOOP-9486. Promoted Windows and Shell related utils from YARN to Hadoop
- Common. (Chris Nauroth via vinodkv)
-
- HADOOP-8844. Add a plaintext fs -text test-case.
- (Akira AJISAKA via harsh)
-
- HADOOP-9432 Add support for markdown .md files in site documentation
- (stevel)
-
- HADOOP-9186. test-patch.sh should report build failure to JIRA.
- (Binglin Chang via Colin Patrick McCabe)
-
- HADOOP-9833 move slf4j to version 1.7.5 (Kousuke Saruta via stevel)
-
- HADOOP-10325. Improve jenkins javadoc warnings from test-patch.sh (cmccabe)
-
- HADOOP-10342. Add a new method to UGI to use a Kerberos login subject to
- build a new UGI. (Larry McCay via omalley)
-
- HADOOP-10485. Remove dead classes in hadoop-streaming. (wheat9)
-
- HADOOP-11013. CLASSPATH handling should be consolidated, debuggable (aw)
-
- HADOOP-11041. VersionInfo specifies subversion (Tsuyoshi OZAWA via aw)
-
- HADOOP-11092. hadoop shell commands should print usage if not given a
- a class (aw)
-
- HADOOP-11231. Remove dead code in ServletUtil. (Li Lu via wheat9)
-
- HADOOP-11025. hadoop-daemons.sh should just call hdfs directly (Masatake
- Iwasaki via aw)
-
- HADOOP-11150. hadoop command should show the reason on failure by
- invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw)
-
- HADOOP-11208. Replace "daemon" with better name in script subcommands (aw)
-
- HADOOP-10926. Improve smart-apply-patch.sh to apply binary diffs (cmccabe)
-
- HADOOP-11081. Document hadoop properties expected to be set by the shell
- code in *-env.sh (aw)
-
- HADOOP-11352 Clean up test-patch.sh to disable "+1 contrib tests"
- (Akira AJISAKA via stevel)
-
- HADOOP-10788. Rewrite kms to use new shell framework (John Smith via aw)
-
- HADOOP-11058. Missing HADOOP_CONF_DIR generates strange results
- (Masatake Iwasaki via aw)
-
- HADOOP-11460. Deprecate shell vars (John Smith via aw)
-
- HADOOP-11346. Rewrite sls/rumen to use new shell framework (John Smith
- via aw)
-
- HADOOP-6964. Allow compact property description in xml (Kengo Seki
- via aw)
-
- HADOOP-11575. Daemon log documentation is misleading
- (Naganarasimha G R via aw)
-
- HADOOP-11596. Allow smart-apply-patch.sh to add new files in binary git
- patches (raviprak)
-
- HADOOP-11593. Convert site documentation from apt to markdown (stragglers)
- (Masatake Iwasaki via aw)
-
- HADOOP-11673. Skip using JUnit Assume in TestCodec. (Brahma Reddy Battula
- via cdouglas)
-
- HADOOP-10115. Exclude duplicate jars in hadoop package under different
- component's lib (Vinayakumar B via aw)
-
- HADOOP-11524. hadoop_do_classpath_subcommand throws a shellcheck warning.
- (cnauroth)
-
- HADOOP-11781. fix race conditions and add URL support to
- smart-apply-patch.sh (Raymie Stata via aw)
-
- HADOOP-11850. Typos in hadoop-common java docs. (Surendra Singh Lilhore
- via jghoman)
-
- HADOOP-11852. Disable symlinks in trunk. (Andrew Wang)
-
- HADOOP-11857. Fix CommandFormat#commandFormat java doc annotation.
- (J.Andreina via jghoman)
-
- HADOOP-11813. releasedocmaker.py should use today's date instead of
- unreleased (Darrell Taylor via aw)
-
- HADOOP-11590. Update sbin commands and documentation to use new --slaves
- option (aw)
-
- HADOOP-10993. Dump java command line to *.out file
- (Kengo Seki via vinayakumarb)
-
- HADOOP-12016. Typo in FileSystem::listStatusIterator
- (Arthur Vigil via jghoman)
-
- HADOOP-11142. Remove hdfs dfs reference from file system shell
- documentation (Kengo Seki via aw)
-
- HADOOP-12149. copy all of test-patch BINDIR prior to re-exec (aw)
-
- HADOOP-10979. Auto-entries in hadoop_usage (aw)
-
- HADOOP-12249. pull argument parsing into a function (aw)
-
- HADOOP-12436. GlobPattern regex library has performance issues with
- wildcard characters (Matthew Paduano via aw)
-
- HADOOP-12385. Include nested stack trace in SaslRpcClient.getServerToken()
- (stevel)
-
- HADOOP-12133. Add schemas to Maven Assembly XMLs (Gábor Lipták via aw)
-
- HADOOP-12541. make re2j dependency consistent (Matthew Paduano via aw)
-
- HADOOP-11887. Introduce Intel ISA-L erasure coding library for native
- erasure encoding support (Kai Zheng via Colin P. McCabe)
-
- HADOOP-11828. Implement the Hitchhiker erasure coding algorithm.
- (Jack Liuquan via zhz)
-
- BUG FIXES
-
- HADOOP-12617. SPNEGO authentication request to non-default realm gets
- default realm name inserted in target server principal. (mattf)
-
- HADOOP-11473. test-patch says "-1 overall" even when all checks are +1
- (Jason Lowe via raviprak)
-
- HADOOP-9451. Fault single-layer config if node group topology is enabled.
- (Junping Du via llu)
-
- HADOOP-8419. Fixed GzipCode NPE reset for IBM JDK. (Yu Li via eyang)
-
- HADOOP-8177. MBeans shouldn't try to register when it fails to create
- MBeanName. (Devaraj K via umamahesh)
-
- HADOOP-8018. Hudson auto test for HDFS has started throwing javadoc
- (Jon Eagles via bobby)
-
- HADOOP-7610. Fix for hadoop debian package (Eric Yang via gkesavan)
-
- HADOOP-7641. Add Apache License to template config files.
- (Eric Yang via atm)
-
- HADOOP-7621. alfredo config should be in a file not readable by users
- (Alejandro Abdelnur via atm)
-
- HADOOP-7669 Fix newly introduced release audit warning.
- (Uma Maheswara Rao G via stevel)
-
- HADOOP-6220. HttpServer wraps InterruptedExceptions by IOExceptions
- if interrupted in startup (stevel)
-
- HADOOP-7703. Improved exception handling of shutting down web server.
- (Devaraj K via Eric Yang)
-
- HADOOP-7704. Reduce number of object created by JMXJsonServlet.
- (Devaraj K via Eric Yang)
-
- HADOOP-7769. TestJMXJsonServlet is failing. (tomwhite)
-
- HADOOP-7770. ViewFS getFileChecksum throws FileNotFoundException for files in
- /tmp and /user. (Ravi Prakash via jitendra)
-
- HADOOP-8141. Add method to SecurityUtil to init krb5 cipher suites.
- (todd)
-
- HADOOP-7788. Add simple HealthMonitor class to watch an HAService (todd)
-
- HADOOP-8312. testpatch.sh should provide a simpler way to see which
- warnings changed (bobby)
-
- HADOOP-8339. jenkins complaining about 16 javadoc warnings
- (Tom White and Robert Evans via tgraves)
-
- HADOOP-8354. test-patch findbugs may fail if a dependent module is changed
- (Tom White and Robert Evans)
-
- HADOOP-8375. test-patch should stop immediately once it has found
- compilation errors (bobby)
-
- HADOOP-8395. Text shell command unnecessarily demands that a
- SequenceFile's key class be WritableComparable (harsh)
-
- HADOOP-8413. test-patch.sh gives out the wrong links for
- newPatchFindbugsWarnings (Colin Patrick McCabe via bobby)
-
- HADOOP-6871. When the value of a configuration key is set to its
- unresolved form, it causes the IllegalStateException in
- Configuration.get() stating that substitution depth is too large.
- (Arvind Prabhakar via harsh)
-
- HADOOP-8548. test-patch.sh shows an incorrect link in Jekins builds
- (Kihwal Lee via bobby)
-
- HADOOP-8110. Fix trash checkpoint collisions (Jason Lowe via daryn)
-
- HADOOP-8584. test-patch.sh should not immediately exit when no
- tests are added or modified. (Colin Patrick McCabe via eli)
-
- HADOOP-8521. Port StreamInputFormat to new Map Reduce API (madhukara
- phatak via bobby)
-
- HADOOP-8593. Add missed @Override annotations in Metric/Metrics2 package.
- (Brandon Li via suresh)
-
- HADOOP-8623. hadoop jar command should respect HADOOP_OPTS.
- (Steven Willis via suresh)
-
- HADOOP-8684. Deadlock between WritableComparator and WritableComparable.
- (Jing Zhao via suresh)
-
- HADOOP-8786. HttpServer continues to start even if AuthenticationFilter
- fails to init (todd)
-
- HADOOP-8767. Secondary namenode is started on slave nodes instead of
- master nodes. (Giovanni Delussu via suresh)
-
- HADOOP-8821. Fix findbugs warning related to concatenating string in a
- for loop in Configuration#dumpDeprecatedKeys(). (suresh)
-
- HADOOP-7256. Resource leak during failure scenario of closing
- of resources. (Ramkrishna S. Vasudevan via harsh)
-
- HADOOP-8813. Add InterfaceAudience and InterfaceStability annotations
- to RPC Server and Client classes. (Brandon Li via suresh)
-
- HADOOP-8386. hadoop script doesn't work if 'cd' prints to stdout
- (default behavior in some bash setups (esp. Ubuntu))
- (Chiristopher Berner and Andy Isaacson via harsh)
-
- HADOOP-8839. test-patch's -1 on @author tag presence doesn't cause
- a -1 to the overall result (harsh)
-
- HADOOP-8918. test-patch.sh is parsing modified files wrong.
- (Raja Aluri via suresh)
-
- HADOOP-8974. TestDFVariations fails on Windows. (Chris Nauroth via suresh)
-
- HADOOP-9037. Bug in test-patch.sh and precommit build process (Kihwal Lee
- via jlowe)
-
- HADOOP-9131. Turn off TestLocalFileSystem#testListStatusWithColons on
- Windows. (Chris Nauroth via suresh)
-
- HADOOP-9139. improve killKdc.sh (Ivan A. Veselovsky via bobby)
-
- HADOOP-9202. test-patch.sh fails during mvn eclipse:eclipse if patch adds
- a new module to the build (Chris Nauroth via bobby)
-
- HADOOP-9249. hadoop-maven-plugins version-info goal causes build failure
- when running with Clover. (Chris Nauroth via suresh)
-
- HADOOP-9393. TestRPC fails with JDK7. (Andrew Wang via atm)
-
- HADOOP-9394. Port findHangingTest.sh from HBase to Hadoop. (Andrew Wang
- via atm)
-
- HADOOP-9431 TestSecurityUtil#testLocalHostNameForNullOrWild on systems where hostname
- contains capital letters (Chris Nauroth via sanjay)
-
- HADOOP-9261 S3n filesystem can move a directory under itself -and so lose data
- (fixed in HADOOP-9258) (stevel)
-
- HADOOP-9265 S3 blockstore filesystem breaks part of the Filesystem contract
- (fixed in HADOOP-9258) (stevel)
-
- HADOOP-9433 TestLocalFileSystem#testHasFileDescriptor leaks file handle
- (Chris Nauroth via sanjay)
-
- HADOOP-9583. test-patch gives +1 despite build failure when running tests.
- (jlowe via kihwal)
-
- HADOOP-9847. TestGlobPath symlink tests fail to cleanup properly.
- (cmccabe via wang)
-
- HADOOP-9740. Fix FsShell '-text' command to be able to read Avro
- files stored in HDFS and other filesystems. (Allan Yan via cutting)
-
- HADOOP-10044 Improve the javadoc of rpc code (sanjay Radia)
-
- HADOOP-10625. Trim configuration names when putting/getting them
- to properties. (Wangda Tan via xgong)
-
- HADOOP-10831. UserProvider is not thread safe. (Benoy Antony via umamahesh)
-
- HADOOP-10834. Typo in CredentialShell usage. (Benoy Antony via umamahesh)
-
- HADOOP-11002. shell escapes are incompatible with previous releases (aw)
-
- HADOOP-10996. Stop violence in the *_HOME (aw)
-
- HADOOP-11033. shell scripts ignore JAVA_HOME on OS X. (aw)
-
- HADOOP-11052. hadoop_verify_secure_prereq's results aren't checked
- in bin/hdfs (aw)
-
- HADOOP-11055. non-daemon pid files are missing (aw)
-
- HADOOP-11022. User replaced functions get lost 2-3 levels deep (e.g.,
- sbin) (aw)
-
- HADOOP-11284. Fix variable name mismatches in hadoop-functions.sh (Masatake
- Iwasaki via aw)
-
- HADOOP-11298. slaves.sh and stop-all.sh are missing slashes (aw)
-
- HADOOP-11296. hadoop-daemons.sh throws 'host1: bash: host3:
- command not found...' (vinayakumarb)
-
- HADOOP-11380. Restore Rack Awareness documenation (aw)
-
- HADOOP-11397. Can't override HADOOP_IDENT_STRING (Kengo Seki via aw)
-
- HADOOP-10908. Common needs updates for shell rewrite (aw)
-
- HADOOP-11559. Add links to RackAwareness and InterfaceClassification
- to site index (Masatake Iwasaki via aw)
-
- HADOOP-11580. Remove SingleNodeSetup.md from trunk (aajisaka)
-
- HADOOP-11583. Fix syntax error in SecureMode.md (Masatake Iwasaki via aw)
-
- HADOOP-11437. Remove the version and author information from distcp's
- README file (Brahma Reddy Battula via aw)
-
- HADOOP-11585. Fix formatting in Tracing.md (Masatake Iwasaki via aw)
-
- HADOOP-11625. Minor fixes to command manual & SLA doc (aw)
-
- HADOOP-11637. bash location hard-coded in shell scripts (aw)
-
- HADOOP-10774. Update KerberosTestUtils for hadoop-auth tests when using
- IBM Java (sangamesh via aw)
-
- HADOOP-11653. shellprofiles should require .sh extension
- (Brahma Reddy Battula via aw)
-
- HADOOP-11668. hadoop-daemons.sh bw compat broke with --slaves change
- (Vinayakumar B via aw)
-
- HADOOP-11703. git should ignore .DS_Store files on Mac OS X (Abin Shahab
- via aw)
-
- HADOOP-11797. releasedocmaker.py needs to put ASF headers on output (aw)
-
- HADOOP-11865. Incorrect path mentioned in document for accessing script
- files (J.Andreina via aw)
-
- HADOOP-11775. Fix Javadoc typos in hadoop-openstack module (Yanjun Wang
- via aw)
-
- HADOOP-9891. CLIMiniCluster instructions fail with MiniYarnCluster
- ClassNotFoundException (Darrell Taylor via aw)
-
- HADOOP-11406. xargs -P is not portable (Kengo Seki via aw)
-
- HADOOP-11983. HADOOP_USER_CLASSPATH_FIRST works the opposite of what it is
- supposed to do (Sangjin Lee via aw)
-
- HADOOP-12022. fix site -Pdocs -Pdist in hadoop-project-dist; cleanout
- remaining forrest bits (aw)
-
- HADOOP-9905. remove dependency of zookeeper for hadoop-client (vinayakumarb)
-
- HADOOP-11347. RawLocalFileSystem#mkdir and create should honor umask (Varun
- Saxena via Colin P. McCabe)
-
- HADOOP-12107. long running apps may have a huge number of StatisticsData
- instances under FileSystem (Sangjin Lee via Ming Ma)
-
- HADOOP-11762. Enable swift distcp to secure HDFS (Chen He via aw)
-
- HADOOP-12009. Clarify FileSystem.listStatus() sorting order & fix
- FileSystemContractBaseTest:testListStatus. (J.Andreina via jghoman)
-
- HADOOP-12244. recover broken rebase during precommit (aw)
-
- HADOOP-11942. Add links to SLGUserGuide to site index.
- (Masatake Iwasaki via xyao)
-
- HADOOP-12364. Deleting pid file after stop is causing the daemons to
- keep restarting (Siqi Li via aw)
-
- HADOOP-12515. Mockito dependency is missing in hadoop-kafka module.
- (Kai Zheng via aajisaka)
-
- HADOOP-12494. fetchdt stores the token based on token kind instead
- of token service (HeeSoo Kim via aw)
-
- HADOOP-12553. [JDK8] Fix javadoc error caused by illegal tag. (aajisaka)
-
- HADOOP-12638. UnsatisfiedLinkError while checking ISA-L in checknative
- command. (Kai Sasaki via Colin P. McCabe)
-
- HADOOP-12615. Fix NPE in MiniKMS.start(). (Wei-Chiu Chuang via zhz)
-
- HADOOP-12647. Add /usr/lib default for ISAL library and fix missing .c file
- in CMakeLists (Kai Zheng via cmccabe)
-
- HADOOP-12070. Some of the bin/hadoop subcommands are not available
- on Windows. (Kengo Seki via Arpit Agarwal)
-
- OPTIMIZATIONS
-
- HADOOP-7761. Improve the performance of raw comparisons. (todd)
-
- HADOOP-8589. ViewFs tests fail when tests and home dirs are nested (sanjay Radia)
-
- BREAKDOWN OF HADOOP-11264 SUBTASKS AND RELATED JIRAS (Common part of HDFS-7285)
-
- HADOOP-11514. Raw Erasure Coder API for concrete encoding and decoding
- (Kai Zheng via umamahesh)
-
- HADOOP-11534. Minor improvements for raw erasure coders
- ( Kai Zheng via vinayakumarb )
-
- HADOOP-11541. Raw XOR coder
- ( Kai Zheng )
-
- HADOOP-11542. Raw Reed-Solomon coder in pure Java. Contributed by Kai Zheng
- ( Kai Zheng )
-
- HADOOP-11643. Define EC schema API for ErasureCodec. Contributed by Kai Zheng
- ( Kai Zheng )
-
- HADOOP-11646. Erasure Coder API for encoding and decoding of block group
- ( Kai Zheng via vinayakumarb )
-
- HADOOP-11705. Make erasure coder configurable. Contributed by Kai Zheng
- ( Kai Zheng )
-
- HADOOP-11706. Refine a little bit erasure coder API. Contributed by Kai Zheng
- ( Kai Zheng )
-
- HADOOP-11707. Add factory to create raw erasure coder. Contributed by Kai Zheng
- ( Kai Zheng )
-
- HADOOP-11647. Reed-Solomon ErasureCoder. Contributed by Kai Zheng
- ( Kai Zheng )
-
- HADOOP-11782 Correct two thrown messages in ECSchema class. Contributed by Xinwei Qin
- ( Xinwei Qin via Kai Zheng )
-
- HADOOP-11740. Combine erasure encoder and decoder interfaces (Zhe Zhang)
-
- HADOOP-11805 Better to rename some raw erasure coders. Contributed by Kai Zheng
- ( Kai Zheng )
-
- HADOOP-11645. Erasure Codec API covering the essential aspects for an erasure code
- ( Kai Zheng via vinayakumarb )
-
- HADOOP-11818. Minor improvements for erasurecode classes. (Rakesh R via Kai Zheng)
-
- HADOOP-11841. Remove unused ecschema-def.xml files. (szetszwo)
-
- HADOOP-11921. Enhance tests for erasure coders. (Kai Zheng via Zhe Zhang)
-
- HADOOP-11920. Refactor some codes for erasure coders. (Kai Zheng via Zhe Zhang)
-
- HADOOP-11566. Add tests and fix for erasure coders to recover erased parity
- units. (Kai Zheng via Zhe Zhang)
-
- HADOOP-11938. Enhance ByteBuffer version encode/decode API of raw erasure
- coder. (Kai Zheng via Zhe Zhang)
-
- HADOOP-12013. Generate fixed data to perform erasure coder test. (Kai Zheng)
-
- HADOOP-12029. Remove chunkSize from ECSchema as its not required for coders
- (vinayakumarb)
-
- HADOOP-11847. Enhance raw coder allowing to read least required inputs in decoding.
- (Kai Zheng)
-
- HADOOP-12011. Allow to dump verbose information to ease debugging in raw erasure coders
- (Kai Zheng)
-
- HADOOP-12065. Using more meaningful keys in EC schema. (Kai Zheng)
-
- HADOOP-12060. Fix ByteBuffer usage for raw erasure coders. (Kai Zheng via
- jing9)
-
- HADOOP-11921. Enhance tests for erasure coders. (Kai Zheng)
-
- HADOOP-12327. Initialize output buffers with ZERO bytes in erasure coder.
- (Kai Zheng via waltersu4549)
-
- HADOOP-12047. Indicate preference not to affect input buffers during
- coding in erasure coder. (Kai Zheng via waltersu4549)
-
- HADOOP-12544. Erasure Coding: create dummy raw coder to isolate performance
- issues in testing. (Rui Li via zhz)
-
- HADOOP-12685. Input buffer position after encode/decode not consistent
- between different kinds of buffers. (Rui Li via zhz)
-
- HADOOP-12757. Findbug compilation fails for 'Kafka Library support'.
- (aajisaka)
-
- HADOOP-12041. Implement another Reed-Solomon coder in pure Java.
- (Kai Zheng via zhz)
-
- HADOOP-12808. Rename the RS coder from HDFS-RAID as legacy.
- (Rui Li via zhz)
-
-Release 2.9.0 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- HADOOP-12702. Add an HDFS metrics sink. (Daniel Templeton via kasha)
-
- IMPROVEMENTS
-
- HADOOP-12321. Make JvmPauseMonitor an AbstractService.
- (Sunil G via Stevel)
-
- HADOOP-12625. Add a config to disable the /logs endpoints (rkanter)
-
- HADOOP-12566. Add NullGroupMapping. (Daniel Templeton via kasha)
-
- HADOOP-12663. Remove Hard-Coded Values From FileSystem.java.
- (BELUGA BEHR via stevel)
-
- HADOOP-12653. Use SO_REUSEADDR to avoid getting "Address already in use"
- when using kerberos and attempting to bind to any port on the local IP
- address (cmccabe)
-
- HADOOP-12683. Add number of samples in last interval in snapshot of
- MutableStat. (Vikram Srivastava via kasha)
-
- HADOOP-8887. Use a Maven plugin to build the native code using CMake
- (cmccabe)
-
- HADOOP-12713. Disable spurious checkstyle checks. (wang)
-
- HADOOP-12662. The build should fail if a -Dbundle option fails (Kai Zheng
- via cmccabe)
-
- HADOOP-12749. Create a threadpoolexecutor that overrides afterExecute to
- log uncaught exceptions/errors. (Sidharta Seethana via vvasudev)
-
- HADOOP-12710. Remove dependency on commons-httpclient for TestHttpServerLogs
- (Wei-Chiu Chuang via iwasakims)
-
- HADOOP-12817. Enable TLS v1.1 and 1.2 (rkanter)
-
- HADOOP-12841. Update s3-related properties in core-default.xml. (Wei-Chiu Chuang via lei)
-
- HADOOP-10321. TestCompositeService should cover all enumerations of
- adding a service to a parent service. (Ray Chiang via kasha)
-
- HADOOP-12859. Disable hiding field style checks in class setters. (Kai Zheng via wang)
-
- BUG FIXES
-
- HADOOP-12605. Fix intermittent failure of TestIPC.testIpcWithReaderQueuing
- (iwasakims)
-
- HADOOP-12655. TestHttpServer.testBindAddress bind port range is wider
- than expected. (Wei-Chiu Chuang via stevel)
-
- HADOOP-12613. TestFind.processArguments occasionally fails.
- (Wei-Chiu Chuang via stevel)
-
- HADOOP-12597. In kms-site.xml configuration
- "hadoop.security.keystore.JavaKeyStoreProvider.password"
- should be updated with new name. (Surendra Singh Lilhore via stevel)
-
- HADOOP-12573. TestRPC.testClientBackOff failing.
- (Xiao Chen via stevel)
-
- HDFS-9517. Fix missing @Test annotation on
- TestDistCpUtils.testUnpackAttributes (Wei-Chiu Chuang via cmccabe)
-
- HADOOP-12712. Fix some cmake plugin and native build warnings (cmccabe)
-
- HADOOP-12714. Fix hadoop-mapreduce-client-nativetask unit test which fails
- because it is not able to open the "glibc bug spill" file. (cmccabe)
-
- HADOOP-12829. StatisticsDataReferenceCleaner swallows interrupt exceptions
- (Gregory Chanan via cmccabe)
-
-Release 2.8.0 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- HADOOP-11746. rewrite test-patch.sh (aw)
-
- HADOOP-12416. Trash messages should be handled by Logger instead of being
- delivered on System.out. (Mingliang Liu via aajisaka)
-
- HADOOP-12552. Fix undeclared/unused dependency to httpclient (iwasakims)
-
- NEW FEATURES
-
- HADOOP-11226. Add a configuration to set ipc.Client's traffic class with
- IPTOS_LOWDELAY|IPTOS_RELIABILITY. (Gopal V via ozawa)
-
- HADOOP-9477. Add posixGroups support for LDAP groups mapping service.
- (Dapeng Sun via Yongjun Zhang)
-
- HADOOP-11843. Make setting up the build environment easier.
- (Niels Basjes via cnauroth)
-
- HADOOP-11949. Add user-provided plugins to test-patch (Sean Busbey via aw)
-
- HADOOP-8934. Shell command ls should include sort options (Jonathan Allen
- via aw)
-
- HADOOP-10971. Add -C flag to make `hadoop fs -ls` print filenames only.
- (Kengo Seki via aajisaka)
-
- HADOOP-5732. Add SFTP FileSystem. (Ramtin Boustani and Inigo Goiri via
- cdouglas)
-
- HADOOP-12360. Create StatsD metrics2 sink. (Dave Marion via stevel)
-
- HADOOP-10300. Allowed deferred sending of call responses. (Daryn Sharp via
- yliu)
-
- HADOOP-12657. Add a option to skip newline on empty files with getMerge -nl.
- (Kanaka Kumar Avvaru via aajisaka)
-
- HADOOP-11262. Enable YARN to use S3A. (Pieter Reuse via lei)
-
- HADOOP-12691. Add CSRF Filter for REST APIs to Hadoop Common.
- (Larry McCay via cnauroth)
-
- HADOOP-12635. Adding Append API support for WASB. (Dushyanth via cnauroth)
-
- HADOOP-12426. Add Entry point for Kerberos health check
- (Steve Loughran via cnauroth)
-
- HADOOP-12548. Read s3a creds from a Credential Provider.
- (Larry McCay via cnauroth)
-
- IMPROVEMENTS
-
- HADOOP-12831. LocalFS/FSOutputSummer NPEs in constructor if bytes per checksum
- set to 0 (Mingliang Liu via gtcarrera9)
-
- HADOOP-12458. Retries is typoed to spell Retires in parts of
- hadoop-yarn and hadoop-common
- (Neelesh Srinivas Salian via harsh)
-
- HADOOP-12271. Hadoop Jar Error Should Be More Explanatory
- (Josh Elser via harsh)
-
- HADOOP-6842. "hadoop fs -text" does not give a useful text representation
- of MapWritable objects (Akira Ajisaka via bobby)
-
- HADOOP-11719. [Fsshell] Remove bin/hadoop reference from
- GenericOptionsParser default help text.
- (Brahma Reddy Battula via harsh)
-
- HADOOP-11692. Improve authentication failure WARN message to avoid user
- confusion. (Yongjun Zhang)
-
- HADOOP-11659. o.a.h.fs.FileSystem.Cache#remove should use a single hash map
- lookup. (Brahma Reddy Battula via aajisaka)
-
- HADOOP-11709. Time.NANOSECONDS_PER_MILLISECOND - use class-level final
- constant instead of method variable (Ajith S via ozawa)
-
- HADOOP-11447. Add a more meaningful toString method to SampleStat and
- MutableStat. (kasha)
-
- HADOOP-11737. mockito's version in hadoop-nfs’ pom.xml shouldn't be
- specified. (Kengo Seki via ozawa)
-
- HADOOP-11741. Add LOG.isDebugEnabled() guard for some LOG.debug().
- (Walter Su via ozawa)
-
- HADOOP-11660. Add support for hardware crc of HDFS checksums on ARM aarch64
- architecture (Edward Nevill via Colin P. McCabe)
-
- HADOOP-9805. Refactor RawLocalFileSystem#rename for improved testability.
- (Jean-Pierre Matsumoto via cnauroth)
-
- HADOOP-11717. Support JWT tokens for web single sign on to the Hadoop
- servers. (Larry McCay via omalley)
-
- HADOOP-11814. Reformat hadoop-annotations, o.a.h.classification.tools.
- (Li Lu via wheat9)
-
- HADOOP-7713. dfs -count -q should label output column (Jonathan Allen
- via aw)
-
- HADOOP-11819. HttpServerFunctionalTest#prepareTestWebapp should create web
- app directory if it does not exist. (Rohith via vinayakumarb)
-
- HADOOP-11827. Speed-up distcp buildListing() using threadpool
- (Zoran Dimitrijevic via raviprak)
-
- HADOOP-10597. RPC Server signals backoff to clients when all request
- queues are full. (Ming Ma via Arpit Agarwal)
-
- HADOOP-11357. Print information of the build enviornment in test-patch.sh
- (aw)
-
- HADOOP-11328. ZKFailoverController does not log Exception when doRun raises
- errors. (Tianyin Xu via ozawa)
-
- HADOOP-11120. hadoop fs -rmr gives wrong advice. (Juliet Hougland via wang)
-
- HADOOP-11911. test-patch should allow configuration of default branch
- (Sean Busbey via aw)
-
- HADOOP-11917. test-patch.sh should work with ${BASEDIR}/patchprocess
- setups (aw)
-
- HADOOP-11925. backport trunk's smart-apply-patch.sh to branch-2 (aw)
-
- HADOOP-11906. test-patch.sh should use 'file' command for patch
- determinism (Sean Busbey via aw)
-
- HADOOP-11950. Add cli option to test-patch to set the project-under-test
- (Sean Busbey via aw)
-
- HADOOP-11948. test-patch's issue matching regex should be configurable.
- (Sean Busbey via aw)
-
- HADOOP-9723. Improve error message when hadoop archive output path already
- exists. (Jean-Baptiste Onofré and Yongjun Zhang via aajisak)
-
- HADOOP-11713. ViewFileSystem should support snapshot methods.
- (Rakesh R via cnauroth)
-
- HADOOP-11960. Enable Azure-Storage Client Side logging.
- (Dushyanth via cnauroth)
-
- HADOOP-11939. Deprecate DistCpV1 and Logalyzer.
- (Brahma Reddy Battula via aajisaka)
-
- HADOOP-11884. test-patch.sh should pull the real findbugs version
- (Kengo Seki via aw)
-
- HADOOP-11944. add option to test-patch to avoid relocating patch process
- directory (Sean Busbey via aw)
-
- HADOOP-1540. Support file exclusion list in distcp. (Rich Haase via jing9)
-
- HADOOP-11103. Clean up RemoteException (Sean Busbey via vinayakumarb)
-
- HADOOP-11995. Make jetty version configurable from the maven command line.
- (Sriharsha Devineni via wheat9)
-
- HADOOP-10366. Add whitespaces between classes for values in
- core-default.xml to fit better in browser.
- (kanaka kumar avvaru via aajisaka)
-
- HADOOP-11594. Improve the readability of site index of documentation.
- (Masatake Iwasaki via aajisaka)
-
- HADOOP-12030. test-patch should only report on newly introduced
- findbugs warnings. (Sean Busbey via aw)
-
- HADOOP-11894. Bump the version of Apache HTrace to 3.2.0-incubating
- (Masatake Iwasaki via Colin P. McCabe)
-
- HADOOP-12043. Display warning if defaultFs is not set when running fs
- commands. (Lei Xu via wang)
-
- HADOOP-12037. Fix wrong classname in example configuration of hadoop-auth
- documentation. (Masatake Iwasaki via wang)
-
- HADOOP-12059. S3Credentials should support use of CredentialProvider.
- (Sean Busbey via wang)
-
- HADOOP-12056. Use DirectoryStream in DiskChecker#checkDirs to detect
- errors when listing a directory. (Zhihai Xu via wang)
-
- HADOOP-12055. Deprecate usage of NativeIO#link. (Andrew Wang via cnauroth)
-
- HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to
- hadoop-common. (Masatake Iwasaki via aajisaka)
-
- HADOOP-11965. determine-flaky-tests needs a summary mode.
- (Yufei Gu via Yongjun Zhang)
-
- HADOOP-11958. MetricsSystemImpl fails to show backtrace when an error
- occurs (Jason Lowe via jeagles)
-
- HADOOP-12158. Improve error message in
- TestCryptoStreamsWithOpensslAesCtrCryptoCodec when OpenSSL is not
- installed. (wang)
-
- HADOOP-12124. Add HTrace support for FsShell (cmccabe)
-
- HADOOP-12171. Shorten overly-long htrace span names for server (cmccabe)
-
- HADOOP-12045. Enable LocalFileSystem#setTimes to change atime.
- (Kazuho Fujii via cnauroth)
-
- HADOOP-11974. Fix FIONREAD #include on Solaris (Alan Burlison via Colin P.
- McCabe)
-
- HADOOP-12193. Rename Touchz.java to Touch.java. (wang)
-
- HADOOP-12195. Add annotation to package-info.java file to workaround
- MCOMPILER-205. (wang)
-
- HADOOP-12201. Add tracing to FileSystem#createFileSystem and Globber#glob
- (cmccabe)
-
- HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common.
- (Chris Douglas via kasha)
-
- HADOOP-12210. Collect network usage on the node (Robert Grandl via cdouglas)
-
- HADOOP-12211. Collect disk usage on the node (Robert Grandl via cdouglas)
-
- HADOOP-12153. ByteBufferReadable doesn't declare @InterfaceAudience and
- @InterfaceStability. (Brahma Reddy Battula via ozawa)
-
- HADOOP-11893. Mark org.apache.hadoop.security.token.Token as
- @InterfaceAudience.Public. (Brahma Reddy Battula via stevel)
-
- HADOOP-12081. Fix UserGroupInformation.java to support 64-bit zLinux.
- (aajisaka)
-
- HADOOP-12214. Parse 'HadoopArchive' commandline using cli Options.
- (vinayakumarb)
-
- HADOOP-12184. Remove unused Linux-specific constants in NativeIO (Martin
- Walsh via Colin P. McCabe)
-
- HADOOP-12161. Add getStoragePolicy API to the FileSystem interface.
- (Brahma Reddy Battula via Arpit Agarwal)
-
- HADOOP-12189. Improve CallQueueManager#swapQueue to make queue elements
- drop nearly impossible. (Zhihai Xu via wang)
-
- HADOOP-12259. Utility to Dynamic port allocation (brahmareddy via rkanter)
-
- HADOOP-12170. hadoop-common's JNIFlags.cmake is redundant and can be
- removed (Alan Burlison via Colin P. McCabe)
-
- HADOOP-11807. add a lint mode to releasedocmaker (ramtin via aw)
-
- HADOOP-12183. Annotate the HTrace span created by FsShell with the
- command-line arguments passed by the user (Masatake Iwasaki via Colin P.
- McCabe)
-
- HADOOP-12318. Expose underlying LDAP exceptions in SaslPlainServer. (Mike
- Yoder via atm)
-
- HADOOP-9891. CLIMiniCluster instructions fail with MiniYarnCluster
- ClassNotFoundException (Darrell Taylor via aw)
-
- HADOOP-12295. Improve NetworkTopology#InnerNode#remove logic. (yliu)
-
- HADOOP-12050. Enable MaxInactiveInterval for hadoop http auth token
- (hzlu via benoyantony)
-
- HADOOP-12325. RPC Metrics : Add the ability track and log slow RPCs.
- (Anu Engineer via xyao)
-
- HADOOP-12368. Mark ViewFileSystemBaseTest and ViewFsBaseTest as abstract.
- (wang)
-
- HADOOP-12367. Move TestFileUtil's test resources to resources folder.
- (wang via yliu)
-
- HADOOP-12369. Point hadoop-project/pom.xml java.security.krb5.conf
- within target folder. (wang)
-
- HADOOP-12358. Add -safely flag to rm to prompt when deleting many files.
- (xyao via wang)
-
- HADOOP-12384. Add "-direct" flag option for fs copy so that user can choose
- not to create "._COPYING_" file (J.Andreina via vinayakumarb)
-
- HADOOP-12324. Better exception reporting in SaslPlainServer.
- (Mike Yoder via stevel)
-
- HADOOP-12404. Disable caching for JarURLConnection to avoid sharing
- JarFile with other users when loading resource from URL in Configuration
- class. (zxu)
-
- HADOOP-12428. Fix inconsistency between log-level guards and statements.
- (Jagadesh Kiran N and Jackie Chang via ozawa)
-
- HADOOP-12446. Undeprecate createNonRecursive() (Ted Yu via kihwal)
-
- HADOOP-12452. Fix tracing documention reflecting the update to htrace-4
- (Masatake Iwasaki via Colin P. McCabe)
-
- HADOOP-12350. WASB Logging: Improve WASB Logging around deletes, reads and
- writes (Dushyanth via cnauroth)
-
- HADOOP-11104. org.apache.hadoop.metrics2.lib.MetricsRegistry needs numerical
- parameter checking. (Ray Chiang via aajisaka)
-
- HADOOP-12344. Improve validateSocketPathSecurity0 error message (Casey
- Brotherton via Colin P. McCabe)
-
- HADOOP-10775. Shell operations to fail with meaningful errors on windows if
- winutils.exe not found. (stevel)
-
- HADOOP-12481. JWTRedirectAuthenticationHandler doesn't Retain Original Query
- String (Larry McCay via cnauroth)
-
- HADOOP-12460. Add overwrite option for 'get' shell command
- (Jagadesh Kiran N via vinayakumarb)
-
- HADOOP-10406. TestIPC.testIpcWithReaderQueuing may fail. (Xiao Chen via wang)
-
- HADOOP-9692. Improving log message when SequenceFile reader throws
- EOFException on zero-length file. (Zhe Zhang and Chu Tong via ozawa)
-
- HADOOP-7266. Deprecate metrics v1. (Akira AJISAKA via ozawa)
-
- HADOOP-12472. Make GenericTestUtils.assertExceptionContains robust.
- (Steve Loughran via jing9)
-
- HADOOP-12514. Make static fields in GenericTestUtils for
- assertExceptionContains() package-private and final.
- (Mingliang Liu via stevel)
-
- HADOOP-12520. Use XInclude in hadoop-azure test configuration to isolate
- Azure Storage account keys for service integration tests. (cnauroth)
-
- HADOOP-11685. StorageException complaining " no lease ID" during HBase
- distributed log splitting (Duo Xu via cnauroth)
-
- HADOOP-12040. Adjust inputs order for the decode API in raw erasure coder.
- (Kai Zheng via yliu)
-
- HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor
- and Aaron Fabbri via lei)
-
- HADOOP-12562. Make hadoop dockerfile usable by Yetus.
- (Allen Wittenauer via wheat9)
-
- HADOOP-12568. Update core-default.xml to describe posixGroups support.
- (Wei-Chiu Chuang via aajisaka)
-
- HADOOP-12564. Upgrade JUnit3 TestCase to JUnit 4 in
- org.apache.hadoop.io package. (Dustin Cote via ozawa)
-
- HADOOP-12582. Using BytesWritable's getLength() and getBytes() instead
- of get() and getSize(). (Akira AJISAKA via ozawa)
-
- HADOOP-12575. Add build instruction for docker toolbox instead of
- boot2docker(Kai Sasaki via ozawa)
-
- HADOOP-10035. Cleanup TestFilterFileSystem. (Suresh Srinivas via wheat9)
-
- HADOOP-10555. Add offset support to MurmurHash.
- (Sergey Shelukhin via wheat9)
-
- HADOOP-10465. Fix use of generics within SortedMapWritable.
- (Bertrand Dechoux via wheat9)
-
- HADOOP-12600. FileContext and AbstractFileSystem should be annotated as a
- Stable interface. (cnauroth)
-
- HADOOP-12618. Fix NPE in TestSequenceFile. (Brahma Reddy Battula via umamahesh)
-
- HADOOP-12639. Improve JavaDoc for getTrimmedStrings. (BELUGA BEHR via umamahesh)
-
- HADOOP-12570. HDFS Secure Mode Documentation updates.
- (Arpit Agarwal via cnauroth)
-
- HADOOP-12686. Update FileSystemShell documentation to mention the meaning
- of each columns of fs -du. (Daisuke Kobayashi via aajisaka)
-
- HADOOP-12658. Clear javadoc and check style issues around DomainSocket
- (Kai Zheng via umamahesh)
-
- HADOOP-12604. Exception may be swallowed in KMSClientProvider.
- (Yongjun Zhang)
-
- HADOOP-12696. Add tests for S3Filesystem Contract (Matt Paduano via raviprak)
-
- HADOOP-12755. Fix typo in defaultFS warning message. (wang)
-
- HADOOP-12292. Make use of DeleteObjects optional.
- (Thomas Demoor via stevel)
-
- HADOOP-12759. RollingFileSystemSink should eagerly rotate directories.
- (Daniel Templeton via wang)
-
- HADOOP-12752. Improve diagnostics/use of envvar/sysprop credential
- propagation (Steve Loughran via cnauroth)
-
- HADOOP-12764. Increase default value of KMX maxHttpHeaderSize and make it
- configurable. (zhz)
-
- HADOOP-12788. OpensslAesCtrCryptoCodec should log which random number generator is used.
- (Wei-Chiu Chuang via umamahesh)
-
- HADOOP-12699. TestKMS#testKMSProvider intermittently fails during
- 'test rollover draining'. (Xiao Chen via wang)
-
- HADOOP-12776. Remove getaclstatus call for non-acl commands in getfacl.
- (Brahma Reddy Battula via vinayakumarb)
-
- HADOOP-11031. Design Document for Credential Provider API.
- (Larry McCay via cnauroth)
-
- HADOOP-12828. Print user when services are started.
- (Wei-Chiu Chuang via Yongjun Zhang)
-
- HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure.
- (Masatake Iwasaki via cnauroth)
-
- HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through
- ssl-server.conf. (Vijay Singh via zhz)
-
- HADOOP-12555. WASB to read credentials from a credential provider.
- (Larry McCay via cnauroth)
-
- HADOOP-12535. Run FileSystem contract tests with hadoop-azure.
- (Madhumita Chakraborty via cnauroth)
-
- HADOOP-12824. Collect network and disk usage on the node running Windows.
- (Inigo Goiri via xyao)
-
- HADOOP-12825. Log slow name resolutions.
- (Sidharta Seethana via stevel)
-
- HADOOP-12622. Improve the loggings in RetryPolicies and RetryInvocationHandler.
- (Junping Du via jianhe)
-
- HADOOP-12793. Write a new group mapping service guide
- (Wei-Chiu Chuang via iwasakims)
-
- OPTIMIZATIONS
-
- HADOOP-11785. Reduce the number of listStatus operation in distcp
- buildListing (Zoran Dimitrijevic via Colin P. McCabe)
-
- HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7
- ThreadLocalRandom. (Sean Busbey via Colin P. McCabe)
-
- HADOOP-11772. RPC Invoker relies on static ClientCache which has
- synchronized(this) blocks. (wheat9)
-
- HADOOP-11242. Record the time of calling in tracing span of
- IPC server. (Masatake Iwasaki via aajisaka)
-
- HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash.
- (wang)
-
- HADOOP-12104. Migrate Hadoop Pipes native build to new CMake framework
- (alanburlison via cmccabe)
-
- HADOOP-12036. Consolidate all of the cmake extensions in one directory
- (alanburlison via cmccabe)
-
- HADOOP-12112. Make hadoop-common-project Native code -Wall-clean
- (alanburlison via cmccabe)
-
- HADOOP-12172. FsShell mkdir -p makes an unnecessary check for the existence
- of the parent. (cnauroth)
-
- HADOOP-12194. Support for incremental generation in the protoc plugin.
- (wang)
-
- HADOOP-11878. FileContext#fixRelativePart should check for not null for a
- more informative exception. (Brahma Reddy Battula via kasha)
-
- HADOOP-11984. Enable parallel JUnit tests in pre-commit.
- (Chris Nauroth via vinayakumarb)
-
- HADOOP-12450. UserGroupInformation should not log at WARN level if no groups
- are found. (Elliott Clark via stevel)
-
- HADOOP-12758. Extend CSRF Filter with UserAgent Checks
- (Larry McCay via cnauroth)
-
- HADOOP-10865. Add a Crc32 chunked verification benchmark for both directly
- and non-directly buffer cases. (szetszwo)
-
- HADOOP-12853. Change WASB documentation regarding page blob support.
- (Madhumita Chakraborty via cnauroth)
-
- BUG FIXES
-
- HADOOP-12352. Delay in checkpointing Trash can leave trash for 2 intervals
- before deleting (Casey Brotherton via harsh)
-
- HADOOP-11568. Description on usage of classpath in hadoop command is
- incomplete. ( Archana T via vinayakumarb )
-
- HADOOP-10027. *Compressor_deflateBytesDirect passes instance instead of
- jclass to GetStaticObjectField. (Hui Zheng via cnauroth)
-
- HADOOP-11724. DistCp throws NPE when the target directory is root.
- (Lei Eddy Xu via Yongjun Zhang)
-
- HADOOP-11760. Fix typo of javadoc in DistCp. (Brahma Reddy Battula via
- ozawa).
-
- HADOOP-11800. Clean up some test methods in TestCodec.java.
- (Brahma Reddy Battula via aajisaka)
-
- HADOOP-11811. Fix typos in hadoop-project/pom.xml and TestAccessControlList.
- (Brahma Reddy Battula via ozawa)
-
- HADOOP-11704. DelegationTokenAuthenticationFilter must pass ipaddress
- instead of hostname to ProxyUsers#authorize (Anubhav Dhoot via asuresh)
-
- HADOOP-11846 TestCertificateUtil.testCorruptPEM failing on Jenkins JDK8.
- (Larry McCay via stevel)
-
- HADOOP-11859. PseudoAuthenticationHandler fails with httpcomponents v4.4.
- (Eugene Koifman via jitendra)
-
- HADOOP-11848. Incorrect arguments to sizeof in DomainSocket.c
- (Malcolm Kavalsky via Colin P. McCabe)
-
- HADOOP-11861. test-patch.sh rewrite addendum patch.
- (Allen Wittenauer via cnauroth)
-
- HADOOP-11864. JWTRedirectAuthenticationHandler breaks java8 javadocs.
- (Larry McCay via stevel)
-
- HADOOP-11876. Refactor code to make it more readable, minor
- maybePrintStats bug (Zoran Dimitrijevic via raviprak)
-
- HADOOP-11870. [JDK8] AuthenticationFilter, CertificateUtil,
- SignerSecretProviders, KeyAuthorizationKeyProvider Javadoc issues (rkanter)
-
- HADOOP-11881. test-patch.sh javac result is wildly wrong (Kengo Seki via
- aw)
-
- HADOOP-11821. Fix findbugs warnings in hadoop-sls.
- (Brahma Reddy Battula via aajisaka)
-
- HADOOP-11866. increase readability and reliability of checkstyle,
- shellcheck, and whitespace reports (aw)
-
- HADOOP-11889. Make checkstyle runnable from root project
- (Gera Shegalov via jeagles)
-
- HADOOP-11900. Add failIfNoTests=false to hadoop-build-tools pom. (gera)
-
- HADOOP-11916. TestStringUtils#testLowerAndUpperStrings failed on MAC
- due to a JVM bug. (Ming Ma via ozawa)
-
- HADOOP-11904. test-patch.sh goes into an infinite loop on non-maven
- builds (aw)
-
- HADOOP-11926. test-patch.sh mv does wrong math (aw)
-
- HADOOP-11912. Extra configuration key used in TraceUtils should respect
- prefix (Masatake Iwasaki via Colin P. McCabe)
-
- HADOOP-10387. Misspelling of threshold in log4j.properties for tests in
- hadoop-common-project. (Brahma Reddy Battula via aajisaka)
-
- HADOOP-11936. Dockerfile references a removed image (aw)
-
- HADOOP-11922. Misspelling of threshold in log4j.properties for tests
- in hadoop-tools (Gabor Liptak via vinayakumarb)
-
- HADOOP-11877. SnappyDecompressor's Logger class name is wrong
- (surendra singh lilhore via vinayakumarb)
-
- HADOOP-10356. Corrections in winutils/chmod.c. (Rene Nyffenegger via
- Arpit Agarwal)
-
- HADOOP-7165. listLocatedStatus(path, filter) is not redefined in FilterFs
- (Hairong Kuang via jlowe)
-
- HADOOP-9729. The example code of org.apache.hadoop.util.Tool is incorrect
- (hellojinjie via jlowe)
-
- HADOOP-11928. Test-patch check for @author tags incorrectly flags
- removal of @author tags (Kengo Seki via aw)
-
- HADOOP-11951. test-patch should give better info about failures to handle
- dev-support updates without resetrepo option (Sean Busbey via aw)
-
- HADOOP-11947. test-patch should return early from determine-issue when
- run in jenkins mode. (Sean Busbey via aw)
-
- HADOOP-11962. Sasl message with MD5 challenge text shouldn't be LOG out
- even in debug level. (Junping Du via wheat9)
-
- HADOOP-8174. Remove confusing comment in Path#isAbsolute()
- (Suresh Srinivas via vinayakumarb)
-
- HADOOP-11988. Fix typo in the document for hadoop fs -find.
- (Kengo Seki via aajisaka)
-
- HADOOP-10582. Fix the test case for copying to non-existent dir in
- TestFsShellCopy. (Kousuke Saruta via aajisaka)
-
- HADOOP-11581. Multithreaded correctness Warnings
- #org.apache.hadoop.fs.shell.Ls (Brahma Reddy Battula via aw)
-
- HADOOP-11963. Metrics documentation for FSNamesystem misspells
- PendingDataNodeMessageCount. (Anu Engineer via cnauroth)
-
- HADOOP-12000. cannot use --java-home in test-patch (aw)
-
- HADOOP-12014. hadoop-config.cmd displays a wrong error message.
- (Kengo Seki via aajisaka)
-
- HADOOP-11955. Fix a typo in the cluster setup doc.
- (Yanjun Wang via aajisaka)
-
- HADOOP-11743. maven doesn't clean all the site files
- (ramtin via vinayakumarb)
-
- HADOOP-11927. Fix "undefined reference to dlopen" error when compiling
- libhadooppipes (Xianyin Xin via Colin P. McCabe)
-
- HADOOP-8751. NPE in Token.toString() when Token is constructed using null
- identifier. (kanaka kumar avvaru via aajisaka)
-
- HADOOP-12004. test-patch breaks with reexec in certain situations (Sean
- Busbey via aw)
-
- HADOOP-12035. shellcheck plugin displays a wrong version potentially
- (Kengo Seki via aw)
-
- HDFS-8429. Avoid stuck threads if there is an error in DomainSocketWatcher
- that stops the thread. (zhouyingchao via cmccabe)
-
- HADOOP-11930. test-patch in offline mode should tell maven to be in
- offline mode (Sean Busbey via aw)
-
- HADOOP-11959. WASB should configure client side socket timeout in storage
- client blob request options. (Ivan Mitic via cnauroth)
-
- HADOOP-12042. Users may see TrashPolicy if hdfs dfs -rm is run
- (Andreina J via vinayakumarb)
-
- HADOOP-11991. test-patch.sh isn't re-executed even if smart-apply-patch.sh
- is modified. (Kengo Seki via aajisaka)
-
- HADOOP-12018. smart-apply-patch.sh fails if the patch edits CR+LF files
- and is created by 'git diff --no-prefix'. (Kengo Seki via aajisaka)
-
- HADOOP-12019. update BUILDING.txt to include python for 'mvn site'
- in windows (vinayakumarb)
-
- HADOOP-11994. smart-apply-patch wrongly assumes that git is infallible.
- (Kengo Seki via Arpit Agarwal)
-
- HADOOP-11924. Tolerate JDK-8047340-related exceptions in
- Shell#isSetSidAvailable preventing class init. (Tsuyoshi Ozawa via gera)
-
- HADOOP-12052 IPC client downgrades all exception types to IOE, breaks
- callers trying to use them. (Brahma Reddy Battula via stevel)
-
- HADOOP-12054. RPC client should not retry for InvalidToken exceptions.
- (Varun Saxena via Arpit Agarwal)
-
- HADOOP-12073. Azure FileSystem PageBlobInputStream does not return -1 on
- EOF. (Ivan Mitic via cnauroth)
-
- HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream
- with accurate .getPos() (kanaka kumar avvaru via vinayakumarb)
-
- HADOOP-12074. in Shell.java#runCommand() rethrow InterruptedException as
- InterruptedIOException (Lavkesh Lahngir via vinayakumarb)
-
- HADOOP-12001. Fixed LdapGroupsMapping to include configurable Posix UID and
- GID attributes during the search. (Patrick White via vinodkv)
-
- HADOOP-12095. org.apache.hadoop.fs.shell.TestCount fails.
- (Brahma Reddy Battula via aajisaka)
-
- HADOOP-12076. Incomplete Cache Mechanism in CredentialProvider API.
- (Larry McCay via cnauroth)
-
- HADOOP-12119. hadoop fs -expunge does not work for federated namespace
- (J.Andreina via vinayakumarb)
-
- HADOOP-12089. StorageException complaining " no lease ID" when updating
- FolderLastModifiedTime in WASB. (Duo Xu via cnauroth)
-
- HADOOP-12154. FileSystem#getUsed() returns the file length only from root '/'
- (J.Andreina via vinayakumarb)
-
- HADOOP-10798. globStatus() should always return a sorted list of files
- (cmccabe)
-
- HADOOP-12159. Move DistCpUtils#compareFs() to org.apache.hadoop.fs.FileUtil
- and fix for HA namespaces (rchiang via rkanter)
-
- HADOOP-12116. Fix unrecommended syntax usages in hadoop/hdfs/yarn script for
- cygwin in branch-2. (Li Lu via cnauroth)
-
- HADOOP-12164. Fix TestMove and TestFsShellReturnCode failed to get command
- name using reflection. (Lei (Eddy) Xu)
-
- HADOOP-12173. NetworkTopology::add calls toString always.
- (Inigo Goiri via cdouglas)
-
- HADOOP-12185. NetworkTopology is not efficient adding/getting/removing
- nodes. (Inigo Goiri via cdouglas)
-
- HADOOP-12117. Potential NPE from Configuration#loadProperty with
- allowNullValueProperties set. (zhihai xu via vinayakumarb)
-
- HADOOP-12200. TestCryptoStreamsWithOpensslAesCtrCryptoCodec should be
- skipped in non-native profile. (Masatake Iwasaki via aajisaka)
-
- HADOOP-10615. FileInputStream in JenkinsHash#main() is never closed.
- (Chen He via ozawa)
-
- HADOOP-12240. Fix tests requiring native library to be skipped in non-native
- profile. (Masatake Iwasaki via ozawa)
-
- HADOOP-12235 hadoop-openstack junit & mockito dependencies should be
- "provided". (Ted Yu via stevel)
-
- HADOOP-12209 Comparable type should be in FileStatus.
- (Yong Zhang via stevel)
-
- HADOOP-12088. KMSClientProvider uses equalsIgnoreCase("application/json").
- (Brahma Reddy Battula via stevel)
-
- HADOOP-11797. releasedocmaker.py needs to put ASF headers on output (aw)
-
- HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal
- tags. (Akira AJISAKA via stevel).
-
- HADOOP-12386. RetryPolicies.RETRY_FOREVER should be able to specify a
- retry interval. (Sunil G via wangda)
-
- HADOOP-8436. NPE In getLocalPathForWrite ( path, conf ) when the
- required context item is not configured
- (Brahma Reddy Battula via harsh)
-
- HADOOP-12252. LocalDirAllocator should not throw NPE with empty string
- configuration. (Zhihai Xu)
-
- HADOOP-11918. Listing an empty s3a root directory throws FileNotFound.
- (Lei (Eddy) Xu via cnauroth)
-
- HADOOP-12440. TestRPC#testRPCServerShutdown did not produce the desired
- thread states before shutting down. (Xiao Chen via mingma)
-
- HADOOP-12447. Clean up some htrace integration issues (cmccabe)
-
- HADOOP-12448. TestTextCommand: use mkdirs rather than mkdir to create test
- directory. (Contributed by Colin Patrick McCabe and Chris Nauroth)
-
- HADOOP-10296. Incorrect null check in SwiftRestClient#buildException().
- (Rahul Palamuttam and Kanaka Kumar Avvaru via aajisaka)
-
- HADOOP-8437. getLocalPathForWrite should throw IOException for invalid
- paths. (Brahma Reddy Battula via zxu)
-
- HADOOP-11098. [JDK8] Max Non Heap Memory default changed between JDK7
- and 8. (ozawa)
-
- HADOOP-12284. UserGroupInformation doAs can throw misleading exception
- (Aaron Dosset via stevel)
-
- HADOOP-12449. TestDNS and TestNetUtils failing if no network. (stevel)
-
- HADOOP-11515. Upgrade jsch lib to jsch-0.1.51 to avoid problems running
- on java7. (stevel and ozawa)
-
- HADOOP-12478. Shell.getWinUtilsPath() has been renamed
- Shell.getWinutilsPath(). (stevel)
-
- HADOOP-12475. Replace guava Cache with ConcurrentHashMap for caching
- Connection in ipc Client (Walter Su via sjlee)
-
- HADOOP-12479. ProtocMojo does not log the reason for a protoc compilation
- failure. (cnauroth)
-
- HADOOP-11628. SPNEGO auth does not work with CNAMEs in JDK8.
- (Daryn Sharp via stevel).
-
- HADOOP-10941. Proxy user verification NPEs if remote host is unresolvable.
- (Benoy Antony via stevel).
-
- HADOOP-12483. Maintain wrapped SASL ordering for postponed IPC responses.
- (Daryn Sharp via yliu)
-
- HADOOP-12513. Dockerfile lacks initial 'apt-get update'.
- (Akihiro Suda via ozawa)
-
- HADOOP-12457. [JDK8] Fix a failure of compiling common by javadoc.
- (Akira AJISAKA via ozawa)
-
- HADOOP-12178. NPE during handling of SASL setup if problem with SASL
- resolver class. (Steve Loughran via zxu)
-
- HADOOP-12519. hadoop-azure tests should avoid creating a metrics
- configuration file in the module root directory. (cnauroth)
-
- HADOOP-12533. Introduce FileNotFoundException in WASB for read and seek API.
- (Dushyanth via cnauroth)
-
- HADOOP-12508. delete fails with exception when lease is held on blob.
- (Gaurav Kanade via cnauroth)
-
- HADOOP-12542. TestDNS fails on Windows after HADOOP-12437. (cnauroth)
-
- HADOOP-12540. TestAzureFileSystemInstrumentation#testClientErrorMetrics
- fails intermittently due to assumption that a lease error will be thrown.
- (Gaurav Kanade via cnauroth)
-
- HADOOP-12560: Fix sprintf warnings in {{DomainSocket.c}} introduced by
- HADOOP-12344 (Mingliang Liu via Colin P. McCabe)
-
- HADOOP-12567: NPE in SaslRpcServer (Sergey Shelukhin via cnauroth)
-
- HADOOP-8419. Fixed GzipCode NPE reset for IBM JDK. (Yu Li via eyang)
-
- HADOOP-11149. TestZKFailoverController times out. (Steve Loughran
- via ozawa)
-
- HADOOP-9822. Create constant MAX_CAPACITY in RetryCache rather than
- hard-coding 16 in RetryCache constructor. (Tsuyoshi Ozawa via wheat9)
-
- HADOOP-11954. Solaris does not support RLIMIT_MEMLOCK as in Linux
- (Alan Burlison via aw)
-
- HADOOP-12598. Add XML namespace declarations for some hadoop/tools modules.
- (Xin Wang via aajisaka)
-
- HADOOP-12468. Partial group resolution failure should not result in user
- lockout. (Wei-Chiu Chuang via Yongjun Zhang)
-
- HADOOP-12656. MiniKdc throws "address in use" BindException.
- (Wei-Chiu Chuang via Arpit Agarwal)
-
- HADOOP-12051. ProtobufRpcEngine.invoke() should use Exception.toString()
- over getMessage() in logging/span events. (Varun Saxena via stevel)
-
- HADOOP-12017. Hadoop archives command should use configurable replication
- factor when closing (Bibin A Chundatt via vinayakumarb)
-
- HADOOP-12239. StorageException complaining " no lease ID" when updating
- FolderLastModifiedTime in WASB. (Duo Xu via cnauroth)
-
- HADOOP-12245. References to misspelled REMAINING_QUATA in
- FileSystemShell.md. (Gabor Liptak via aajisaka)
-
- HADOOP-12175. FsShell must load SpanReceierHost to support tracing
- (Masatake Iwasaki via Colin P. McCabe)
-
- HADOOP-10945. 4-digit octal umask permissions throws a parse error (Chang
- Li via jlowe)
-
- HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for
- each platform, not hardcoded to their Linux values (Martin Walsh via Colin
- P. McCabe)
-
- HADOOP-12268. AbstractContractAppendTest#testRenameFileBeingAppended
- misses rename operation. (Zhihai Xu)
-
- HADOOP-12274. Remove direct download link from BULIDING.txt.
- (Caleb Severn via aajisaka)
-
- HADOOP-12302. Fix native compilation on Windows after HADOOP-7824
- (Vinayakumar B via Colin P. McCabe)
-
- HADOOP-12258. Need translate java.nio.file.NoSuchFileException to
- FileNotFoundException to avoid regression. (Zhihai Xu via cnauroth)
-
- HADOOP-12322. typos in rpcmetrics.java. (Anu Engineer via
- Arpit Agarwal)
-
- HADOOP-12317. Applications fail on NM restart on some linux distro
- because NM container recovery declares AM container as LOST
- (adhoot via rkanter)
-
- HADOOP-12347. Fix mismatch parameter name in javadocs of
- AuthToken#setMaxInactives (xyao)
-
- HADOOP-12362. Set hadoop.tmp.dir and hadoop.log.dir in pom.
- (Charlie Helin via wang)
-
- HADOOP-10318. Incorrect reference to nodeFile in RumenToSLSConverter
- error message. (Wei Yan via ozawa)
-
- HADOOP-12388. Fix components' version information in the web page
- 'About the Cluster'. (Jun Gong via zxu)
-
- HADOOP-12407. Test failing: hadoop.ipc.TestSaslRPC. (stevel)
-
- HADOOP-12417. TestWebDelegationToken failing with port in use.
- (Mingliang Liu via wheat9)
-
- HADOOP-12438. Reset RawLocalFileSystem.useDeprecatedFileStatus in
- TestLocalFileSystem. (Chris Nauroth via wheat9)
-
- HADOOP-12437. Allow SecurityUtil to lookup alternate hostnames.
- (Arpit Agarwal)
-
- HADOOP-12442. Display help if the command option to 'hdfs dfs' is not valid
- (nijel via vinayakumarb)
-
- HADOOP-12441. Fixed shell-kill command behaviour to work correctly on some
- Linux distributions after HADOOP-12317. (Wangda Tan via vinodkv)
-
- HADOOP-12474. MiniKMS should use random ports for Jetty server by default.
- (Mingliang Liu via wheat9)
-
- HADOOP-12418. TestRPC.testRPCInterruptedSimple fails intermittently.
- (kihwal)
-
- HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass
- Azure Storage Throttling after retries. (Gaurav Kanade via cnauroth)
-
- HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race
- Condition Scenarios. (Gaurav Kanade via cnauroth)
-
- HADOOP-11901. BytesWritable fails to support 2G chunks due to integer
- overflow. (Reynold Xin via wheat9)
-
- HADOOP-11218. Add TLSv1.1,TLSv1.2 to KMS, HttpFS, SSLFactory.
- (Vijay Singh via wheat9)
-
- HADOOP-12467. Respect user-defined JAVA_LIBRARY_PATH in Windows Hadoop
- scripts. (Radhey Shah via wheat9)
-
- HADOOP-12181. Fix intermittent test failure of TestZKSignerSecretProvider.
- (Masatake Iwasaki via wheat9)
-
- HADOOP-12098. Remove redundant test dependencies in Hadoop Archives.
- (Varun Saxena via wheat9)
-
- HADOOP-10068. Improve log4j regex in testFindContainingJar.
- (Robert Rati via wheat9)
-
- HADOOP-11677. Add cookie flags for logs and static contexts.
- (nijel via wheat9)
-
- HADOOP-12313. NPE in JvmPauseMonitor when calling stop() before start().
- (Gabor Liptak via wheat9)
-
- HADOOP-12609. Fix intermittent failure of TestDecayRpcScheduler.
- (Masatake Iwasaki via Arpit Agarwal)
-
- HADOOP-10729. Add tests for PB RPC in case version mismatch of client and
- server. (Junping Du via wheat9)
-
- HADOOP-12636. Prevent ServiceLoader failure init for unused FileSystems.
- (Inigo Goiri via cdouglas)
-
- HADOOP-12681. start-build-env.sh fails in branch-2.
- (Kengo Seki via aajisaka)
-
- HADOOP-12559. KMS connection failures should trigger TGT renewal.
- (Zhe Zhang via xyao)
-
- HADOOP-12682. Fix TestKMS#testKMSRestart* failure.
- (Wei-Chiu Chuang via xyao)
-
- HADOOP-12689. S3 filesystem operations stopped working correctly
- (Matt Paduano via raviprak)
-
- HADOOP-12634. Change Lazy Rename Pending Operation Completion of WASB to
- address case of potential data loss due to partial copy
- (Gaurav Kanade via cnauroth)
-
- HADOOP-12675. Fix description about retention period in usage of expunge
- command. (Masatake Iwasaki via stevel)
-
- HADOOP-12678. Handle empty rename pending metadata file during atomic rename
- in redo path. (Madhumita Chakraborty via cnauroth)
-
- HADOOP-12551. Introduce FileNotFoundException for WASB FileSystem API
- (Dushyanth via cnauroth)
-
- HADOOP-12584. Disable browsing the static directory in HttpServer2.
- (Robert Kanter via aajisaka)
-
- HADOOP-12603. TestSymlinkLocalFSFileContext#testSetTimesSymlinkToDir
- occasionally fail. (Wei-Chiu Chuang via cnauroth)
-
- HADOOP-12590. TestCompressorDecompressor failing without stack traces
- (John Zhuge via stevel)
-
- HADOOP-12700. Remove unused import in TestCompressorDecompressor.java.
- (John Zhuge via aajisaka)
-
- HADOOP-12356. Fix computing CPU usage statistics on Windows.
- (Inigo Goiri via wangda)
-
- HADOOP-12423. Handle failure of registering shutdownhook by
- ShutdownHookManager in static block (Abhishek Agarwal via vinayakumarb)
-
- HADOOP-7161. Remove unnecessary oro package from dependency management
- section. (Sean Busbey via aajisaka)
-
- HADOOP-12659. Incorrect usage of config parameters in token manager of
- KMS. (Mingliang Liu via xyao)
-
- HADOOP-12730. Hadoop streaming -mapper and -reducer options are wrongly
- documented as required. (Kengo Seki via aajisaka)
-
- HADOOP-12731. Remove useless boxing/unboxing code.
- (Kousuke Saruta via aajisaka)
-
- HADOOP-12718. Incorrect error message by fs -put local dir without
- permission. (John Zhuge via Yongjun Zhang)
-
- HADOOP-12735. core-default.xml misspells
- hadoop.workaround.non.threadsafe.getpwuid (Ray Chiang via cmccabe)
-
- HADOOP-12771. Fix typo in JvmPauseMonitor#getNumGcWarnThreadholdExceeded.
- (Xiaobing Zhou via xyao)
-
- HADOOP-12766. The default value of
- "hadoop.workaround.non.threadsafe.getpwuid" is different between
- core-default.xml and NativeIO.java (Akira AJISAKA via cmccabe)
-
- HADOOP-12795. KMS does not log detailed stack trace for unexpected errors.
- (cnauroth)
-
- HADOOP-12780. During atomic rename handle crash when one directory has been
- renamed but not file under it. (Madhumita Chakraborty via cnauroth)
-
- HADOOP-12878. KMS SPNEGO sequence does not work with WEBHDFS. (xyao)
-
- HADOOP-12716. KerberosAuthenticator#doSpnegoSequence use incorrect class to
- determine isKeyTab in JDK8. (Xiaoyu Yao via cnauroth)
-
- HADOOP-12849. TestSymlinkLocalFSFileSystem fails intermittently.
- (Mingliang Liu via cnauroth)
-
- HADOOP-12846. Credential Provider Recursive Dependencies.
- (Larry McCay via cnauroth)
-
- HADOOP-12813. Migrate TestRPC and related codes to rebase on
- ProtobufRpcEngine. (Kai Zheng via wheat9)
-
- HADOOP-12851. S3AFileSystem Uptake of
- ProviderUtils.excludeIncompatibleCredentialProviders.
- (Larry McCay via cnauroth)
-
- HADOOP-12843. Fix findbugs warnings in hadoop-common (branch-2). (aajisaka)
-
-Release 2.7.3 - UNRELEASED
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- HADOOP-12413. AccessControlList should avoid calling getGroupNames in
- isUserInList with empty groups. (Zhihai Xu via cnauroth)
-
- HADOOP-12794. Support additional compression levels for GzipCodec.
- (Ravi Mutyala via junping_du)
-
- OPTIMIZATIONS
-
- HADOOP-12810. FileSystem#listLocatedStatus causes unnecessary RPC calls
- (Ryan Blue via vinayakumarb)
-
- BUG FIXES
-
- HADOOP-12296. when setnetgrent returns 0 in linux, exception should be
- thrown (Chang Li via jlowe)
-
- HADOOP-12545. Hadoop javadoc has broken links for AccessControlList,
- ImpersonationProvider, DefaultImpersonationProvider, and DistCp.
- (Arshad Mohammad via aajisaka)
-
- HADOOP-12374. Updated expunge command description.
- (WeiWei Yang via eyang)
-
- HADOOP-12348. MetricsSystemImpl creates MetricsSourceAdapter with wrong
- time unit parameter. (zxu via rkanter)
-
- HADOOP-12482. Race condition in JMX cache update. (Tony Wu via lei)
-
- HADOOP-12565. Replace DSA with RSA for SSH key type in SingleCluster.md.
- (Mingliang Liu via aajisaka)
-
- HADOOP-12602. TestMetricsSystemImpl#testQSize occasionally fails.
- (Masatake Iwasaki via aajisaka)
-
- HADOOP-12588. Fix intermittent test failure of TestGangliaMetrics.
- (Masatake Iwasaki via aajisaka)
-
- HADOOP-12107. long running apps may have a huge number of StatisticsData
- instances under FileSystem (Sangjin Lee via Ming Ma)
-
- HADOOP-12706. TestLocalFsFCStatistics#testStatisticsThreadLocalDataCleanUp
- times out occasionally (Sangjin Lee and Colin Patrick McCabe via jlowe)
-
- HADOOP-12761. incremental maven build is not really incremental (sjlee)
-
- HADOOP-12772. NetworkTopologyWithNodeGroup.getNodeGroup() can loop
- infinitely for invalid 'loc' values (Kuhu Shukla via kihwal)
-
- HADOOP-12792. TestUserGroupInformation#testGetServerSideGroups fails in
- chroot (Eric Badger via jlowe)
-
- HADOOP-8818. Use equals instead == in MD5MD5CRC32FileChecksum
- and TFileDumper. (Brandon Li via suresh)
-
- HADOOP-9121. InodeTree.java has redundant check for vName while
- throwing exception. (Arup Malakar via suresh)
-
- HADOOP-12870. Fix typo admininistration in CommandsManual.md.
- (John Zhuge via aajisaka)
-
- HADOOP-12871. Fix dead link to NativeLibraries.html in CommandsManual.md.
- (Brahma Reddy Battula via aajisaka)
-
- HADOOP-12872. Fix formatting in ServiceLevelAuth.md.
- (Brahma Reddy Battula via aajisaka)
-
-Release 2.7.2 - 2016-01-25
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- HADOOP-12232. Upgrade Tomcat dependency to 6.0.44. (cnauroth)
-
- HADOOP-5323. Trash documentation should describe its directory structure and
- configurations. (Weiwei Yang via ozawa)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- HADOOP-12186. ActiveStandbyElector shouldn't call monitorLockNodeAsync
- multiple times (zhihai xu via vinayakumarb)
-
- HADOOP-12191. Bzip2Factory is not thread safe. (Brahma Reddy Battula
- via ozawa)
-
- HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile.
- (kanaka kumar avvaru via wheat9)
-
- HADOOP-12304. Applications using FileContext fail with the default file
- system configured to be wasb/s3/etc. (cnauroth)
-
- HADOOP-12061. Incorrect command in single cluster setup document.
- (Kengo Seki via aajisaka)
-
- HADOOP-12359. hadoop fs -getmerge doc is wrong.
- (Jagadesh Kiran N via aajisaka)
-
- HADOOP-10365. BufferedOutputStream in FileUtil#unpackEntries() should be
- closed in finally block. (Kiran Kumar M R and Sanghyun Yun via ozawa)
-
- HADOOP-12213. Interrupted exception can occur when Client#stop is called.
- (Kuhu Shukla via ozawa)
-
- HADOOP-12465. Incorrect javadoc in WritableUtils.java.
- (Jagadesh Kiran N via aajisaka)
-
- HADOOP-12464. Interrupted client may try to fail-over and retry (kihwal)
-
- HADOOP-12415. Fixed pom files to correctly include compile-time dependency on
- netty. (Tom Zeng via cos)
-
- HADOOP-12786. "hadoop key" command usage is not documented.
- (Xiao Chen via aajisaka)
-
-Release 2.7.1 - 2015-07-06
-
- INCOMPATIBLE CHANGES
-
- NEW FEATURES
-
- IMPROVEMENTS
-
- HADOOP-12103. Small refactoring of DelegationTokenAuthenticationFilter to
- allow code sharing. (Yongjun Zhang)
-
- OPTIMIZATIONS
-
- BUG FIXES
-
- HADOOP-11868. Invalid user logins trigger large backtraces in server log
- (Chang Li via jlowe)
-
- HADOOP-11872. "hadoop dfs" command prints message about using "yarn jar" on
- Windows(branch-2 only) (Varun Vasudev via cnauroth)
-
- HADOOP-11891. OsSecureRandom should lazily fill its reservoir (asuresh)
-
- HADOOP-9658. SnappyCodec#checkNativeCodeLoaded may unexpectedly fail when
- native code is not loaded. (Zhijie Shen via ozawa)
-
- HADOOP-11663. Remove description about Java 6 from docs.
- (Masatake Iwasaki via aajisaka)
-
- HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed
- through hadoop-daemon.sh. (cnauroth)
-
- HADOOP-11973. Ensure ZkDelegationTokenSecretManager namespace znodes get
- created with ACLs. (Gregory Chanan via asuresh)
-
- HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages.
- (Kazuho Fujii via aajisaka)
-
- HADOOP-12078. The default retry policy does not handle RetriableException
- correctly. (Arpit Agarwal)
-
- HADOOP-12100. ImmutableFsPermission should not override applyUmask since
- that method doesn't modify the FsPermission (Bibin A Chundatt via Colin P.
- McCabe)
-
-Release 2.7.0 - 2015-04-20
-
- INCOMPATIBLE CHANGES
-
- HADOOP-10530 Make hadoop build on Java7+ only (stevel)
-
- NEW FEATURES
-
- HADOOP-10987. Provide an iterator-based listing API for FileSystem (kihwal)
-
- HADOOP-7984. Add hadoop --loglevel option to change log level.
- (Akira AJISAKA via cnauroth)
-
- HADOOP-9629. Support Windows Azure Storage - Blob as a file system in Hadoop.
- (Dexter Bradshaw, Mostafa Elhemali, Xi Fang, Johannes Klein, David Lao,
- Mike Liddell, Chuan Liu, Lengning Liu, Ivan Mitic, Michael Rys,
- Alexander Stojanovic, Brian Swan, and Min Wei via cnauroth)
-
- HADOOP-10728. Metrics system for Windows Azure Storage Filesystem.
- (Dexter Bradshaw, Mostafa Elhemali, Xi Fang, Johannes Klein, David Lao,
- Mike Liddell, Chuan Liu, Lengning Liu, Ivan Mitic, Michael Rys,
- Alexander Stojanovich, Brian Swan, and Min Wei via cnauroth)
-
- HADOOP-8989. hadoop fs -find feature (Jonathan Allen via aw)
-
- HADOOP-11490. Expose truncate API via FileSystem and shell command.
- (Milan Desai via shv)
-
- HADOOP-11045. Introducing a tool to detect flaky tests of hadoop jenkins testing
- job. (Yongjun Zhang and Todd Lipcon via ozawa)
-
- HADOOP-11510. Expose truncate API via FileContext. (yliu)
-
- HADOOP-11569. Provide Merge API for MapFile to merge multiple similar MapFiles
- to one MapFile. (Vinayakumar B via ozawa)
-
- IMPROVEMENTS
-
- HADOOP-11483. HardLink.java should use the jdk7 createLink method (aajisaka)
-
- HADOOP-11156. DelegateToFileSystem should implement
- getFsStatus(final Path f). (Zhihai Xu via wang)
-
- HADOOP-11172. Improve error message in Shell#runCommand on OutOfMemoryError.
- (Yongjun Zhang via wang)
-
- HADOOP-10748. HttpServer2 should not load JspServlet. (wheat9)
-
- HADOOP-10847. Remove the usage of sun.security.x509.* in testing code.
- (Pascal Oliva via wheat9)
-
- HADOOP-10563. Remove the dependency of jsp in trunk. (wheat9)
-
- HADOOP-11291. Log the cause of SASL connection failures.
- (Stephen Chu via cnauroth)
-
- HADOOP-11173. Improve error messages for some KeyShell commands. (wang)
-
- HADOOP-11257: Update "hadoop jar" documentation to warn against using it
- for launching yarn jars (iwasakims via cmccabe)
-
- HADOOP-11341. KMS support for whitelist key ACLs. (Arun Suresh via wang)
-
- HADOOP-11301. [optionally] update jmx cache to drop old metrics
- (Maysam Yabandeh via stack)
-
- HADOOP-11313. Adding a document about NativeLibraryChecker.
- (Tsuyoshi OZAWA via cnauroth)
-
- HADOOP-11287. Simplify UGI#reloginFromKeytab for Java 7+.
- (Li Lu via wheat9)
-
- HADOOP-10476) Bumping the findbugs version to 3.0.0. (wheat9)
-
- HADOOP-11410. Make the rpath of libhadoop.so configurable (cmccabe)
-
- HADOOP-11416. Move ChunkedArrayList into hadoop-common (cmccabe)
-
- HADOOP-10840. Fix OutOfMemoryError caused by metrics system in Azure File
- System. (Shanyu Zhao via cnauroth)
-
- HADOOP-11248. Add hadoop configuration to disable Azure Filesystem metrics
- collection. (Shanyu Zhao via cnauroth)
-
- HADOOP-11421. Add IOUtils#listDirectory (cmccabe)
-
- HADOOP-11427. ChunkedArrayList: fix removal via iterator and implement get
- (cmccabe)
-
- HADOOP-11430. Add GenericTestUtils#disableLog, GenericTestUtils#setLogLevel
- (cmccabe)
-
- HADOOP-11422. Check CryptoCodec is AES-CTR for Crypto input/output stream
- (Yi Liu via Colin P. McCabe)
-
- HADOOP-11213. Typos in html pages: SecureMode and EncryptedShuffle.
- (Wei Yan via kasha)
-
- HADOOP-11395. Add site documentation for Azure Storage FileSystem
- integration. (Chris Nauroth via Arpit Agarwal)
-
- HDFS-7555. Remove the support of unmanaged connectors in HttpServer2.
- (wheat9)
-
- HADOOP-11399. Java Configuration file and .xml files should be
- automatically cross-compared (rchiang via rkanter)
-
- HADOOP-11455. KMS and Credential CLI should request confirmation for
- deletion by default. (Charles Lamb via yliu)
-
- HADOOP-11390 Metrics 2 ganglia provider to include hostname in
- unresolved address problems. (Varun Saxena via stevel)
-
- HADOOP-11032. Replace use of Guava's Stopwatch with Hadoop's StopWatch
- (ozawa)
-
- HADOOP-11464. Reinstate support for launching Hadoop processes on Windows
- using Cygwin. (cnauroth)
-
- HADOOP-9992. Modify the NN loadGenerator to optionally run as a MapReduce job
- (Akshay Radia via brandonli)
-
- HADOOP-11465. Fix findbugs warnings in hadoop-gridmix. (Varun Saxena via
- Arpit Agarwal)
-
- HADOOP-11481. ClassCastException while using a key created by keytool to
- create encryption zone. (Charles Lamb via Colin P. Mccabe)
-
- HADOOP-8757. Metrics should disallow names with invalid characters
- (rchiang via rkanter)
-
- HADOOP-11261 Set custom endpoint for S3A. (Thomas Demoor via stevel)
-
- HADOOP-11171 Enable using a proxy server to connect to S3a.
- (Thomas Demoor via stevel)
-
- HADOOP-11489 Dropping dependency on io.netty from hadoop-nfs' pom.xml
- (Ted Yu via ozawa)
-
- HADOOP-11419 Improve hadoop-maven-plugins. (Herve Boutemy via stevel)
-
- HADOOP-11450. Cleanup DistCpV1 not to use deprecated methods and fix
- javadocs. (Varun Saxena via ozawa)
-
- HADOOP-4297. Enable Java assertions when running tests.
- (Tsz Wo Nicholas Sze via wheat9)
-
- HADOOP-10626. Limit Returning Attributes for LDAP search. (Jason Hubbard
- via atm)
-
- HADOOP-11317. Increment SLF4J version to 1.7.10. (Tim Robertson via ozawa)
-
- HADOOP-10525. Remove DRFA.MaxBackupIndex config from log4j.properties
- (aajisaka)
-
- HADOOP-10574. Bump the maven plugin versions too -moving the numbers into
- properties. (aajisaka)
-
- HADOOP-11441. Hadoop-azure: Change few methods scope to public.
- (Shashank Khandelwal via cnauroth)
-
- HADOOP-9137. Support connection limiting in IPC server (kihwal)
-
- HADOOP-11498. Bump the version of HTrace to 3.1.0-incubating (Masatake
- Iwasaki via Colin P. McCabe)
-
- HADOOP-11442. hadoop-azure: Create test jar.
- (Shashank Khandelwal via cnauroth)
-
- HADOOP-11544. Remove unused configuration keys for tracing. (Masatake
- Iwasaki via aajisaka)
-
- HADOOP-11492. Bump up curator version to 2.7.1. (Arun Suresh and
- Karthik Kambatla via kasha)
-
- HADOOP-11463 Replace method-local TransferManager object with
- S3AFileSystem#transfers. (Ted Yu via stevel)
-
- HADOOP-11520. Clean incomplete multi-part uploads in S3A tests.
- (Thomas Demoor via stevel)
-
- HADOOP-11579. Documentation for truncate. (shv)
-
- HADOOP-10140. Specification of HADOOP_CONF_DIR via the environment in
- hadoop_config.cmd. (Kiran Kumar M R via cnauroth)
-
- HADOOP-11543. Improve help message for hadoop/yarn command. (Brahma
- Reddy Battula via ozawa).
-
- HADOOP-11586. Update use of Iterator to Iterable in
- AbstractMetricsContext.java. (Ray Chiang via aajisaka)
-
- HADOOP-9869. Configuration.getSocketAddr()/getEnum() should use
- getTrimmed(). (Tsuyoshi Ozawa via aajisaka)
-
- HADOOP-11589. NetUtils.createSocketAddr should trim the input URI.
- (Rakesh R via ozawa)
-
- HADOOP-11600. Fix up source codes to be compiled with Guava 17.0. (ozawa)
-
- HADOOP-11522. Update S3A Documentation. (Thomas Demoor via stevel)
-
- HADOOP-11521. Make connection timeout configurable in s3a.
- (Thomas Demoor via stevel)
-
- HADOOP-11440. Use "test.build.data" instead of "build.test.dir" for testing
- in ClientBaseWithFixes. (Kengo Seki via aajisaka)
-
- HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel)
-
- HADOOP-11495. Convert site documentation from apt to markdown
- (Masatake Iwasaki via aw)
-
- HADOOP-11632. Cleanup Find.java to remove SupressWarnings annotations.
- (Akira Ajisaka via ozawa)
-
- HADOOP-10976. moving the source code of hadoop-tools docs to the
- directory under hadoop-tools (Masatake Iwasaki via aw)
-
- HADOOP-11658. Externalize io.compression.codecs property.
- (Kai Zheng via aajisaka)
-
- HADOOP-11648. Set DomainSocketWatcher thread name explicitly.
- (Liang Xie via ozawa)
-
- HADOOP-11642. Upgrade azure sdk version from 0.6.0 to 2.0.0.
- (Shashank Khandelwal and Ivan Mitic via cnauroth)
-
- HADOOP-11714. Add more trace log4j messages to SpanReceiverHost (cmccabe)
-
- HADOOP-8059. Add javadoc to InterfaceAudience and InterfaceStability.
- (Brandon Li via suresh)
-
- HADOOP-9329. document native build dependencies in BUILDING.txt (Vijay Bhat
- via Colin P. McCabe)
-
- HADOOP-10670. Allow AuthenticationFilters to load secret from signature
- secret files. (Kai Zheng via wheat9)
-
- HADOOP-11801. Update BUILDING.txt for Ubuntu. (Gabor Liptak via
- Arpit Agarwal)
-
- HADOOP-9737. JarFinder#getJar should delete the jar file upon destruction
- of the JVM (jbonofre via rkanter)
-
- OPTIMIZATIONS
-
- HADOOP-11323. WritableComparator#compare keeps reference to byte array.
- (Wilfred Spiegelenburg via wang)
-
- HADOOP-10809. hadoop-azure: page blob support. (Dexter Bradshaw,
- Mostafa Elhemali, Eric Hanson, and Mike Liddell via cnauroth)
-
- HADOOP-11188. hadoop-azure: automatically expand page blobs when they become
- full. (Eric Hanson via cnauroth)
-
- HADOOP-11620. Add support for load balancing across a group of KMS for HA.
- (Arun Suresh via wang)
-
- HADOOP-11183. Memory-based S3AOutputstream. (Thomas Demoor via stevel)
-
- BUG FIXES
-
- HADOOP-11512. Use getTrimmedStrings when reading serialization keys
- (Ryan P via harsh)
-
- HADOOP-11488. Difference in default connection timeout for S3A FS
- (Daisuke Kobayashi via harsh)
-
- HADOOP-11256. Some site docs have inconsistent appearance (Masatake
- Iwasaki via aw)
-
- HADOOP-11318. Update the document for hadoop fs -stat (aajisaka)
-
- HADOOP 11400. GraphiteSink does not reconnect to Graphite after 'broken pipe'
- (Kamil Gorlo via raviprak)
-
- HADOOP-11236. NFS: Fix javadoc warning in RpcProgram.java (Abhiraj Butala via harsh)
-
- HADOOP-11166. Remove ulimit from test-patch.sh. (wang)
-
- HDFS-7227. Fix findbugs warning about NP_DEREFERENCE_OF_READLINE_VALUE in
- SpanReceiverHost (cmccabe)
-
- HADOOP-11186. documentation should talk about
- hadoop.htrace.spanreceiver.classes, not hadoop.trace.spanreceiver.classes (cmccabe)
-
- HADOOP-11165. TestUTF8 fails when run against java 8.
- (Stephen Chu via cnauroth)
-
- HADOOP-10717. HttpServer2 should load jsp DTD from local jars instead of
- going remote. (Dapeng Sun via wheat9)
-
- HADOOP-11268. Update BUILDING.txt to remove the workaround for tools.jar.
- (Li Lu via wheat9)
-
- HADOOP-11230. Add missing dependency of bouncycastle for kms, httpfs, hdfs, MR
- and YARN. (Robert Kanter via wheat9)
-
- HADOOP-11269. Add java 8 profile for hadoop-annotations. (Li Lu via wheat9)
-
- HADOOP-11271. Use Time.monotonicNow() in Shell.java instead of Time.now()
- (vinayakumarb)
-
- HADOOP-11266. Remove no longer supported activation properties for packaging
- from pom. (Masatake Iwasaki via wheat9)
-
- HADOOP-11267. TestSecurityUtil fails when run with JDK8 because of empty
- principal names. (Stephen Chu via wheat9)
-
- HADOOP-10714. AmazonS3Client.deleteObjects() need to be limited to 1000
- entries per call. (Juan Yu via atm)
-
- HADOOP-11272. Allow ZKSignerSecretProvider and
- ZKDelegationTokenSecretManager to use the same curator client. (Arun Suresh via atm)
-
- HADOOP-11187 NameNode - KMS communication fails after a long period of
- inactivity. (Arun Suresh via atm)
-
- HADOOP-11289. Fix typo in RpcUtil log message. (Charles Lamb via wheat9)
-
- HADOOP-11294. Nfs3FileAttributes should not change the values of rdev,
- nlink and size in the constructor. (Brandon Li via wheat9)
-
- HADOOP-11157. ZKDelegationTokenSecretManager never shuts down
- listenerThreadPool. (Arun Suresh via atm)
-
- HADOOP-11311. Restrict uppercase key names from being created with JCEKS.
- (wang)
-
- HADOOP-11309. System class pattern package.Foo should match
- package.Foo$Bar, too (Gera Shegalov via jlowe)
-
- HADOOP-11312. Fix unit tests to not use uppercase key names. (wang)
-
- HADOOP-11201. Hadoop Archives should support globs resolving to files.
- (Gera Shegalov via cnauroth)
-
- HADOOP-11322. key based ACL check in KMS always check KeyOpType.MANAGEMENT
- even actual KeyOpType is not MANAGEMENT. (Dian Fu via yliu)
-
- HADOOP-11300. KMS startup scripts must not display the keystore /
- truststore passwords. (Arun Suresh via wang)
-
- HADOOP-11337. KeyAuthorizationKeyProvider access checks need to be done
- atomically. (Dian Fu via wang)
-
- HADOOP-11344. KMS kms-config.sh sets a default value for the keystore
- password even in non-ssl setup. (Arun Suresh via wang)
-
- HADOOP-11342. KMS key ACL should ignore ALL operation for default key ACL
- and whitelist key ACL. (Dian Fu via wang)
-
- HADOOP-11332. KerberosAuthenticator#doSpnegoSequence should check if
- kerberos TGT is available in the subject. (Dian Fu via atm)
-
- HADOOP-11348. Remove unused variable from CMake error message for finding
- openssl (Dian Fu via Colin P. McCabe)
-
- HADOOP-11355. When accessing data in HDFS and the key has been deleted,
- a Null Pointer Exception is shown. (Arun Suresh via wang)
-
- HADOOP-11354. ThrottledInputStream doesn't perform effective throttling.
- (Ted Yu via jing9)
-
- HADOOP-11329. Add JAVA_LIBRARY_PATH to KMS startup options. (Arun Suresh via wang)
-
- HADOOP-11363 Hadoop maven surefire-plugin uses must set heap size. (stevel)
-
- HADOOP-10134 [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc
- comments. (apurtell via stevel)
-
- HADOOP-11367. Fix warnings from findbugs 3.0 in hadoop-streaming. (Li Lu via wheat9)
-
- HADOOP-11369. Fix new findbugs warnings in hadoop-mapreduce-client,
- non-core directories. (Li Lu via wheat9)
-
- HADOOP-11372. Fix new findbugs warnings in mapreduce-examples.
- (Li Lu via wheat9)
-
- HADOOP-11273. TestMiniKdc failure: login options not compatible with IBM
- JDK. (Gao Zhong Liang via wheat9)
-
- HADOOP-11379. Fix new findbugs warnings in hadoop-auth*. (Li Lu via wheat9)
-
- HADOOP-11378. Fix new findbugs warnings in hadoop-kms. (Li Lu via wheat9)
-
- HADOOP-11349. RawLocalFileSystem leaks file descriptor while creating a
- file if creat succeeds but chmod fails. (Varun Saxena via Colin P. McCabe)
-
- HADOOP-11381. Fix findbugs warnings in hadoop-distcp, hadoop-aws,
- hadoop-azure, and hadoop-openstack. (Li Lu via wheat9)
-
- HADOOP-10482. Fix various findbugs warnings in hadoop-common. (wheat9)
-
- HADOOP-11388. Remove deprecated o.a.h.metrics.file.FileContext.
- (Li Lu via wheat9)
-
- HADOOP-11386. Replace \n by %n in format hadoop-common format strings.
- (Li Lu via wheat9)
-
- HADOOP-11211. mapreduce.job.classloader.system.classes semantics should be
- be order-independent. (Yitong Zhou via gera)
-
- HADOOP-11389. Clean up byte to string encoding issues in hadoop-common.
- (wheat9)
-
- HADOOP-11394. hadoop-aws documentation missing. (cnauroth)
-
- HADOOP-11396. Provide navigation in the site documentation linking to the
- Hadoop Compatible File Systems. (cnauroth)
-
- HADOOP-11412 POMs mention "The Apache Software License" rather than
- "Apache License". (Herve Boutemy via stevel)
-
- HADOOP-11321. copyToLocal cannot save a file to an SMB share unless the user
- has Full Control permissions. (cnauroth)
-
- HADOOP-11420. Use latest maven-site-plugin and replace link to svn with
- link to git. (Herve Boutemy via wheat9)
-
- HADOOP-10689. InputStream is not closed in
- AzureNativeFileSystemStore#retrieve(). (Chen He via cnauroth)
-
- HADOOP-10690. Lack of synchronization on access to InputStream in
- NativeAzureFileSystem#NativeAzureFsInputStream#close().
- (Chen He via cnauroth)
-
- HADOOP-11358. Tests for encryption/decryption with IV calculation
- overflow. (yliu)
-
- HADOOP-11125. Remove redundant tests in TestOsSecureRandom.
- (Masanori Oyama via wheat9)
-
- HADOOP-11385. Prevent cross site scripting attack on JMXJSONSe
<TRUNCATED>
[5/5] hadoop git commit: HADOOP-11792. Remove all of the CHANGES.txt
files.
Posted by wa...@apache.org.
HADOOP-11792. Remove all of the CHANGES.txt files.
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/1bc45461
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/1bc45461
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/1bc45461
Branch: refs/heads/trunk
Commit: 1bc454615e916e66b132eb688a4bc8e76dcb0820
Parents: ff0ee84
Author: Andrew Wang <wa...@apache.org>
Authored: Thu Mar 3 21:02:31 2016 -0800
Committer: Andrew Wang <wa...@apache.org>
Committed: Thu Mar 3 21:02:31 2016 -0800
----------------------------------------------------------------------
hadoop-common-project/hadoop-common/CHANGES.txt | 18364 -----------------
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt | 14611 -------------
hadoop-mapreduce-project/CHANGES.txt | 8906 --------
hadoop-yarn-project/CHANGES.txt | 6905 -------
4 files changed, 48786 deletions(-)
----------------------------------------------------------------------