You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-dev@hadoop.apache.org by Sunil G <su...@apache.org> on 2018/11/23 12:06:50 UTC

[VOTE] Release Apache Hadoop 3.2.0 - RC0

Hi folks,



Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/



RC tag in git is release-3.2.0-RC0.



The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/


This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.



3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Sunil G <su...@apache.org>.

Hi Eric,

Thanks for helping in verifying the release.

Post YARN-7370, preemption configs are refreshable. I tried to test by
making some changes in capacity-scheduler.xml and invoking yarn rmadmin
-refreshQueues.
I can see the changes reflected as per logs after refresh. Could you please
help to give some more scenarios so that i can try to reproduce.
Meanwhile i ll try some other combinations as well and let you know.

....
reservationsContinueLooking = true
*preemptionDisabled = true*
defaultAppPriorityPerQueue = 0
priority = 0
maxLifetime = -1 seconds
defaultLifetime = -1 seconds
2018-11-29 06:25:53,792 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=sunilgovindan IP=127.0.0.1 OPERATION=refreshQueues TARGET=AdminService
RESULT=SUCCESS
2018-11-29 06:25:55,900 INFO
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy:
Capacity Scheduler configuration changed, updated preemption properties to:
max_ignored_over_capacity = 0.1
natural_termination_factor = 0.2
max_wait_before_kill = 15000
monitoring_interval = 3000
*total_preemption_per_round = 0.4*
observe_only = false
lazy-preemption-enabled = false
*intra-queue-preemption.enabled = false*
*intra-queue-preemption.max-allowable-limit = 0.4*
intra-queue-preemption.minimum-threshold = 0.5
intra-queue-preemption.preemption-order-policy = USERLIMIT_FIRST
priority-utilization.underutilized-preemption.enabled = false
select_based_on_reserved_containers = false
additional_res_balance_based_on_reserved_containers = false
Preemption-to-balance-queue-enabled = false

*now i disabled preemption for default queue and made some changed in
intraqueue-preemption params.*

reservationsContinueLooking = true
*preemptionDisabled = false*
defaultAppPriorityPerQueue = 0
priority = 0
maxLifetime = -1 seconds
defaultLifetime = -1 seconds
2018-11-29 06:29:32,620 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=sunilgovindan IP=127.0.0.1 OPERATION=refreshQueues TARGET=AdminService
RESULT=SUCCESS
2018-11-29 06:29:34,893 INFO
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy:
Capacity Scheduler configuration changed, updated preemption properties to:
max_ignored_over_capacity = 0.1
natural_termination_factor = 0.2
max_wait_before_kill = 15000
monitoring_interval = 3000
*total_preemption_per_round = 0.7*
observe_only = false
lazy-preemption-enabled = false
*intra-queue-preemption.enabled = true*
*intra-queue-preemption.max-allowable-limit = 0.5*
intra-queue-preemption.minimum-threshold = 0.5
intra-queue-preemption.preemption-order-policy = USERLIMIT_FIRST
priority-utilization.underutilized-preemption.enabled = false
select_based_on_reserved_containers = false
additional_res_balance_based_on_reserved_containers = false
Preemption-to-balance-queue-enabled = false

On Thu, Nov 29, 2018 at 4:19 AM Eric Payne <er...@yahoo.com.invalid>
wrote:

> Sunil,
>
> So, the basic symptoms are that if preemption is enabled on any queue, the
> preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all
> of the preemption-specific properties are set back to the default values.
>
> This was introduced in branch-3.1, so it is NOT new behavior for release
> 3.2.0. I am still tracking down the cause. I will open a JIRA once I do
> further investigation if there is not one already.
>
> This will be a problem for installations which use preemption and which
> use the refreshQueues feature.
>
> Thanks,
> -Eric
>
>
> On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <
> eric.payne1000@yahoo.com> wrote:
>
>
>
>
>
> Sunil, thanks for all of the hard work on this release.
>
> I have discovered that queue refresh doesn't work in some cases. For
> example, when I change
> yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take
> effect unless I restart the RM.
>
> I am still investigating, but I thought I should bring this up asap.
>
> Thanks,
> -Eric
>
>
>
>
> On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org>
> wrote:
>
>
>
>
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Sunil G <su...@apache.org>.

Hi Eric,

Thanks for helping in verifying the release.

Post YARN-7370, preemption configs are refreshable. I tried to test by
making some changes in capacity-scheduler.xml and invoking yarn rmadmin
-refreshQueues.
I can see the changes reflected as per logs after refresh. Could you please
help to give some more scenarios so that i can try to reproduce.
Meanwhile i ll try some other combinations as well and let you know.

....
reservationsContinueLooking = true
*preemptionDisabled = true*
defaultAppPriorityPerQueue = 0
priority = 0
maxLifetime = -1 seconds
defaultLifetime = -1 seconds
2018-11-29 06:25:53,792 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=sunilgovindan IP=127.0.0.1 OPERATION=refreshQueues TARGET=AdminService
RESULT=SUCCESS
2018-11-29 06:25:55,900 INFO
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy:
Capacity Scheduler configuration changed, updated preemption properties to:
max_ignored_over_capacity = 0.1
natural_termination_factor = 0.2
max_wait_before_kill = 15000
monitoring_interval = 3000
*total_preemption_per_round = 0.4*
observe_only = false
lazy-preemption-enabled = false
*intra-queue-preemption.enabled = false*
*intra-queue-preemption.max-allowable-limit = 0.4*
intra-queue-preemption.minimum-threshold = 0.5
intra-queue-preemption.preemption-order-policy = USERLIMIT_FIRST
priority-utilization.underutilized-preemption.enabled = false
select_based_on_reserved_containers = false
additional_res_balance_based_on_reserved_containers = false
Preemption-to-balance-queue-enabled = false

*now i disabled preemption for default queue and made some changed in
intraqueue-preemption params.*

reservationsContinueLooking = true
*preemptionDisabled = false*
defaultAppPriorityPerQueue = 0
priority = 0
maxLifetime = -1 seconds
defaultLifetime = -1 seconds
2018-11-29 06:29:32,620 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=sunilgovindan IP=127.0.0.1 OPERATION=refreshQueues TARGET=AdminService
RESULT=SUCCESS
2018-11-29 06:29:34,893 INFO
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy:
Capacity Scheduler configuration changed, updated preemption properties to:
max_ignored_over_capacity = 0.1
natural_termination_factor = 0.2
max_wait_before_kill = 15000
monitoring_interval = 3000
*total_preemption_per_round = 0.7*
observe_only = false
lazy-preemption-enabled = false
*intra-queue-preemption.enabled = true*
*intra-queue-preemption.max-allowable-limit = 0.5*
intra-queue-preemption.minimum-threshold = 0.5
intra-queue-preemption.preemption-order-policy = USERLIMIT_FIRST
priority-utilization.underutilized-preemption.enabled = false
select_based_on_reserved_containers = false
additional_res_balance_based_on_reserved_containers = false
Preemption-to-balance-queue-enabled = false

On Thu, Nov 29, 2018 at 4:19 AM Eric Payne <er...@yahoo.com.invalid>
wrote:

> Sunil,
>
> So, the basic symptoms are that if preemption is enabled on any queue, the
> preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all
> of the preemption-specific properties are set back to the default values.
>
> This was introduced in branch-3.1, so it is NOT new behavior for release
> 3.2.0. I am still tracking down the cause. I will open a JIRA once I do
> further investigation if there is not one already.
>
> This will be a problem for installations which use preemption and which
> use the refreshQueues feature.
>
> Thanks,
> -Eric
>
>
> On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <
> eric.payne1000@yahoo.com> wrote:
>
>
>
>
>
> Sunil, thanks for all of the hard work on this release.
>
> I have discovered that queue refresh doesn't work in some cases. For
> example, when I change
> yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take
> effect unless I restart the RM.
>
> I am still investigating, but I thought I should bring this up asap.
>
> Thanks,
> -Eric
>
>
>
>
> On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org>
> wrote:
>
>
>
>
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Sunil G <su...@apache.org>.

Thanks @Eric Payne <er...@yahoo.com>
Due to another issue, we have to spin an RC1. In this case, could we revert
the patch which caused this problem.

Cancelling this RC0. Thanks to everyone who voted.
I will spin RC1 as soon as possible.

- Sunil


On Fri, Nov 30, 2018 at 4:14 AM Eric Payne <er...@yahoo.com.invalid>
wrote:

> The problem is not with preemption. The yarn-site.xml that I use for my
> pseudo-cluster includes a second xml:
> xi:include href=".../yarn-scheduler.xml"
>
> The property for yarn.resourcemanager.scheduler.monitor.enable = true is
> in this yarn-scheduler.xml.
>
> This value IS READ when then RM starts.
>
> However, when the refreshQueues command is run, this value IS NOT READ.
>
> So, it looks like xml include files are not read on refresh. This will
> affect any property. I just happened to notice it on the preemption
> properties.
>
> I would like input from all of you to determine if this is a blocker for
> release. I'm on the fence.
>
> Thanks,
> -Eric
>
>
>
>
>
>
> On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne <
> erichadoop-1@yahoo.com.INVALID> wrote:
>
>
>
>
>
> Sunil,
>
> So, the basic symptoms are that if preemption is enabled on any queue, the
> preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all
> of the preemption-specific properties are set back to the default values.
>
> This was introduced in branch-3.1, so it is NOT new behavior for release
> 3.2.0. I am still tracking down the cause. I will open a JIRA once I do
> further investigation if there is not one already.
>
> This will be a problem for installations which use preemption and which
> use the refreshQueues feature.
>
> Thanks,
> -Eric
>
>
> On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <
> eric.payne1000@yahoo.com> wrote:
>
>
>
>
>
> Sunil, thanks for all of the hard work on this release.
>
> I have discovered that queue refresh doesn't work in some cases. For
> example, when I change
> yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take
> effect unless I restart the RM.
>
> I am still investigating, but I thought I should bring this up asap.
>
> Thanks,
> -Eric
>
>
>
>
> On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org>
> wrote:
>
>
>
>
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Sunil G <su...@apache.org>.

Thanks @Eric Payne <er...@yahoo.com>
Due to another issue, we have to spin an RC1. In this case, could we revert
the patch which caused this problem.

Cancelling this RC0. Thanks to everyone who voted.
I will spin RC1 as soon as possible.

- Sunil


On Fri, Nov 30, 2018 at 4:14 AM Eric Payne <er...@yahoo.com.invalid>
wrote:

> The problem is not with preemption. The yarn-site.xml that I use for my
> pseudo-cluster includes a second xml:
> xi:include href=".../yarn-scheduler.xml"
>
> The property for yarn.resourcemanager.scheduler.monitor.enable = true is
> in this yarn-scheduler.xml.
>
> This value IS READ when then RM starts.
>
> However, when the refreshQueues command is run, this value IS NOT READ.
>
> So, it looks like xml include files are not read on refresh. This will
> affect any property. I just happened to notice it on the preemption
> properties.
>
> I would like input from all of you to determine if this is a blocker for
> release. I'm on the fence.
>
> Thanks,
> -Eric
>
>
>
>
>
>
> On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne <
> erichadoop-1@yahoo.com.INVALID> wrote:
>
>
>
>
>
> Sunil,
>
> So, the basic symptoms are that if preemption is enabled on any queue, the
> preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all
> of the preemption-specific properties are set back to the default values.
>
> This was introduced in branch-3.1, so it is NOT new behavior for release
> 3.2.0. I am still tracking down the cause. I will open a JIRA once I do
> further investigation if there is not one already.
>
> This will be a problem for installations which use preemption and which
> use the refreshQueues feature.
>
> Thanks,
> -Eric
>
>
> On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <
> eric.payne1000@yahoo.com> wrote:
>
>
>
>
>
> Sunil, thanks for all of the hard work on this release.
>
> I have discovered that queue refresh doesn't work in some cases. For
> example, when I change
> yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take
> effect unless I restart the RM.
>
> I am still investigating, but I thought I should bring this up asap.
>
> Thanks,
> -Eric
>
>
>
>
> On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org>
> wrote:
>
>
>
>
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Sunil G <su...@apache.org>.

Thanks @Eric Payne <er...@yahoo.com>
Due to another issue, we have to spin an RC1. In this case, could we revert
the patch which caused this problem.

Cancelling this RC0. Thanks to everyone who voted.
I will spin RC1 as soon as possible.

- Sunil


On Fri, Nov 30, 2018 at 4:14 AM Eric Payne <er...@yahoo.com.invalid>
wrote:

> The problem is not with preemption. The yarn-site.xml that I use for my
> pseudo-cluster includes a second xml:
> xi:include href=".../yarn-scheduler.xml"
>
> The property for yarn.resourcemanager.scheduler.monitor.enable = true is
> in this yarn-scheduler.xml.
>
> This value IS READ when then RM starts.
>
> However, when the refreshQueues command is run, this value IS NOT READ.
>
> So, it looks like xml include files are not read on refresh. This will
> affect any property. I just happened to notice it on the preemption
> properties.
>
> I would like input from all of you to determine if this is a blocker for
> release. I'm on the fence.
>
> Thanks,
> -Eric
>
>
>
>
>
>
> On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne <
> erichadoop-1@yahoo.com.INVALID> wrote:
>
>
>
>
>
> Sunil,
>
> So, the basic symptoms are that if preemption is enabled on any queue, the
> preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all
> of the preemption-specific properties are set back to the default values.
>
> This was introduced in branch-3.1, so it is NOT new behavior for release
> 3.2.0. I am still tracking down the cause. I will open a JIRA once I do
> further investigation if there is not one already.
>
> This will be a problem for installations which use preemption and which
> use the refreshQueues feature.
>
> Thanks,
> -Eric
>
>
> On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <
> eric.payne1000@yahoo.com> wrote:
>
>
>
>
>
> Sunil, thanks for all of the hard work on this release.
>
> I have discovered that queue refresh doesn't work in some cases. For
> example, when I change
> yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take
> effect unless I restart the RM.
>
> I am still investigating, but I thought I should bring this up asap.
>
> Thanks,
> -Eric
>
>
>
>
> On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org>
> wrote:
>
>
>
>
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Sunil G <su...@apache.org>.

Thanks @Eric Payne <er...@yahoo.com>
Due to another issue, we have to spin an RC1. In this case, could we revert
the patch which caused this problem.

Cancelling this RC0. Thanks to everyone who voted.
I will spin RC1 as soon as possible.

- Sunil


On Fri, Nov 30, 2018 at 4:14 AM Eric Payne <er...@yahoo.com.invalid>
wrote:

> The problem is not with preemption. The yarn-site.xml that I use for my
> pseudo-cluster includes a second xml:
> xi:include href=".../yarn-scheduler.xml"
>
> The property for yarn.resourcemanager.scheduler.monitor.enable = true is
> in this yarn-scheduler.xml.
>
> This value IS READ when then RM starts.
>
> However, when the refreshQueues command is run, this value IS NOT READ.
>
> So, it looks like xml include files are not read on refresh. This will
> affect any property. I just happened to notice it on the preemption
> properties.
>
> I would like input from all of you to determine if this is a blocker for
> release. I'm on the fence.
>
> Thanks,
> -Eric
>
>
>
>
>
>
> On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne <
> erichadoop-1@yahoo.com.INVALID> wrote:
>
>
>
>
>
> Sunil,
>
> So, the basic symptoms are that if preemption is enabled on any queue, the
> preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all
> of the preemption-specific properties are set back to the default values.
>
> This was introduced in branch-3.1, so it is NOT new behavior for release
> 3.2.0. I am still tracking down the cause. I will open a JIRA once I do
> further investigation if there is not one already.
>
> This will be a problem for installations which use preemption and which
> use the refreshQueues feature.
>
> Thanks,
> -Eric
>
>
> On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <
> eric.payne1000@yahoo.com> wrote:
>
>
>
>
>
> Sunil, thanks for all of the hard work on this release.
>
> I have discovered that queue refresh doesn't work in some cases. For
> example, when I change
> yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take
> effect unless I restart the RM.
>
> I am still investigating, but I thought I should bring this up asap.
>
> Thanks,
> -Eric
>
>
>
>
> On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org>
> wrote:
>
>
>
>
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

The problem is not with preemption. The yarn-site.xml that I use for my pseudo-cluster includes a second xml:
xi:include href=".../yarn-scheduler.xml"

The property for yarn.resourcemanager.scheduler.monitor.enable = true is in this yarn-scheduler.xml.

This value IS READ when then RM starts.

However, when the refreshQueues command is run, this value IS NOT READ.

So, it looks like xml include files are not read on refresh. This will affect any property. I just happened to notice it on the preemption properties.

I would like input from all of you to determine if this is a blocker for release. I'm on the fence.

Thanks,
-Eric

On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne <er...@yahoo.com.INVALID> wrote:

Sunil,

So, the basic symptoms are that if preemption is enabled on any queue, the preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all of the preemption-specific properties are set back to the default values.

This was introduced in branch-3.1, so it is NOT new behavior for release 3.2.0. I am still tracking down the cause. I will open a JIRA once I do further investigation if there is not one already.

This will be a problem for installations which use preemption and which use the refreshQueues feature.

Thanks,
-Eric

On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <er...@yahoo.com> wrote:

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

The problem is not with preemption. The yarn-site.xml that I use for my pseudo-cluster includes a second xml:
xi:include href=".../yarn-scheduler.xml"

The property for yarn.resourcemanager.scheduler.monitor.enable = true is in this yarn-scheduler.xml.

This value IS READ when then RM starts.

However, when the refreshQueues command is run, this value IS NOT READ.

So, it looks like xml include files are not read on refresh. This will affect any property. I just happened to notice it on the preemption properties.

I would like input from all of you to determine if this is a blocker for release. I'm on the fence.

Thanks,
-Eric

On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne <er...@yahoo.com.INVALID> wrote:

Sunil,

This was introduced in branch-3.1, so it is NOT new behavior for release 3.2.0. I am still tracking down the cause. I will open a JIRA once I do further investigation if there is not one already.

This will be a problem for installations which use preemption and which use the refreshQueues feature.

Thanks,
-Eric

On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <er...@yahoo.com> wrote:

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Sunil G <su...@apache.org>.

Hi Eric,

Thanks for helping in verifying the release.

Post YARN-7370, preemption configs are refreshable. I tried to test by
making some changes in capacity-scheduler.xml and invoking yarn rmadmin
-refreshQueues.
I can see the changes reflected as per logs after refresh. Could you please
help to give some more scenarios so that i can try to reproduce.
Meanwhile i ll try some other combinations as well and let you know.

....
reservationsContinueLooking = true
*preemptionDisabled = true*
defaultAppPriorityPerQueue = 0
priority = 0
maxLifetime = -1 seconds
defaultLifetime = -1 seconds
2018-11-29 06:25:53,792 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=sunilgovindan IP=127.0.0.1 OPERATION=refreshQueues TARGET=AdminService
RESULT=SUCCESS
2018-11-29 06:25:55,900 INFO
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy:
Capacity Scheduler configuration changed, updated preemption properties to:
max_ignored_over_capacity = 0.1
natural_termination_factor = 0.2
max_wait_before_kill = 15000
monitoring_interval = 3000
*total_preemption_per_round = 0.4*
observe_only = false
lazy-preemption-enabled = false
*intra-queue-preemption.enabled = false*
*intra-queue-preemption.max-allowable-limit = 0.4*
intra-queue-preemption.minimum-threshold = 0.5
intra-queue-preemption.preemption-order-policy = USERLIMIT_FIRST
priority-utilization.underutilized-preemption.enabled = false
select_based_on_reserved_containers = false
additional_res_balance_based_on_reserved_containers = false
Preemption-to-balance-queue-enabled = false

*now i disabled preemption for default queue and made some changed in
intraqueue-preemption params.*

reservationsContinueLooking = true
*preemptionDisabled = false*
defaultAppPriorityPerQueue = 0
priority = 0
maxLifetime = -1 seconds
defaultLifetime = -1 seconds
2018-11-29 06:29:32,620 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=sunilgovindan IP=127.0.0.1 OPERATION=refreshQueues TARGET=AdminService
RESULT=SUCCESS
2018-11-29 06:29:34,893 INFO
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy:
Capacity Scheduler configuration changed, updated preemption properties to:
max_ignored_over_capacity = 0.1
natural_termination_factor = 0.2
max_wait_before_kill = 15000
monitoring_interval = 3000
*total_preemption_per_round = 0.7*
observe_only = false
lazy-preemption-enabled = false
*intra-queue-preemption.enabled = true*
*intra-queue-preemption.max-allowable-limit = 0.5*
intra-queue-preemption.minimum-threshold = 0.5
intra-queue-preemption.preemption-order-policy = USERLIMIT_FIRST
priority-utilization.underutilized-preemption.enabled = false
select_based_on_reserved_containers = false
additional_res_balance_based_on_reserved_containers = false
Preemption-to-balance-queue-enabled = false

On Thu, Nov 29, 2018 at 4:19 AM Eric Payne <er...@yahoo.com.invalid>
wrote:

> Sunil,
>
> So, the basic symptoms are that if preemption is enabled on any queue, the
> preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all
> of the preemption-specific properties are set back to the default values.
>
> This was introduced in branch-3.1, so it is NOT new behavior for release
> 3.2.0. I am still tracking down the cause. I will open a JIRA once I do
> further investigation if there is not one already.
>
> This will be a problem for installations which use preemption and which
> use the refreshQueues feature.
>
> Thanks,
> -Eric
>
>
> On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <
> eric.payne1000@yahoo.com> wrote:
>
>
>
>
>
> Sunil, thanks for all of the hard work on this release.
>
> I have discovered that queue refresh doesn't work in some cases. For
> example, when I change
> yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take
> effect unless I restart the RM.
>
> I am still investigating, but I thought I should bring this up asap.
>
> Thanks,
> -Eric
>
>
>
>
> On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org>
> wrote:
>
>
>
>
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

The problem is not with preemption. The yarn-site.xml that I use for my pseudo-cluster includes a second xml:
xi:include href=".../yarn-scheduler.xml"

The property for yarn.resourcemanager.scheduler.monitor.enable = true is in this yarn-scheduler.xml.

This value IS READ when then RM starts.

However, when the refreshQueues command is run, this value IS NOT READ.

So, it looks like xml include files are not read on refresh. This will affect any property. I just happened to notice it on the preemption properties.

I would like input from all of you to determine if this is a blocker for release. I'm on the fence.

Thanks,
-Eric

On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne <er...@yahoo.com.INVALID> wrote:

Sunil,

This was introduced in branch-3.1, so it is NOT new behavior for release 3.2.0. I am still tracking down the cause. I will open a JIRA once I do further investigation if there is not one already.

This will be a problem for installations which use preemption and which use the refreshQueues feature.

Thanks,
-Eric

On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <er...@yahoo.com> wrote:

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

The problem is not with preemption. The yarn-site.xml that I use for my pseudo-cluster includes a second xml:
xi:include href=".../yarn-scheduler.xml"

The property for yarn.resourcemanager.scheduler.monitor.enable = true is in this yarn-scheduler.xml.

This value IS READ when then RM starts.

However, when the refreshQueues command is run, this value IS NOT READ.

So, it looks like xml include files are not read on refresh. This will affect any property. I just happened to notice it on the preemption properties.

I would like input from all of you to determine if this is a blocker for release. I'm on the fence.

Thanks,
-Eric

On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne <er...@yahoo.com.INVALID> wrote:

Sunil,

This was introduced in branch-3.1, so it is NOT new behavior for release 3.2.0. I am still tracking down the cause. I will open a JIRA once I do further investigation if there is not one already.

This will be a problem for installations which use preemption and which use the refreshQueues feature.

Thanks,
-Eric

On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <er...@yahoo.com> wrote:

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Sunil G <su...@apache.org>.

Hi Eric,

Thanks for helping in verifying the release.

Post YARN-7370, preemption configs are refreshable. I tried to test by
making some changes in capacity-scheduler.xml and invoking yarn rmadmin
-refreshQueues.
I can see the changes reflected as per logs after refresh. Could you please
help to give some more scenarios so that i can try to reproduce.
Meanwhile i ll try some other combinations as well and let you know.

....
reservationsContinueLooking = true
*preemptionDisabled = true*
defaultAppPriorityPerQueue = 0
priority = 0
maxLifetime = -1 seconds
defaultLifetime = -1 seconds
2018-11-29 06:25:53,792 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=sunilgovindan IP=127.0.0.1 OPERATION=refreshQueues TARGET=AdminService
RESULT=SUCCESS
2018-11-29 06:25:55,900 INFO
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy:
Capacity Scheduler configuration changed, updated preemption properties to:
max_ignored_over_capacity = 0.1
natural_termination_factor = 0.2
max_wait_before_kill = 15000
monitoring_interval = 3000
*total_preemption_per_round = 0.4*
observe_only = false
lazy-preemption-enabled = false
*intra-queue-preemption.enabled = false*
*intra-queue-preemption.max-allowable-limit = 0.4*
intra-queue-preemption.minimum-threshold = 0.5
intra-queue-preemption.preemption-order-policy = USERLIMIT_FIRST
priority-utilization.underutilized-preemption.enabled = false
select_based_on_reserved_containers = false
additional_res_balance_based_on_reserved_containers = false
Preemption-to-balance-queue-enabled = false

*now i disabled preemption for default queue and made some changed in
intraqueue-preemption params.*

reservationsContinueLooking = true
*preemptionDisabled = false*
defaultAppPriorityPerQueue = 0
priority = 0
maxLifetime = -1 seconds
defaultLifetime = -1 seconds
2018-11-29 06:29:32,620 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=sunilgovindan IP=127.0.0.1 OPERATION=refreshQueues TARGET=AdminService
RESULT=SUCCESS
2018-11-29 06:29:34,893 INFO
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy:
Capacity Scheduler configuration changed, updated preemption properties to:
max_ignored_over_capacity = 0.1
natural_termination_factor = 0.2
max_wait_before_kill = 15000
monitoring_interval = 3000
*total_preemption_per_round = 0.7*
observe_only = false
lazy-preemption-enabled = false
*intra-queue-preemption.enabled = true*
*intra-queue-preemption.max-allowable-limit = 0.5*
intra-queue-preemption.minimum-threshold = 0.5
intra-queue-preemption.preemption-order-policy = USERLIMIT_FIRST
priority-utilization.underutilized-preemption.enabled = false
select_based_on_reserved_containers = false
additional_res_balance_based_on_reserved_containers = false
Preemption-to-balance-queue-enabled = false

On Thu, Nov 29, 2018 at 4:19 AM Eric Payne <er...@yahoo.com.invalid>
wrote:

> Sunil,
>
> So, the basic symptoms are that if preemption is enabled on any queue, the
> preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all
> of the preemption-specific properties are set back to the default values.
>
> This was introduced in branch-3.1, so it is NOT new behavior for release
> 3.2.0. I am still tracking down the cause. I will open a JIRA once I do
> further investigation if there is not one already.
>
> This will be a problem for installations which use preemption and which
> use the refreshQueues feature.
>
> Thanks,
> -Eric
>
>
> On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <
> eric.payne1000@yahoo.com> wrote:
>
>
>
>
>
> Sunil, thanks for all of the hard work on this release.
>
> I have discovered that queue refresh doesn't work in some cases. For
> example, when I change
> yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take
> effect unless I restart the RM.
>
> I am still investigating, but I thought I should bring this up asap.
>
> Thanks,
> -Eric
>
>
>
>
> On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org>
> wrote:
>
>
>
>
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

Sunil,

This was introduced in branch-3.1, so it is NOT new behavior for release 3.2.0. I am still tracking down the cause. I will open a JIRA once I do further investigation if there is not one already.

This will be a problem for installations which use preemption and which use the refreshQueues feature.

Thanks,
-Eric

On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <er...@yahoo.com> wrote:

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

Sunil,

This was introduced in branch-3.1, so it is NOT new behavior for release 3.2.0. I am still tracking down the cause. I will open a JIRA once I do further investigation if there is not one already.

This will be a problem for installations which use preemption and which use the refreshQueues feature.

Thanks,
-Eric

On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <er...@yahoo.com> wrote:

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

Sunil,

This was introduced in branch-3.1, so it is NOT new behavior for release 3.2.0. I am still tracking down the cause. I will open a JIRA once I do further investigation if there is not one already.

This will be a problem for installations which use preemption and which use the refreshQueues feature.

Thanks,
-Eric

On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <er...@yahoo.com> wrote:

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

Sunil,

This was introduced in branch-3.1, so it is NOT new behavior for release 3.2.0. I am still tracking down the cause. I will open a JIRA once I do further investigation if there is not one already.

This will be a problem for installations which use preemption and which use the refreshQueues feature.

Thanks,
-Eric

On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <er...@yahoo.com> wrote:

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Takanobu Asanuma <ta...@yahoo-corp.jp>.

Thanks for driving the release, Sunil!

+1(non-binding)
- verified checksums
- succeeded in building the package (CentOS7, Java 8u181)
- started NN HA clusters
- verified Web UI (NN, Router)
- verified some operations of Erasure Coding
- verified some operations of Router-based Federation

Regards,
-Takanobu

on 2018/11/23 21:07, "Sunil G"<su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Chandni Singh <si...@gmail.com>.

+1 (non-binding)

- Build from source
- Launched example sleeper service
- Tested upgrade of sleeper service
- Tested cancellation of upgrade
- Tested express upgrade

On Wed, Nov 28, 2018 at 8:22 AM Peter Bacsko <pb...@cloudera.com.invalid>
wrote:

> +1 (non-binding)
>
> - Built from source at tag 3.2.0-rc0 (Ubuntu 18.10, JDK1.8.0_191)
> - Verified checksums of hadoop-3.2.0.tar.gz
> - Installed on a 3-node physical cluster
> - Ran teragen/terasort/teravalidate
> - Ran distributed shell a couple of times
> - Checked UIs (RM, NM, DN, JHS)
>
> Peter
>
> On Wed, Nov 28, 2018 at 5:17 PM Jason Lowe <jl...@apache.org> wrote:
>
> > Thanks for driving this release, Sunil!
> >
> > +1 (binding)
> >
> > - Verified signatures and digests
> > - Successfully performed a native build
> > - Deployed a single-node cluster
> > - Ran some sample jobs
> >
> > Jason
> >
> > On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:
> >
> > > Hi folks,
> > >
> > >
> > >
> > > Thanks to all contributors who helped in this release [1]. I have
> created
> > >
> > > first release candidate (RC0) for Apache Hadoop 3.2.0.
> > >
> > >
> > > Artifacts for this RC are available here:
> > >
> > > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > >
> > >
> > >
> > > RC tag in git is release-3.2.0-RC0.
> > >
> > >
> > >
> > > The maven artifacts are available via repository.apache.org at
> > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > >
> > >
> > > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> PST.
> > >
> > >
> > >
> > > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > > additions
> > >
> > > are the highlights of this release.
> > >
> > > 1. Node Attributes Support in YARN
> > >
> > > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> > >
> > > 3. Support service upgrade via YARN Service API and CLI
> > >
> > > 4. HDFS Storage Policy Satisfier
> > >
> > > 5. Support Windows Azure Storage - Blob file system in Hadoop
> > >
> > > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > >
> > > 7. Improvements in Router-based HDFS federation
> > >
> > >
> > >
> > > Thanks to Wangda, Vinod, Marton for helping me in preparing the
> release.
> > >
> > > I have done few testing with my pseudo cluster. My +1 to start.
> > >
> > >
> > >
> > > Regards,
> > >
> > > Sunil
> > >
> > >
> > >
> > > [1]
> > >
> > >
> > >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > >
> > > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> (3.2.0)
> > > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > > ORDER BY fixVersion ASC
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Chandni Singh <si...@gmail.com>.

+1 (non-binding)

- Build from source
- Launched example sleeper service
- Tested upgrade of sleeper service
- Tested cancellation of upgrade
- Tested express upgrade

On Wed, Nov 28, 2018 at 8:22 AM Peter Bacsko <pb...@cloudera.com.invalid>
wrote:

> +1 (non-binding)
>
> - Built from source at tag 3.2.0-rc0 (Ubuntu 18.10, JDK1.8.0_191)
> - Verified checksums of hadoop-3.2.0.tar.gz
> - Installed on a 3-node physical cluster
> - Ran teragen/terasort/teravalidate
> - Ran distributed shell a couple of times
> - Checked UIs (RM, NM, DN, JHS)
>
> Peter
>
> On Wed, Nov 28, 2018 at 5:17 PM Jason Lowe <jl...@apache.org> wrote:
>
> > Thanks for driving this release, Sunil!
> >
> > +1 (binding)
> >
> > - Verified signatures and digests
> > - Successfully performed a native build
> > - Deployed a single-node cluster
> > - Ran some sample jobs
> >
> > Jason
> >
> > On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:
> >
> > > Hi folks,
> > >
> > >
> > >
> > > Thanks to all contributors who helped in this release [1]. I have
> created
> > >
> > > first release candidate (RC0) for Apache Hadoop 3.2.0.
> > >
> > >
> > > Artifacts for this RC are available here:
> > >
> > > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > >
> > >
> > >
> > > RC tag in git is release-3.2.0-RC0.
> > >
> > >
> > >
> > > The maven artifacts are available via repository.apache.org at
> > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > >
> > >
> > > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> PST.
> > >
> > >
> > >
> > > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > > additions
> > >
> > > are the highlights of this release.
> > >
> > > 1. Node Attributes Support in YARN
> > >
> > > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> > >
> > > 3. Support service upgrade via YARN Service API and CLI
> > >
> > > 4. HDFS Storage Policy Satisfier
> > >
> > > 5. Support Windows Azure Storage - Blob file system in Hadoop
> > >
> > > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > >
> > > 7. Improvements in Router-based HDFS federation
> > >
> > >
> > >
> > > Thanks to Wangda, Vinod, Marton for helping me in preparing the
> release.
> > >
> > > I have done few testing with my pseudo cluster. My +1 to start.
> > >
> > >
> > >
> > > Regards,
> > >
> > > Sunil
> > >
> > >
> > >
> > > [1]
> > >
> > >
> > >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > >
> > > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> (3.2.0)
> > > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > > ORDER BY fixVersion ASC
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Chandni Singh <si...@gmail.com>.

+1 (non-binding)

- Build from source
- Launched example sleeper service
- Tested upgrade of sleeper service
- Tested cancellation of upgrade
- Tested express upgrade

On Wed, Nov 28, 2018 at 8:22 AM Peter Bacsko <pb...@cloudera.com.invalid>
wrote:

> +1 (non-binding)
>
> - Built from source at tag 3.2.0-rc0 (Ubuntu 18.10, JDK1.8.0_191)
> - Verified checksums of hadoop-3.2.0.tar.gz
> - Installed on a 3-node physical cluster
> - Ran teragen/terasort/teravalidate
> - Ran distributed shell a couple of times
> - Checked UIs (RM, NM, DN, JHS)
>
> Peter
>
> On Wed, Nov 28, 2018 at 5:17 PM Jason Lowe <jl...@apache.org> wrote:
>
> > Thanks for driving this release, Sunil!
> >
> > +1 (binding)
> >
> > - Verified signatures and digests
> > - Successfully performed a native build
> > - Deployed a single-node cluster
> > - Ran some sample jobs
> >
> > Jason
> >
> > On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:
> >
> > > Hi folks,
> > >
> > >
> > >
> > > Thanks to all contributors who helped in this release [1]. I have
> created
> > >
> > > first release candidate (RC0) for Apache Hadoop 3.2.0.
> > >
> > >
> > > Artifacts for this RC are available here:
> > >
> > > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > >
> > >
> > >
> > > RC tag in git is release-3.2.0-RC0.
> > >
> > >
> > >
> > > The maven artifacts are available via repository.apache.org at
> > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > >
> > >
> > > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> PST.
> > >
> > >
> > >
> > > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > > additions
> > >
> > > are the highlights of this release.
> > >
> > > 1. Node Attributes Support in YARN
> > >
> > > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> > >
> > > 3. Support service upgrade via YARN Service API and CLI
> > >
> > > 4. HDFS Storage Policy Satisfier
> > >
> > > 5. Support Windows Azure Storage - Blob file system in Hadoop
> > >
> > > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > >
> > > 7. Improvements in Router-based HDFS federation
> > >
> > >
> > >
> > > Thanks to Wangda, Vinod, Marton for helping me in preparing the
> release.
> > >
> > > I have done few testing with my pseudo cluster. My +1 to start.
> > >
> > >
> > >
> > > Regards,
> > >
> > > Sunil
> > >
> > >
> > >
> > > [1]
> > >
> > >
> > >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > >
> > > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> (3.2.0)
> > > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > > ORDER BY fixVersion ASC
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Chandni Singh <si...@gmail.com>.

+1 (non-binding)

- Build from source
- Launched example sleeper service
- Tested upgrade of sleeper service
- Tested cancellation of upgrade
- Tested express upgrade

On Wed, Nov 28, 2018 at 8:22 AM Peter Bacsko <pb...@cloudera.com.invalid>
wrote:

> +1 (non-binding)
>
> - Built from source at tag 3.2.0-rc0 (Ubuntu 18.10, JDK1.8.0_191)
> - Verified checksums of hadoop-3.2.0.tar.gz
> - Installed on a 3-node physical cluster
> - Ran teragen/terasort/teravalidate
> - Ran distributed shell a couple of times
> - Checked UIs (RM, NM, DN, JHS)
>
> Peter
>
> On Wed, Nov 28, 2018 at 5:17 PM Jason Lowe <jl...@apache.org> wrote:
>
> > Thanks for driving this release, Sunil!
> >
> > +1 (binding)
> >
> > - Verified signatures and digests
> > - Successfully performed a native build
> > - Deployed a single-node cluster
> > - Ran some sample jobs
> >
> > Jason
> >
> > On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:
> >
> > > Hi folks,
> > >
> > >
> > >
> > > Thanks to all contributors who helped in this release [1]. I have
> created
> > >
> > > first release candidate (RC0) for Apache Hadoop 3.2.0.
> > >
> > >
> > > Artifacts for this RC are available here:
> > >
> > > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > >
> > >
> > >
> > > RC tag in git is release-3.2.0-RC0.
> > >
> > >
> > >
> > > The maven artifacts are available via repository.apache.org at
> > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > >
> > >
> > > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> PST.
> > >
> > >
> > >
> > > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > > additions
> > >
> > > are the highlights of this release.
> > >
> > > 1. Node Attributes Support in YARN
> > >
> > > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> > >
> > > 3. Support service upgrade via YARN Service API and CLI
> > >
> > > 4. HDFS Storage Policy Satisfier
> > >
> > > 5. Support Windows Azure Storage - Blob file system in Hadoop
> > >
> > > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > >
> > > 7. Improvements in Router-based HDFS federation
> > >
> > >
> > >
> > > Thanks to Wangda, Vinod, Marton for helping me in preparing the
> release.
> > >
> > > I have done few testing with my pseudo cluster. My +1 to start.
> > >
> > >
> > >
> > > Regards,
> > >
> > > Sunil
> > >
> > >
> > >
> > > [1]
> > >
> > >
> > >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > >
> > > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> (3.2.0)
> > > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > > ORDER BY fixVersion ASC
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Peter Bacsko <pb...@cloudera.com.INVALID>.

+1 (non-binding)

- Built from source at tag 3.2.0-rc0 (Ubuntu 18.10, JDK1.8.0_191)
- Verified checksums of hadoop-3.2.0.tar.gz
- Installed on a 3-node physical cluster
- Ran teragen/terasort/teravalidate
- Ran distributed shell a couple of times
- Checked UIs (RM, NM, DN, JHS)

Peter

On Wed, Nov 28, 2018 at 5:17 PM Jason Lowe <jl...@apache.org> wrote:

> Thanks for driving this release, Sunil!
>
> +1 (binding)
>
> - Verified signatures and digests
> - Successfully performed a native build
> - Deployed a single-node cluster
> - Ran some sample jobs
>
> Jason
>
> On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:
>
> > Hi folks,
> >
> >
> >
> > Thanks to all contributors who helped in this release [1]. I have created
> >
> > first release candidate (RC0) for Apache Hadoop 3.2.0.
> >
> >
> > Artifacts for this RC are available here:
> >
> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >
> >
> >
> > RC tag in git is release-3.2.0-RC0.
> >
> >
> >
> > The maven artifacts are available via repository.apache.org at
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >
> >
> > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> >
> >
> >
> > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > additions
> >
> > are the highlights of this release.
> >
> > 1. Node Attributes Support in YARN
> >
> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >
> > 3. Support service upgrade via YARN Service API and CLI
> >
> > 4. HDFS Storage Policy Satisfier
> >
> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >
> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >
> > 7. Improvements in Router-based HDFS federation
> >
> >
> >
> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >
> > I have done few testing with my pseudo cluster. My +1 to start.
> >
> >
> >
> > Regards,
> >
> > Sunil
> >
> >
> >
> > [1]
> >
> >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > ORDER BY fixVersion ASC
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Peter Bacsko <pb...@cloudera.com.INVALID>.

+1 (non-binding)

- Built from source at tag 3.2.0-rc0 (Ubuntu 18.10, JDK1.8.0_191)
- Verified checksums of hadoop-3.2.0.tar.gz
- Installed on a 3-node physical cluster
- Ran teragen/terasort/teravalidate
- Ran distributed shell a couple of times
- Checked UIs (RM, NM, DN, JHS)

Peter

On Wed, Nov 28, 2018 at 5:17 PM Jason Lowe <jl...@apache.org> wrote:

> Thanks for driving this release, Sunil!
>
> +1 (binding)
>
> - Verified signatures and digests
> - Successfully performed a native build
> - Deployed a single-node cluster
> - Ran some sample jobs
>
> Jason
>
> On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:
>
> > Hi folks,
> >
> >
> >
> > Thanks to all contributors who helped in this release [1]. I have created
> >
> > first release candidate (RC0) for Apache Hadoop 3.2.0.
> >
> >
> > Artifacts for this RC are available here:
> >
> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >
> >
> >
> > RC tag in git is release-3.2.0-RC0.
> >
> >
> >
> > The maven artifacts are available via repository.apache.org at
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >
> >
> > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> >
> >
> >
> > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > additions
> >
> > are the highlights of this release.
> >
> > 1. Node Attributes Support in YARN
> >
> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >
> > 3. Support service upgrade via YARN Service API and CLI
> >
> > 4. HDFS Storage Policy Satisfier
> >
> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >
> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >
> > 7. Improvements in Router-based HDFS federation
> >
> >
> >
> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >
> > I have done few testing with my pseudo cluster. My +1 to start.
> >
> >
> >
> > Regards,
> >
> > Sunil
> >
> >
> >
> > [1]
> >
> >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > ORDER BY fixVersion ASC
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Peter Bacsko <pb...@cloudera.com.INVALID>.

+1 (non-binding)

- Built from source at tag 3.2.0-rc0 (Ubuntu 18.10, JDK1.8.0_191)
- Verified checksums of hadoop-3.2.0.tar.gz
- Installed on a 3-node physical cluster
- Ran teragen/terasort/teravalidate
- Ran distributed shell a couple of times
- Checked UIs (RM, NM, DN, JHS)

Peter

On Wed, Nov 28, 2018 at 5:17 PM Jason Lowe <jl...@apache.org> wrote:

> Thanks for driving this release, Sunil!
>
> +1 (binding)
>
> - Verified signatures and digests
> - Successfully performed a native build
> - Deployed a single-node cluster
> - Ran some sample jobs
>
> Jason
>
> On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:
>
> > Hi folks,
> >
> >
> >
> > Thanks to all contributors who helped in this release [1]. I have created
> >
> > first release candidate (RC0) for Apache Hadoop 3.2.0.
> >
> >
> > Artifacts for this RC are available here:
> >
> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >
> >
> >
> > RC tag in git is release-3.2.0-RC0.
> >
> >
> >
> > The maven artifacts are available via repository.apache.org at
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >
> >
> > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> >
> >
> >
> > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > additions
> >
> > are the highlights of this release.
> >
> > 1. Node Attributes Support in YARN
> >
> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >
> > 3. Support service upgrade via YARN Service API and CLI
> >
> > 4. HDFS Storage Policy Satisfier
> >
> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >
> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >
> > 7. Improvements in Router-based HDFS federation
> >
> >
> >
> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >
> > I have done few testing with my pseudo cluster. My +1 to start.
> >
> >
> >
> > Regards,
> >
> > Sunil
> >
> >
> >
> > [1]
> >
> >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > ORDER BY fixVersion ASC
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Peter Bacsko <pb...@cloudera.com.INVALID>.

+1 (non-binding)

- Built from source at tag 3.2.0-rc0 (Ubuntu 18.10, JDK1.8.0_191)
- Verified checksums of hadoop-3.2.0.tar.gz
- Installed on a 3-node physical cluster
- Ran teragen/terasort/teravalidate
- Ran distributed shell a couple of times
- Checked UIs (RM, NM, DN, JHS)

Peter

On Wed, Nov 28, 2018 at 5:17 PM Jason Lowe <jl...@apache.org> wrote:

> Thanks for driving this release, Sunil!
>
> +1 (binding)
>
> - Verified signatures and digests
> - Successfully performed a native build
> - Deployed a single-node cluster
> - Ran some sample jobs
>
> Jason
>
> On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:
>
> > Hi folks,
> >
> >
> >
> > Thanks to all contributors who helped in this release [1]. I have created
> >
> > first release candidate (RC0) for Apache Hadoop 3.2.0.
> >
> >
> > Artifacts for this RC are available here:
> >
> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >
> >
> >
> > RC tag in git is release-3.2.0-RC0.
> >
> >
> >
> > The maven artifacts are available via repository.apache.org at
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >
> >
> > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> >
> >
> >
> > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > additions
> >
> > are the highlights of this release.
> >
> > 1. Node Attributes Support in YARN
> >
> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >
> > 3. Support service upgrade via YARN Service API and CLI
> >
> > 4. HDFS Storage Policy Satisfier
> >
> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >
> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >
> > 7. Improvements in Router-based HDFS federation
> >
> >
> >
> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >
> > I have done few testing with my pseudo cluster. My +1 to start.
> >
> >
> >
> > Regards,
> >
> > Sunil
> >
> >
> >
> > [1]
> >
> >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > ORDER BY fixVersion ASC
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Jason Lowe <jl...@apache.org>.

Thanks for driving this release, Sunil!

+1 (binding)

- Verified signatures and digests
- Successfully performed a native build
- Deployed a single-node cluster
- Ran some sample jobs

Jason

On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:

> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Jason Lowe <jl...@apache.org>.

Thanks for driving this release, Sunil!

+1 (binding)

- Verified signatures and digests
- Successfully performed a native build
- Deployed a single-node cluster
- Ran some sample jobs

Jason

On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:

> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Takanobu Asanuma <ta...@yahoo-corp.jp>.

Thanks for driving the release, Sunil!

Regards,
-Takanobu

on 2018/11/23 21:07, "Sunil G"<su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Jason Lowe <jl...@apache.org>.

Thanks for driving this release, Sunil!

+1 (binding)

- Verified signatures and digests
- Successfully performed a native build
- Deployed a single-node cluster
- Ran some sample jobs

Jason

On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:

> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Zsolt Venczel <zv...@cloudera.com.INVALID>.

Thanks Sunil for preparing this release!

+1 (non-binding)

- Built from source at tag 3.2.0-rc0 (Ubuntu 16.04, JDK1.8.0_191)
- Installed on a 4-node virtual cluster
- Executed teragen/terasort/teravalidate with and without EC (using
XOR-2-1-1024k policy)
- Created two snapshots and performed snapshot diff
- Checked web UIs (NN, DN, RM)
- Run unit tests for hadoop-hdfs (found one failure case of
https://issues.apache.org/jira/browse/HDFS-12116 that is not a regression)

Best regards,
Zsolt

On Thu, Nov 29, 2018 at 6:15 AM Wilfred Spiegelenburg
<ws...@cloudera.com.invalid> wrote:

> +1 (non binding)
>
> - build from source on MacOSX 10.14.1, 1.8.0u181
> - successful native build on Ubuntu 16.04.3
> - confirmed the checksum and signature
> - deployed a single node cluster  (jdk 1.8u191 / centos 7.5)
>
> Wilfred
>
> > On 23 Nov 2018, at 23:06, Sunil G <su...@apache.org> wrote:
> >
> > Hi folks,
> >
> >
> >
> > Thanks to all contributors who helped in this release [1]. I have created
> >
> > first release candidate (RC0) for Apache Hadoop 3.2.0.
> >
> >
> > Artifacts for this RC are available here:
> >
> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >
> >
> >
> > RC tag in git is release-3.2.0-RC0.
> >
> >
> >
> > The maven artifacts are available via repository.apache.org at
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >
> >
> > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> >
> >
> >
> > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > additions
> >
> > are the highlights of this release.
> >
> > 1. Node Attributes Support in YARN
> >
> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >
> > 3. Support service upgrade via YARN Service API and CLI
> >
> > 4. HDFS Storage Policy Satisfier
> >
> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >
> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >
> > 7. Improvements in Router-based HDFS federation
> >
> >
> >
> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >
> > I have done few testing with my pseudo cluster. My +1 to start.
> >
> >
> >
> > Regards,
> >
> > Sunil
> >
> >
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > ORDER BY fixVersion ASC
>
>
> Wilfred Spiegelenburg | Software Engineer
> cloudera.com <https://www.cloudera.com/>
>
>
>
>
>
>
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Zsolt Venczel <zv...@cloudera.com.INVALID>.

Thanks Sunil for preparing this release!

+1 (non-binding)

- Built from source at tag 3.2.0-rc0 (Ubuntu 16.04, JDK1.8.0_191)
- Installed on a 4-node virtual cluster
- Executed teragen/terasort/teravalidate with and without EC (using
XOR-2-1-1024k policy)
- Created two snapshots and performed snapshot diff
- Checked web UIs (NN, DN, RM)
- Run unit tests for hadoop-hdfs (found one failure case of
https://issues.apache.org/jira/browse/HDFS-12116 that is not a regression)

Best regards,
Zsolt

On Thu, Nov 29, 2018 at 6:15 AM Wilfred Spiegelenburg
<ws...@cloudera.com.invalid> wrote:

> +1 (non binding)
>
> - build from source on MacOSX 10.14.1, 1.8.0u181
> - successful native build on Ubuntu 16.04.3
> - confirmed the checksum and signature
> - deployed a single node cluster  (jdk 1.8u191 / centos 7.5)
>
> Wilfred
>
> > On 23 Nov 2018, at 23:06, Sunil G <su...@apache.org> wrote:
> >
> > Hi folks,
> >
> >
> >
> > Thanks to all contributors who helped in this release [1]. I have created
> >
> > first release candidate (RC0) for Apache Hadoop 3.2.0.
> >
> >
> > Artifacts for this RC are available here:
> >
> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >
> >
> >
> > RC tag in git is release-3.2.0-RC0.
> >
> >
> >
> > The maven artifacts are available via repository.apache.org at
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >
> >
> > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> >
> >
> >
> > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > additions
> >
> > are the highlights of this release.
> >
> > 1. Node Attributes Support in YARN
> >
> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >
> > 3. Support service upgrade via YARN Service API and CLI
> >
> > 4. HDFS Storage Policy Satisfier
> >
> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >
> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >
> > 7. Improvements in Router-based HDFS federation
> >
> >
> >
> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >
> > I have done few testing with my pseudo cluster. My +1 to start.
> >
> >
> >
> > Regards,
> >
> > Sunil
> >
> >
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > ORDER BY fixVersion ASC
>
>
> Wilfred Spiegelenburg | Software Engineer
> cloudera.com <https://www.cloudera.com/>
>
>
>
>
>
>
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Wilfred Spiegelenburg <ws...@cloudera.com.INVALID>.

+1 (non binding)

- build from source on MacOSX 10.14.1, 1.8.0u181
- successful native build on Ubuntu 16.04.3
- confirmed the checksum and signature
- deployed a single node cluster  (jdk 1.8u191 / centos 7.5)

Wilfred

> On 23 Nov 2018, at 23:06, Sunil G <su...@apache.org> wrote:
> 
> Hi folks,
> 
> 
> 
> Thanks to all contributors who helped in this release [1]. I have created
> 
> first release candidate (RC0) for Apache Hadoop 3.2.0.
> 
> 
> Artifacts for this RC are available here:
> 
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> 
> 
> 
> RC tag in git is release-3.2.0-RC0.
> 
> 
> 
> The maven artifacts are available via repository.apache.org at
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> 
> 
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> 
> 
> 
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
> 
> are the highlights of this release.
> 
> 1. Node Attributes Support in YARN
> 
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> 
> 3. Support service upgrade via YARN Service API and CLI
> 
> 4. HDFS Storage Policy Satisfier
> 
> 5. Support Windows Azure Storage - Blob file system in Hadoop
> 
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> 
> 7. Improvements in Router-based HDFS federation
> 
> 
> 
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> 
> I have done few testing with my pseudo cluster. My +1 to start.
> 
> 
> 
> Regards,
> 
> Sunil
> 
> 
> 
> [1]
> 
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> 
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC


Wilfred Spiegelenburg | Software Engineer
cloudera.com <https://www.cloudera.com/>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Iñigo Goiri <el...@gmail.com>.

+1 (non-binding)

- Deployed a cluster with 3 NNs, 3 RMs and 1 DN/NM on Azure
- Tested the Active probe for the Load Balancer in front of the NNs and the
RMs
- Checked the NN, RBF, and RM Web UIs
- Executed a wordcount, TeraGen, TeraSort  and TeraValidate
- Executed a YARN service with a TensorFlow app on Docker
- Scaled up the cluster to 5 DNs/NMs and executed the tests again
- Checked the HDFS output folders through the Web UI



On Tue, Nov 27, 2018 at 9:37 AM Gabor Bota <ga...@cloudera.com.invalid>
wrote:

> Thanks for the work Sunil!
>
> +1 (non-binding)
>
> I've done the following:
>
>    - checked out git tag release-3.2.0-RC0
>    - built from source on Mac OS X 10.14.1, java: 8.0.181-oracle
>    - run hadoop-aws tests on AWS S3 against eu-west-1, with no unknown
>    issues
>    - deployed on a 3 node cluster
>    - verified example pi job
>    - ran teragen, terasort and teravalidate without any error
>
>
> Regards,
> Gabor Bota
>
> On Tue, Nov 27, 2018 at 5:56 PM Eric Badger <eb...@oath.com.invalid>
> wrote:
>
> > +1 (non-binding)
> >
> > - Verified all hashes and checksums
> > - Built from source on macOS 10.14.1, Java 1.8.0u65
> > - Deployed a pseudo cluster
> > - Ran some example jobs
> >
> > Eric
> >
> > On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:
> >
> > > Sunil, Thanks for driving this release!!!
> > >
> > > +1 (non-binding)
> > > -Built from source on Ubuntu 17.04 and JDK 8
> > > -Ran basic Hdfs Commands
> > > -Ran basic EC Commands
> > > -Ran basic RBF commands
> > > -Browsed HDFS and RBF WEB UI
> > > -Ran TeraGen/TeraSort
> > >
> > > Regards
> > > Ayush
> > >
> > > > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi
> <knanasi@cloudera.com.INVALID
> > >
> > > wrote:
> > > >
> > > > Thanks Sunil for the driving the release!
> > > >
> > > > +1 (non-binding)
> > > >
> > > > - checked out git tag release-3.2.0-RC0
> > > > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > > > - deployed on a 5 node cluster
> > > > - ran terasort, teragen, teravalidate with success
> > > > - executed basic hdfs, dfsadmin and ec commands
> > > >
> > > > Best,
> > > > Kitti
> > > >
> > > >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> > > >>
> > > >> Hi folks,
> > > >>
> > > >>
> > > >>
> > > >> Thanks to all contributors who helped in this release [1]. I have
> > > created
> > > >>
> > > >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> > > >>
> > > >>
> > > >> Artifacts for this RC are available here:
> > > >>
> > > >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > > >>
> > > >>
> > > >>
> > > >> RC tag in git is release-3.2.0-RC0.
> > > >>
> > > >>
> > > >>
> > > >> The maven artifacts are available via repository.apache.org at
> > > >>
> > > >>
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > > >>
> > > >>
> > > >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> > > PST.
> > > >>
> > > >>
> > > >>
> > > >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > > >> additions
> > > >>
> > > >> are the highlights of this release.
> > > >>
> > > >> 1. Node Attributes Support in YARN
> > > >>
> > > >> 2. Hadoop Submarine project for running Deep Learning workloads on
> > YARN
> > > >>
> > > >> 3. Support service upgrade via YARN Service API and CLI
> > > >>
> > > >> 4. HDFS Storage Policy Satisfier
> > > >>
> > > >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> > > >>
> > > >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > > >>
> > > >> 7. Improvements in Router-based HDFS federation
> > > >>
> > > >>
> > > >>
> > > >> Thanks to Wangda, Vinod, Marton for helping me in preparing the
> > release.
> > > >>
> > > >> I have done few testing with my pseudo cluster. My +1 to start.
> > > >>
> > > >>
> > > >>
> > > >> Regards,
> > > >>
> > > >> Sunil
> > > >>
> > > >>
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > > >>
> > > >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> > (3.2.0)
> > > >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
> > Resolved
> > > >> ORDER BY fixVersion ASC
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Iñigo Goiri <el...@gmail.com>.

+1 (non-binding)

- Deployed a cluster with 3 NNs, 3 RMs and 1 DN/NM on Azure
- Tested the Active probe for the Load Balancer in front of the NNs and the
RMs
- Checked the NN, RBF, and RM Web UIs
- Executed a wordcount, TeraGen, TeraSort  and TeraValidate
- Executed a YARN service with a TensorFlow app on Docker
- Scaled up the cluster to 5 DNs/NMs and executed the tests again
- Checked the HDFS output folders through the Web UI



On Tue, Nov 27, 2018 at 9:37 AM Gabor Bota <ga...@cloudera.com.invalid>
wrote:

> Thanks for the work Sunil!
>
> +1 (non-binding)
>
> I've done the following:
>
>    - checked out git tag release-3.2.0-RC0
>    - built from source on Mac OS X 10.14.1, java: 8.0.181-oracle
>    - run hadoop-aws tests on AWS S3 against eu-west-1, with no unknown
>    issues
>    - deployed on a 3 node cluster
>    - verified example pi job
>    - ran teragen, terasort and teravalidate without any error
>
>
> Regards,
> Gabor Bota
>
> On Tue, Nov 27, 2018 at 5:56 PM Eric Badger <eb...@oath.com.invalid>
> wrote:
>
> > +1 (non-binding)
> >
> > - Verified all hashes and checksums
> > - Built from source on macOS 10.14.1, Java 1.8.0u65
> > - Deployed a pseudo cluster
> > - Ran some example jobs
> >
> > Eric
> >
> > On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:
> >
> > > Sunil, Thanks for driving this release!!!
> > >
> > > +1 (non-binding)
> > > -Built from source on Ubuntu 17.04 and JDK 8
> > > -Ran basic Hdfs Commands
> > > -Ran basic EC Commands
> > > -Ran basic RBF commands
> > > -Browsed HDFS and RBF WEB UI
> > > -Ran TeraGen/TeraSort
> > >
> > > Regards
> > > Ayush
> > >
> > > > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi
> <knanasi@cloudera.com.INVALID
> > >
> > > wrote:
> > > >
> > > > Thanks Sunil for the driving the release!
> > > >
> > > > +1 (non-binding)
> > > >
> > > > - checked out git tag release-3.2.0-RC0
> > > > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > > > - deployed on a 5 node cluster
> > > > - ran terasort, teragen, teravalidate with success
> > > > - executed basic hdfs, dfsadmin and ec commands
> > > >
> > > > Best,
> > > > Kitti
> > > >
> > > >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> > > >>
> > > >> Hi folks,
> > > >>
> > > >>
> > > >>
> > > >> Thanks to all contributors who helped in this release [1]. I have
> > > created
> > > >>
> > > >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> > > >>
> > > >>
> > > >> Artifacts for this RC are available here:
> > > >>
> > > >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > > >>
> > > >>
> > > >>
> > > >> RC tag in git is release-3.2.0-RC0.
> > > >>
> > > >>
> > > >>
> > > >> The maven artifacts are available via repository.apache.org at
> > > >>
> > > >>
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > > >>
> > > >>
> > > >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> > > PST.
> > > >>
> > > >>
> > > >>
> > > >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > > >> additions
> > > >>
> > > >> are the highlights of this release.
> > > >>
> > > >> 1. Node Attributes Support in YARN
> > > >>
> > > >> 2. Hadoop Submarine project for running Deep Learning workloads on
> > YARN
> > > >>
> > > >> 3. Support service upgrade via YARN Service API and CLI
> > > >>
> > > >> 4. HDFS Storage Policy Satisfier
> > > >>
> > > >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> > > >>
> > > >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > > >>
> > > >> 7. Improvements in Router-based HDFS federation
> > > >>
> > > >>
> > > >>
> > > >> Thanks to Wangda, Vinod, Marton for helping me in preparing the
> > release.
> > > >>
> > > >> I have done few testing with my pseudo cluster. My +1 to start.
> > > >>
> > > >>
> > > >>
> > > >> Regards,
> > > >>
> > > >> Sunil
> > > >>
> > > >>
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > > >>
> > > >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> > (3.2.0)
> > > >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
> > Resolved
> > > >> ORDER BY fixVersion ASC
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Gergely Pollak <sh...@cloudera.com.INVALID>.

Thanks for the work Sunil!

+1 (non-binding)

 - Checked out release-3.2.0-RC0
 - Built it on Debian 9.6 with Oracle Java 1.8.0_191-b12
 - Created a pseudo distributed cluser
 - Tested with tera(gen/sort/validate)
 - Also executed the pi job

Regards,
  Gergely Pollak


On Tue, Nov 27, 2018 at 6:37 PM Gabor Bota <ga...@cloudera.com.invalid>
wrote:

> Thanks for the work Sunil!
>
> +1 (non-binding)
>
> I've done the following:
>
>    - checked out git tag release-3.2.0-RC0
>    - built from source on Mac OS X 10.14.1, java: 8.0.181-oracle
>    - run hadoop-aws tests on AWS S3 against eu-west-1, with no unknown
>    issues
>    - deployed on a 3 node cluster
>    - verified example pi job
>    - ran teragen, terasort and teravalidate without any error
>
>
> Regards,
> Gabor Bota
>
> On Tue, Nov 27, 2018 at 5:56 PM Eric Badger <eb...@oath.com.invalid>
> wrote:
>
> > +1 (non-binding)
> >
> > - Verified all hashes and checksums
> > - Built from source on macOS 10.14.1, Java 1.8.0u65
> > - Deployed a pseudo cluster
> > - Ran some example jobs
> >
> > Eric
> >
> > On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:
> >
> > > Sunil, Thanks for driving this release!!!
> > >
> > > +1 (non-binding)
> > > -Built from source on Ubuntu 17.04 and JDK 8
> > > -Ran basic Hdfs Commands
> > > -Ran basic EC Commands
> > > -Ran basic RBF commands
> > > -Browsed HDFS and RBF WEB UI
> > > -Ran TeraGen/TeraSort
> > >
> > > Regards
> > > Ayush
> > >
> > > > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi
> <knanasi@cloudera.com.INVALID
> > >
> > > wrote:
> > > >
> > > > Thanks Sunil for the driving the release!
> > > >
> > > > +1 (non-binding)
> > > >
> > > > - checked out git tag release-3.2.0-RC0
> > > > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > > > - deployed on a 5 node cluster
> > > > - ran terasort, teragen, teravalidate with success
> > > > - executed basic hdfs, dfsadmin and ec commands
> > > >
> > > > Best,
> > > > Kitti
> > > >
> > > >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> > > >>
> > > >> Hi folks,
> > > >>
> > > >>
> > > >>
> > > >> Thanks to all contributors who helped in this release [1]. I have
> > > created
> > > >>
> > > >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> > > >>
> > > >>
> > > >> Artifacts for this RC are available here:
> > > >>
> > > >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > > >>
> > > >>
> > > >>
> > > >> RC tag in git is release-3.2.0-RC0.
> > > >>
> > > >>
> > > >>
> > > >> The maven artifacts are available via repository.apache.org at
> > > >>
> > > >>
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > > >>
> > > >>
> > > >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> > > PST.
> > > >>
> > > >>
> > > >>
> > > >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > > >> additions
> > > >>
> > > >> are the highlights of this release.
> > > >>
> > > >> 1. Node Attributes Support in YARN
> > > >>
> > > >> 2. Hadoop Submarine project for running Deep Learning workloads on
> > YARN
> > > >>
> > > >> 3. Support service upgrade via YARN Service API and CLI
> > > >>
> > > >> 4. HDFS Storage Policy Satisfier
> > > >>
> > > >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> > > >>
> > > >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > > >>
> > > >> 7. Improvements in Router-based HDFS federation
> > > >>
> > > >>
> > > >>
> > > >> Thanks to Wangda, Vinod, Marton for helping me in preparing the
> > release.
> > > >>
> > > >> I have done few testing with my pseudo cluster. My +1 to start.
> > > >>
> > > >>
> > > >>
> > > >> Regards,
> > > >>
> > > >> Sunil
> > > >>
> > > >>
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > > >>
> > > >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> > (3.2.0)
> > > >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
> > Resolved
> > > >> ORDER BY fixVersion ASC
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Iñigo Goiri <el...@gmail.com>.

+1 (non-binding)

- Deployed a cluster with 3 NNs, 3 RMs and 1 DN/NM on Azure
- Tested the Active probe for the Load Balancer in front of the NNs and the
RMs
- Checked the NN, RBF, and RM Web UIs
- Executed a wordcount, TeraGen, TeraSort  and TeraValidate
- Executed a YARN service with a TensorFlow app on Docker
- Scaled up the cluster to 5 DNs/NMs and executed the tests again
- Checked the HDFS output folders through the Web UI



On Tue, Nov 27, 2018 at 9:37 AM Gabor Bota <ga...@cloudera.com.invalid>
wrote:

> Thanks for the work Sunil!
>
> +1 (non-binding)
>
> I've done the following:
>
>    - checked out git tag release-3.2.0-RC0
>    - built from source on Mac OS X 10.14.1, java: 8.0.181-oracle
>    - run hadoop-aws tests on AWS S3 against eu-west-1, with no unknown
>    issues
>    - deployed on a 3 node cluster
>    - verified example pi job
>    - ran teragen, terasort and teravalidate without any error
>
>
> Regards,
> Gabor Bota
>
> On Tue, Nov 27, 2018 at 5:56 PM Eric Badger <eb...@oath.com.invalid>
> wrote:
>
> > +1 (non-binding)
> >
> > - Verified all hashes and checksums
> > - Built from source on macOS 10.14.1, Java 1.8.0u65
> > - Deployed a pseudo cluster
> > - Ran some example jobs
> >
> > Eric
> >
> > On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:
> >
> > > Sunil, Thanks for driving this release!!!
> > >
> > > +1 (non-binding)
> > > -Built from source on Ubuntu 17.04 and JDK 8
> > > -Ran basic Hdfs Commands
> > > -Ran basic EC Commands
> > > -Ran basic RBF commands
> > > -Browsed HDFS and RBF WEB UI
> > > -Ran TeraGen/TeraSort
> > >
> > > Regards
> > > Ayush
> > >
> > > > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi
> <knanasi@cloudera.com.INVALID
> > >
> > > wrote:
> > > >
> > > > Thanks Sunil for the driving the release!
> > > >
> > > > +1 (non-binding)
> > > >
> > > > - checked out git tag release-3.2.0-RC0
> > > > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > > > - deployed on a 5 node cluster
> > > > - ran terasort, teragen, teravalidate with success
> > > > - executed basic hdfs, dfsadmin and ec commands
> > > >
> > > > Best,
> > > > Kitti
> > > >
> > > >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> > > >>
> > > >> Hi folks,
> > > >>
> > > >>
> > > >>
> > > >> Thanks to all contributors who helped in this release [1]. I have
> > > created
> > > >>
> > > >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> > > >>
> > > >>
> > > >> Artifacts for this RC are available here:
> > > >>
> > > >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > > >>
> > > >>
> > > >>
> > > >> RC tag in git is release-3.2.0-RC0.
> > > >>
> > > >>
> > > >>
> > > >> The maven artifacts are available via repository.apache.org at
> > > >>
> > > >>
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > > >>
> > > >>
> > > >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> > > PST.
> > > >>
> > > >>
> > > >>
> > > >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > > >> additions
> > > >>
> > > >> are the highlights of this release.
> > > >>
> > > >> 1. Node Attributes Support in YARN
> > > >>
> > > >> 2. Hadoop Submarine project for running Deep Learning workloads on
> > YARN
> > > >>
> > > >> 3. Support service upgrade via YARN Service API and CLI
> > > >>
> > > >> 4. HDFS Storage Policy Satisfier
> > > >>
> > > >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> > > >>
> > > >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > > >>
> > > >> 7. Improvements in Router-based HDFS federation
> > > >>
> > > >>
> > > >>
> > > >> Thanks to Wangda, Vinod, Marton for helping me in preparing the
> > release.
> > > >>
> > > >> I have done few testing with my pseudo cluster. My +1 to start.
> > > >>
> > > >>
> > > >>
> > > >> Regards,
> > > >>
> > > >> Sunil
> > > >>
> > > >>
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > > >>
> > > >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> > (3.2.0)
> > > >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
> > Resolved
> > > >> ORDER BY fixVersion ASC
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Iñigo Goiri <el...@gmail.com>.

+1 (non-binding)

- Deployed a cluster with 3 NNs, 3 RMs and 1 DN/NM on Azure
- Tested the Active probe for the Load Balancer in front of the NNs and the
RMs
- Checked the NN, RBF, and RM Web UIs
- Executed a wordcount, TeraGen, TeraSort  and TeraValidate
- Executed a YARN service with a TensorFlow app on Docker
- Scaled up the cluster to 5 DNs/NMs and executed the tests again
- Checked the HDFS output folders through the Web UI



On Tue, Nov 27, 2018 at 9:37 AM Gabor Bota <ga...@cloudera.com.invalid>
wrote:

> Thanks for the work Sunil!
>
> +1 (non-binding)
>
> I've done the following:
>
>    - checked out git tag release-3.2.0-RC0
>    - built from source on Mac OS X 10.14.1, java: 8.0.181-oracle
>    - run hadoop-aws tests on AWS S3 against eu-west-1, with no unknown
>    issues
>    - deployed on a 3 node cluster
>    - verified example pi job
>    - ran teragen, terasort and teravalidate without any error
>
>
> Regards,
> Gabor Bota
>
> On Tue, Nov 27, 2018 at 5:56 PM Eric Badger <eb...@oath.com.invalid>
> wrote:
>
> > +1 (non-binding)
> >
> > - Verified all hashes and checksums
> > - Built from source on macOS 10.14.1, Java 1.8.0u65
> > - Deployed a pseudo cluster
> > - Ran some example jobs
> >
> > Eric
> >
> > On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:
> >
> > > Sunil, Thanks for driving this release!!!
> > >
> > > +1 (non-binding)
> > > -Built from source on Ubuntu 17.04 and JDK 8
> > > -Ran basic Hdfs Commands
> > > -Ran basic EC Commands
> > > -Ran basic RBF commands
> > > -Browsed HDFS and RBF WEB UI
> > > -Ran TeraGen/TeraSort
> > >
> > > Regards
> > > Ayush
> > >
> > > > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi
> <knanasi@cloudera.com.INVALID
> > >
> > > wrote:
> > > >
> > > > Thanks Sunil for the driving the release!
> > > >
> > > > +1 (non-binding)
> > > >
> > > > - checked out git tag release-3.2.0-RC0
> > > > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > > > - deployed on a 5 node cluster
> > > > - ran terasort, teragen, teravalidate with success
> > > > - executed basic hdfs, dfsadmin and ec commands
> > > >
> > > > Best,
> > > > Kitti
> > > >
> > > >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> > > >>
> > > >> Hi folks,
> > > >>
> > > >>
> > > >>
> > > >> Thanks to all contributors who helped in this release [1]. I have
> > > created
> > > >>
> > > >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> > > >>
> > > >>
> > > >> Artifacts for this RC are available here:
> > > >>
> > > >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > > >>
> > > >>
> > > >>
> > > >> RC tag in git is release-3.2.0-RC0.
> > > >>
> > > >>
> > > >>
> > > >> The maven artifacts are available via repository.apache.org at
> > > >>
> > > >>
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > > >>
> > > >>
> > > >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> > > PST.
> > > >>
> > > >>
> > > >>
> > > >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > > >> additions
> > > >>
> > > >> are the highlights of this release.
> > > >>
> > > >> 1. Node Attributes Support in YARN
> > > >>
> > > >> 2. Hadoop Submarine project for running Deep Learning workloads on
> > YARN
> > > >>
> > > >> 3. Support service upgrade via YARN Service API and CLI
> > > >>
> > > >> 4. HDFS Storage Policy Satisfier
> > > >>
> > > >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> > > >>
> > > >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > > >>
> > > >> 7. Improvements in Router-based HDFS federation
> > > >>
> > > >>
> > > >>
> > > >> Thanks to Wangda, Vinod, Marton for helping me in preparing the
> > release.
> > > >>
> > > >> I have done few testing with my pseudo cluster. My +1 to start.
> > > >>
> > > >>
> > > >>
> > > >> Regards,
> > > >>
> > > >> Sunil
> > > >>
> > > >>
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > > >>
> > > >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> > (3.2.0)
> > > >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
> > Resolved
> > > >> ORDER BY fixVersion ASC
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Gergely Pollak <sh...@cloudera.com.INVALID>.

Thanks for the work Sunil!

+1 (non-binding)

 - Checked out release-3.2.0-RC0
 - Built it on Debian 9.6 with Oracle Java 1.8.0_191-b12
 - Created a pseudo distributed cluser
 - Tested with tera(gen/sort/validate)
 - Also executed the pi job

Regards,
  Gergely Pollak


On Tue, Nov 27, 2018 at 6:37 PM Gabor Bota <ga...@cloudera.com.invalid>
wrote:

> Thanks for the work Sunil!
>
> +1 (non-binding)
>
> I've done the following:
>
>    - checked out git tag release-3.2.0-RC0
>    - built from source on Mac OS X 10.14.1, java: 8.0.181-oracle
>    - run hadoop-aws tests on AWS S3 against eu-west-1, with no unknown
>    issues
>    - deployed on a 3 node cluster
>    - verified example pi job
>    - ran teragen, terasort and teravalidate without any error
>
>
> Regards,
> Gabor Bota
>
> On Tue, Nov 27, 2018 at 5:56 PM Eric Badger <eb...@oath.com.invalid>
> wrote:
>
> > +1 (non-binding)
> >
> > - Verified all hashes and checksums
> > - Built from source on macOS 10.14.1, Java 1.8.0u65
> > - Deployed a pseudo cluster
> > - Ran some example jobs
> >
> > Eric
> >
> > On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:
> >
> > > Sunil, Thanks for driving this release!!!
> > >
> > > +1 (non-binding)
> > > -Built from source on Ubuntu 17.04 and JDK 8
> > > -Ran basic Hdfs Commands
> > > -Ran basic EC Commands
> > > -Ran basic RBF commands
> > > -Browsed HDFS and RBF WEB UI
> > > -Ran TeraGen/TeraSort
> > >
> > > Regards
> > > Ayush
> > >
> > > > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi
> <knanasi@cloudera.com.INVALID
> > >
> > > wrote:
> > > >
> > > > Thanks Sunil for the driving the release!
> > > >
> > > > +1 (non-binding)
> > > >
> > > > - checked out git tag release-3.2.0-RC0
> > > > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > > > - deployed on a 5 node cluster
> > > > - ran terasort, teragen, teravalidate with success
> > > > - executed basic hdfs, dfsadmin and ec commands
> > > >
> > > > Best,
> > > > Kitti
> > > >
> > > >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> > > >>
> > > >> Hi folks,
> > > >>
> > > >>
> > > >>
> > > >> Thanks to all contributors who helped in this release [1]. I have
> > > created
> > > >>
> > > >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> > > >>
> > > >>
> > > >> Artifacts for this RC are available here:
> > > >>
> > > >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > > >>
> > > >>
> > > >>
> > > >> RC tag in git is release-3.2.0-RC0.
> > > >>
> > > >>
> > > >>
> > > >> The maven artifacts are available via repository.apache.org at
> > > >>
> > > >>
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > > >>
> > > >>
> > > >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> > > PST.
> > > >>
> > > >>
> > > >>
> > > >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > > >> additions
> > > >>
> > > >> are the highlights of this release.
> > > >>
> > > >> 1. Node Attributes Support in YARN
> > > >>
> > > >> 2. Hadoop Submarine project for running Deep Learning workloads on
> > YARN
> > > >>
> > > >> 3. Support service upgrade via YARN Service API and CLI
> > > >>
> > > >> 4. HDFS Storage Policy Satisfier
> > > >>
> > > >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> > > >>
> > > >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > > >>
> > > >> 7. Improvements in Router-based HDFS federation
> > > >>
> > > >>
> > > >>
> > > >> Thanks to Wangda, Vinod, Marton for helping me in preparing the
> > release.
> > > >>
> > > >> I have done few testing with my pseudo cluster. My +1 to start.
> > > >>
> > > >>
> > > >>
> > > >> Regards,
> > > >>
> > > >> Sunil
> > > >>
> > > >>
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > > >>
> > > >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> > (3.2.0)
> > > >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
> > Resolved
> > > >> ORDER BY fixVersion ASC
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Gabor Bota <ga...@cloudera.com.INVALID>.

Thanks for the work Sunil!

+1 (non-binding)

I've done the following:

   - checked out git tag release-3.2.0-RC0
   - built from source on Mac OS X 10.14.1, java: 8.0.181-oracle
   - run hadoop-aws tests on AWS S3 against eu-west-1, with no unknown
   issues
   - deployed on a 3 node cluster
   - verified example pi job
   - ran teragen, terasort and teravalidate without any error


Regards,
Gabor Bota

On Tue, Nov 27, 2018 at 5:56 PM Eric Badger <eb...@oath.com.invalid>
wrote:

> +1 (non-binding)
>
> - Verified all hashes and checksums
> - Built from source on macOS 10.14.1, Java 1.8.0u65
> - Deployed a pseudo cluster
> - Ran some example jobs
>
> Eric
>
> On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:
>
> > Sunil, Thanks for driving this release!!!
> >
> > +1 (non-binding)
> > -Built from source on Ubuntu 17.04 and JDK 8
> > -Ran basic Hdfs Commands
> > -Ran basic EC Commands
> > -Ran basic RBF commands
> > -Browsed HDFS and RBF WEB UI
> > -Ran TeraGen/TeraSort
> >
> > Regards
> > Ayush
> >
> > > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi <knanasi@cloudera.com.INVALID
> >
> > wrote:
> > >
> > > Thanks Sunil for the driving the release!
> > >
> > > +1 (non-binding)
> > >
> > > - checked out git tag release-3.2.0-RC0
> > > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > > - deployed on a 5 node cluster
> > > - ran terasort, teragen, teravalidate with success
> > > - executed basic hdfs, dfsadmin and ec commands
> > >
> > > Best,
> > > Kitti
> > >
> > >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> > >>
> > >> Hi folks,
> > >>
> > >>
> > >>
> > >> Thanks to all contributors who helped in this release [1]. I have
> > created
> > >>
> > >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> > >>
> > >>
> > >> Artifacts for this RC are available here:
> > >>
> > >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > >>
> > >>
> > >>
> > >> RC tag in git is release-3.2.0-RC0.
> > >>
> > >>
> > >>
> > >> The maven artifacts are available via repository.apache.org at
> > >>
> > >>
> > https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > >>
> > >>
> > >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> > PST.
> > >>
> > >>
> > >>
> > >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > >> additions
> > >>
> > >> are the highlights of this release.
> > >>
> > >> 1. Node Attributes Support in YARN
> > >>
> > >> 2. Hadoop Submarine project for running Deep Learning workloads on
> YARN
> > >>
> > >> 3. Support service upgrade via YARN Service API and CLI
> > >>
> > >> 4. HDFS Storage Policy Satisfier
> > >>
> > >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> > >>
> > >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > >>
> > >> 7. Improvements in Router-based HDFS federation
> > >>
> > >>
> > >>
> > >> Thanks to Wangda, Vinod, Marton for helping me in preparing the
> release.
> > >>
> > >> I have done few testing with my pseudo cluster. My +1 to start.
> > >>
> > >>
> > >>
> > >> Regards,
> > >>
> > >> Sunil
> > >>
> > >>
> > >>
> > >> [1]
> > >>
> > >>
> > >>
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > >>
> > >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> (3.2.0)
> > >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
> Resolved
> > >> ORDER BY fixVersion ASC
> > >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Gabor Bota <ga...@cloudera.com.INVALID>.

Thanks for the work Sunil!

+1 (non-binding)

I've done the following:

   - checked out git tag release-3.2.0-RC0
   - built from source on Mac OS X 10.14.1, java: 8.0.181-oracle
   - run hadoop-aws tests on AWS S3 against eu-west-1, with no unknown
   issues
   - deployed on a 3 node cluster
   - verified example pi job
   - ran teragen, terasort and teravalidate without any error


Regards,
Gabor Bota

On Tue, Nov 27, 2018 at 5:56 PM Eric Badger <eb...@oath.com.invalid>
wrote:

> +1 (non-binding)
>
> - Verified all hashes and checksums
> - Built from source on macOS 10.14.1, Java 1.8.0u65
> - Deployed a pseudo cluster
> - Ran some example jobs
>
> Eric
>
> On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:
>
> > Sunil, Thanks for driving this release!!!
> >
> > +1 (non-binding)
> > -Built from source on Ubuntu 17.04 and JDK 8
> > -Ran basic Hdfs Commands
> > -Ran basic EC Commands
> > -Ran basic RBF commands
> > -Browsed HDFS and RBF WEB UI
> > -Ran TeraGen/TeraSort
> >
> > Regards
> > Ayush
> >
> > > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi <knanasi@cloudera.com.INVALID
> >
> > wrote:
> > >
> > > Thanks Sunil for the driving the release!
> > >
> > > +1 (non-binding)
> > >
> > > - checked out git tag release-3.2.0-RC0
> > > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > > - deployed on a 5 node cluster
> > > - ran terasort, teragen, teravalidate with success
> > > - executed basic hdfs, dfsadmin and ec commands
> > >
> > > Best,
> > > Kitti
> > >
> > >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> > >>
> > >> Hi folks,
> > >>
> > >>
> > >>
> > >> Thanks to all contributors who helped in this release [1]. I have
> > created
> > >>
> > >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> > >>
> > >>
> > >> Artifacts for this RC are available here:
> > >>
> > >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> > >>
> > >>
> > >>
> > >> RC tag in git is release-3.2.0-RC0.
> > >>
> > >>
> > >>
> > >> The maven artifacts are available via repository.apache.org at
> > >>
> > >>
> > https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> > >>
> > >>
> > >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> > PST.
> > >>
> > >>
> > >>
> > >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > >> additions
> > >>
> > >> are the highlights of this release.
> > >>
> > >> 1. Node Attributes Support in YARN
> > >>
> > >> 2. Hadoop Submarine project for running Deep Learning workloads on
> YARN
> > >>
> > >> 3. Support service upgrade via YARN Service API and CLI
> > >>
> > >> 4. HDFS Storage Policy Satisfier
> > >>
> > >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> > >>
> > >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> > >>
> > >> 7. Improvements in Router-based HDFS federation
> > >>
> > >>
> > >>
> > >> Thanks to Wangda, Vinod, Marton for helping me in preparing the
> release.
> > >>
> > >> I have done few testing with my pseudo cluster. My +1 to start.
> > >>
> > >>
> > >>
> > >> Regards,
> > >>
> > >> Sunil
> > >>
> > >>
> > >>
> > >> [1]
> > >>
> > >>
> > >>
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> > >>
> > >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> (3.2.0)
> > >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
> Resolved
> > >> ORDER BY fixVersion ASC
> > >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
> >
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Badger <eb...@oath.com.INVALID>.

+1 (non-binding)

- Verified all hashes and checksums
- Built from source on macOS 10.14.1, Java 1.8.0u65
- Deployed a pseudo cluster
- Ran some example jobs

Eric

On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:

> Sunil, Thanks for driving this release!!!
>
> +1 (non-binding)
> -Built from source on Ubuntu 17.04 and JDK 8
> -Ran basic Hdfs Commands
> -Ran basic EC Commands
> -Ran basic RBF commands
> -Browsed HDFS and RBF WEB UI
> -Ran TeraGen/TeraSort
>
> Regards
> Ayush
>
> > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi <kn...@cloudera.com.INVALID>
> wrote:
> >
> > Thanks Sunil for the driving the release!
> >
> > +1 (non-binding)
> >
> > - checked out git tag release-3.2.0-RC0
> > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > - deployed on a 5 node cluster
> > - ran terasort, teragen, teravalidate with success
> > - executed basic hdfs, dfsadmin and ec commands
> >
> > Best,
> > Kitti
> >
> >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> >>
> >> Hi folks,
> >>
> >>
> >>
> >> Thanks to all contributors who helped in this release [1]. I have
> created
> >>
> >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> >>
> >>
> >> Artifacts for this RC are available here:
> >>
> >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >>
> >>
> >>
> >> RC tag in git is release-3.2.0-RC0.
> >>
> >>
> >>
> >> The maven artifacts are available via repository.apache.org at
> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >>
> >>
> >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> PST.
> >>
> >>
> >>
> >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> >> additions
> >>
> >> are the highlights of this release.
> >>
> >> 1. Node Attributes Support in YARN
> >>
> >> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >>
> >> 3. Support service upgrade via YARN Service API and CLI
> >>
> >> 4. HDFS Storage Policy Satisfier
> >>
> >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> >>
> >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >>
> >> 7. Improvements in Router-based HDFS federation
> >>
> >>
> >>
> >> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >>
> >> I have done few testing with my pseudo cluster. My +1 to start.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Sunil
> >>
> >>
> >>
> >> [1]
> >>
> >>
> >>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >>
> >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> >> ORDER BY fixVersion ASC
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Badger <eb...@oath.com.INVALID>.

+1 (non-binding)

- Verified all hashes and checksums
- Built from source on macOS 10.14.1, Java 1.8.0u65
- Deployed a pseudo cluster
- Ran some example jobs

Eric

On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:

> Sunil, Thanks for driving this release!!!
>
> +1 (non-binding)
> -Built from source on Ubuntu 17.04 and JDK 8
> -Ran basic Hdfs Commands
> -Ran basic EC Commands
> -Ran basic RBF commands
> -Browsed HDFS and RBF WEB UI
> -Ran TeraGen/TeraSort
>
> Regards
> Ayush
>
> > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi <kn...@cloudera.com.INVALID>
> wrote:
> >
> > Thanks Sunil for the driving the release!
> >
> > +1 (non-binding)
> >
> > - checked out git tag release-3.2.0-RC0
> > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > - deployed on a 5 node cluster
> > - ran terasort, teragen, teravalidate with success
> > - executed basic hdfs, dfsadmin and ec commands
> >
> > Best,
> > Kitti
> >
> >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> >>
> >> Hi folks,
> >>
> >>
> >>
> >> Thanks to all contributors who helped in this release [1]. I have
> created
> >>
> >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> >>
> >>
> >> Artifacts for this RC are available here:
> >>
> >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >>
> >>
> >>
> >> RC tag in git is release-3.2.0-RC0.
> >>
> >>
> >>
> >> The maven artifacts are available via repository.apache.org at
> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >>
> >>
> >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> PST.
> >>
> >>
> >>
> >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> >> additions
> >>
> >> are the highlights of this release.
> >>
> >> 1. Node Attributes Support in YARN
> >>
> >> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >>
> >> 3. Support service upgrade via YARN Service API and CLI
> >>
> >> 4. HDFS Storage Policy Satisfier
> >>
> >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> >>
> >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >>
> >> 7. Improvements in Router-based HDFS federation
> >>
> >>
> >>
> >> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >>
> >> I have done few testing with my pseudo cluster. My +1 to start.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Sunil
> >>
> >>
> >>
> >> [1]
> >>
> >>
> >>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >>
> >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> >> ORDER BY fixVersion ASC
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Badger <eb...@oath.com.INVALID>.

+1 (non-binding)

- Verified all hashes and checksums
- Built from source on macOS 10.14.1, Java 1.8.0u65
- Deployed a pseudo cluster
- Ran some example jobs

Eric

On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:

> Sunil, Thanks for driving this release!!!
>
> +1 (non-binding)
> -Built from source on Ubuntu 17.04 and JDK 8
> -Ran basic Hdfs Commands
> -Ran basic EC Commands
> -Ran basic RBF commands
> -Browsed HDFS and RBF WEB UI
> -Ran TeraGen/TeraSort
>
> Regards
> Ayush
>
> > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi <kn...@cloudera.com.INVALID>
> wrote:
> >
> > Thanks Sunil for the driving the release!
> >
> > +1 (non-binding)
> >
> > - checked out git tag release-3.2.0-RC0
> > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > - deployed on a 5 node cluster
> > - ran terasort, teragen, teravalidate with success
> > - executed basic hdfs, dfsadmin and ec commands
> >
> > Best,
> > Kitti
> >
> >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> >>
> >> Hi folks,
> >>
> >>
> >>
> >> Thanks to all contributors who helped in this release [1]. I have
> created
> >>
> >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> >>
> >>
> >> Artifacts for this RC are available here:
> >>
> >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >>
> >>
> >>
> >> RC tag in git is release-3.2.0-RC0.
> >>
> >>
> >>
> >> The maven artifacts are available via repository.apache.org at
> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >>
> >>
> >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> PST.
> >>
> >>
> >>
> >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> >> additions
> >>
> >> are the highlights of this release.
> >>
> >> 1. Node Attributes Support in YARN
> >>
> >> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >>
> >> 3. Support service upgrade via YARN Service API and CLI
> >>
> >> 4. HDFS Storage Policy Satisfier
> >>
> >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> >>
> >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >>
> >> 7. Improvements in Router-based HDFS federation
> >>
> >>
> >>
> >> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >>
> >> I have done few testing with my pseudo cluster. My +1 to start.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Sunil
> >>
> >>
> >>
> >> [1]
> >>
> >>
> >>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >>
> >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> >> ORDER BY fixVersion ASC
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Badger <eb...@oath.com.INVALID>.

+1 (non-binding)

- Verified all hashes and checksums
- Built from source on macOS 10.14.1, Java 1.8.0u65
- Deployed a pseudo cluster
- Ran some example jobs

Eric

On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena <ay...@gmail.com> wrote:

> Sunil, Thanks for driving this release!!!
>
> +1 (non-binding)
> -Built from source on Ubuntu 17.04 and JDK 8
> -Ran basic Hdfs Commands
> -Ran basic EC Commands
> -Ran basic RBF commands
> -Browsed HDFS and RBF WEB UI
> -Ran TeraGen/TeraSort
>
> Regards
> Ayush
>
> > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi <kn...@cloudera.com.INVALID>
> wrote:
> >
> > Thanks Sunil for the driving the release!
> >
> > +1 (non-binding)
> >
> > - checked out git tag release-3.2.0-RC0
> > - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> > - deployed on a 5 node cluster
> > - ran terasort, teragen, teravalidate with success
> > - executed basic hdfs, dfsadmin and ec commands
> >
> > Best,
> > Kitti
> >
> >> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
> >>
> >> Hi folks,
> >>
> >>
> >>
> >> Thanks to all contributors who helped in this release [1]. I have
> created
> >>
> >> first release candidate (RC0) for Apache Hadoop 3.2.0.
> >>
> >>
> >> Artifacts for this RC are available here:
> >>
> >> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >>
> >>
> >>
> >> RC tag in git is release-3.2.0-RC0.
> >>
> >>
> >>
> >> The maven artifacts are available via repository.apache.org at
> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >>
> >>
> >> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm
> PST.
> >>
> >>
> >>
> >> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> >> additions
> >>
> >> are the highlights of this release.
> >>
> >> 1. Node Attributes Support in YARN
> >>
> >> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >>
> >> 3. Support service upgrade via YARN Service API and CLI
> >>
> >> 4. HDFS Storage Policy Satisfier
> >>
> >> 5. Support Windows Azure Storage - Blob file system in Hadoop
> >>
> >> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >>
> >> 7. Improvements in Router-based HDFS federation
> >>
> >>
> >>
> >> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >>
> >> I have done few testing with my pseudo cluster. My +1 to start.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Sunil
> >>
> >>
> >>
> >> [1]
> >>
> >>
> >>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >>
> >> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> >> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> >> ORDER BY fixVersion ASC
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Ayush Saxena <ay...@gmail.com>.

Sunil, Thanks for driving this release!!!

+1 (non-binding)
-Built from source on Ubuntu 17.04 and JDK 8
-Ran basic Hdfs Commands
-Ran basic EC Commands
-Ran basic RBF commands
-Browsed HDFS and RBF WEB UI
-Ran TeraGen/TeraSort

Regards
Ayush

> On 27-Nov-2018, at 4:06 PM, Kitti Nanasi <kn...@cloudera.com.INVALID> wrote:
> 
> Thanks Sunil for the driving the release!
> 
> +1 (non-binding)
> 
> - checked out git tag release-3.2.0-RC0
> - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> - deployed on a 5 node cluster
> - ran terasort, teragen, teravalidate with success
> - executed basic hdfs, dfsadmin and ec commands
> 
> Best,
> Kitti
> 
>> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
>> 
>> Hi folks,
>> 
>> 
>> 
>> Thanks to all contributors who helped in this release [1]. I have created
>> 
>> first release candidate (RC0) for Apache Hadoop 3.2.0.
>> 
>> 
>> Artifacts for this RC are available here:
>> 
>> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>> 
>> 
>> 
>> RC tag in git is release-3.2.0-RC0.
>> 
>> 
>> 
>> The maven artifacts are available via repository.apache.org at
>> 
>> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>> 
>> 
>> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>> 
>> 
>> 
>> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
>> additions
>> 
>> are the highlights of this release.
>> 
>> 1. Node Attributes Support in YARN
>> 
>> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>> 
>> 3. Support service upgrade via YARN Service API and CLI
>> 
>> 4. HDFS Storage Policy Satisfier
>> 
>> 5. Support Windows Azure Storage - Blob file system in Hadoop
>> 
>> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>> 
>> 7. Improvements in Router-based HDFS federation
>> 
>> 
>> 
>> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>> 
>> I have done few testing with my pseudo cluster. My +1 to start.
>> 
>> 
>> 
>> Regards,
>> 
>> Sunil
>> 
>> 
>> 
>> [1]
>> 
>> 
>> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>> 
>> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
>> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
>> ORDER BY fixVersion ASC
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Ayush Saxena <ay...@gmail.com>.

Sunil, Thanks for driving this release!!!

+1 (non-binding)
-Built from source on Ubuntu 17.04 and JDK 8
-Ran basic Hdfs Commands
-Ran basic EC Commands
-Ran basic RBF commands
-Browsed HDFS and RBF WEB UI
-Ran TeraGen/TeraSort

Regards
Ayush

> On 27-Nov-2018, at 4:06 PM, Kitti Nanasi <kn...@cloudera.com.INVALID> wrote:
> 
> Thanks Sunil for the driving the release!
> 
> +1 (non-binding)
> 
> - checked out git tag release-3.2.0-RC0
> - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
> - deployed on a 5 node cluster
> - ran terasort, teragen, teravalidate with success
> - executed basic hdfs, dfsadmin and ec commands
> 
> Best,
> Kitti
> 
>> On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:
>> 
>> Hi folks,
>> 
>> 
>> 
>> Thanks to all contributors who helped in this release [1]. I have created
>> 
>> first release candidate (RC0) for Apache Hadoop 3.2.0.
>> 
>> 
>> Artifacts for this RC are available here:
>> 
>> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>> 
>> 
>> 
>> RC tag in git is release-3.2.0-RC0.
>> 
>> 
>> 
>> The maven artifacts are available via repository.apache.org at
>> 
>> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>> 
>> 
>> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>> 
>> 
>> 
>> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
>> additions
>> 
>> are the highlights of this release.
>> 
>> 1. Node Attributes Support in YARN
>> 
>> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>> 
>> 3. Support service upgrade via YARN Service API and CLI
>> 
>> 4. HDFS Storage Policy Satisfier
>> 
>> 5. Support Windows Azure Storage - Blob file system in Hadoop
>> 
>> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>> 
>> 7. Improvements in Router-based HDFS federation
>> 
>> 
>> 
>> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>> 
>> I have done few testing with my pseudo cluster. My +1 to start.
>> 
>> 
>> 
>> Regards,
>> 
>> Sunil
>> 
>> 
>> 
>> [1]
>> 
>> 
>> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>> 
>> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
>> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
>> ORDER BY fixVersion ASC
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Kitti Nanasi <kn...@cloudera.com.INVALID>.

Thanks Sunil for the driving the release!

+1 (non-binding)

- checked out git tag release-3.2.0-RC0
- built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
- deployed on a 5 node cluster
- ran terasort, teragen, teravalidate with success
- executed basic hdfs, dfsadmin and ec commands

Best,
Kitti

On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:

> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Jason Lowe <jl...@apache.org>.

Thanks for driving this release, Sunil!

+1 (binding)

- Verified signatures and digests
- Successfully performed a native build
- Deployed a single-node cluster
- Ran some sample jobs

Jason

On Fri, Nov 23, 2018 at 6:07 AM Sunil G <su...@apache.org> wrote:

> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Takanobu Asanuma <ta...@yahoo-corp.jp>.

Thanks for driving the release, Sunil!

Regards,
-Takanobu

on 2018/11/23 21:07, "Sunil G"<su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Akira Ajisaka <aa...@apache.org>.

Thanks Sunil for driving this release.

+1 (binding)

- verified signatures and checksums
- checked the documents including changes and release notes
- built from source on CentOS 7.5, Java 1.8.0_191
- ran some MapReduce jobs in pseudo-distributed cluster
- checked some web UIs (NN, DN, RM, NM)

Thanks,
Akira
2018年11月23日(金) 21:07 Sunil G <su...@apache.org>:
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Akira Ajisaka <aa...@apache.org>.

Thanks Sunil for driving this release.

+1 (binding)

- verified signatures and checksums
- checked the documents including changes and release notes
- built from source on CentOS 7.5, Java 1.8.0_191
- ran some MapReduce jobs in pseudo-distributed cluster
- checked some web UIs (NN, DN, RM, NM)

Thanks,
Akira
2018年11月23日(金) 21:07 Sunil G <su...@apache.org>:
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Kitti Nanasi <kn...@cloudera.com.INVALID>.

Thanks Sunil for the driving the release!

+1 (non-binding)

- checked out git tag release-3.2.0-RC0
- built from source on Mac OS X 10.13.4, java version 8.0.172-zulu
- deployed on a 5 node cluster
- ran terasort, teragen, teravalidate with success
- executed basic hdfs, dfsadmin and ec commands

Best,
Kitti

On Fri, Nov 23, 2018 at 1:07 PM Sunil G <su...@apache.org> wrote:

> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Wilfred Spiegelenburg <ws...@cloudera.com.INVALID>.

+1 (non binding)

- build from source on MacOSX 10.14.1, 1.8.0u181
- successful native build on Ubuntu 16.04.3
- confirmed the checksum and signature
- deployed a single node cluster  (jdk 1.8u191 / centos 7.5)

Wilfred

> On 23 Nov 2018, at 23:06, Sunil G <su...@apache.org> wrote:
> 
> Hi folks,
> 
> 
> 
> Thanks to all contributors who helped in this release [1]. I have created
> 
> first release candidate (RC0) for Apache Hadoop 3.2.0.
> 
> 
> Artifacts for this RC are available here:
> 
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> 
> 
> 
> RC tag in git is release-3.2.0-RC0.
> 
> 
> 
> The maven artifacts are available via repository.apache.org at
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> 
> 
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> 
> 
> 
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
> 
> are the highlights of this release.
> 
> 1. Node Attributes Support in YARN
> 
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> 
> 3. Support service upgrade via YARN Service API and CLI
> 
> 4. HDFS Storage Policy Satisfier
> 
> 5. Support Windows Azure Storage - Blob file system in Hadoop
> 
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> 
> 7. Improvements in Router-based HDFS federation
> 
> 
> 
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> 
> I have done few testing with my pseudo cluster. My +1 to start.
> 
> 
> 
> Regards,
> 
> Sunil
> 
> 
> 
> [1]
> 
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> 
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC


Wilfred Spiegelenburg | Software Engineer
cloudera.com <https://www.cloudera.com/>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Akira Ajisaka <aa...@apache.org>.

Thanks Sunil for driving this release.

+1 (binding)

- verified signatures and checksums
- checked the documents including changes and release notes
- built from source on CentOS 7.5, Java 1.8.0_191
- ran some MapReduce jobs in pseudo-distributed cluster
- checked some web UIs (NN, DN, RM, NM)

Thanks,
Akira
2018年11月23日(金) 21:07 Sunil G <su...@apache.org>:
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Eric Payne <er...@yahoo.com.INVALID>.

Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric

On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G <su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Wilfred Spiegelenburg <ws...@cloudera.com.INVALID>.

+1 (non binding)

- build from source on MacOSX 10.14.1, 1.8.0u181
- successful native build on Ubuntu 16.04.3
- confirmed the checksum and signature
- deployed a single node cluster  (jdk 1.8u191 / centos 7.5)

Wilfred

> On 23 Nov 2018, at 23:06, Sunil G <su...@apache.org> wrote:
> 
> Hi folks,
> 
> 
> 
> Thanks to all contributors who helped in this release [1]. I have created
> 
> first release candidate (RC0) for Apache Hadoop 3.2.0.
> 
> 
> Artifacts for this RC are available here:
> 
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> 
> 
> 
> RC tag in git is release-3.2.0-RC0.
> 
> 
> 
> The maven artifacts are available via repository.apache.org at
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> 
> 
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> 
> 
> 
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
> 
> are the highlights of this release.
> 
> 1. Node Attributes Support in YARN
> 
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> 
> 3. Support service upgrade via YARN Service API and CLI
> 
> 4. HDFS Storage Policy Satisfier
> 
> 5. Support Windows Azure Storage - Blob file system in Hadoop
> 
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> 
> 7. Improvements in Router-based HDFS federation
> 
> 
> 
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> 
> I have done few testing with my pseudo cluster. My +1 to start.
> 
> 
> 
> Regards,
> 
> Sunil
> 
> 
> 
> [1]
> 
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> 
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC


Wilfred Spiegelenburg | Software Engineer
cloudera.com <https://www.cloudera.com/>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Takanobu Asanuma <ta...@yahoo-corp.jp>.

Thanks for driving the release, Sunil!

Regards,
-Takanobu

on 2018/11/23 21:07, "Sunil G"<su...@apache.org> wrote:

Hi folks,

Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.

Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/

RC tag in git is release-3.2.0-RC0.

The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/

This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.

3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation

Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.

Regards,

Sunil

[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Wilfred Spiegelenburg <ws...@cloudera.com.INVALID>.

+1 (non binding)

- build from source on MacOSX 10.14.1, 1.8.0u181
- successful native build on Ubuntu 16.04.3
- confirmed the checksum and signature
- deployed a single node cluster  (jdk 1.8u191 / centos 7.5)

Wilfred

> On 23 Nov 2018, at 23:06, Sunil G <su...@apache.org> wrote:
> 
> Hi folks,
> 
> 
> 
> Thanks to all contributors who helped in this release [1]. I have created
> 
> first release candidate (RC0) for Apache Hadoop 3.2.0.
> 
> 
> Artifacts for this RC are available here:
> 
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> 
> 
> 
> RC tag in git is release-3.2.0-RC0.
> 
> 
> 
> The maven artifacts are available via repository.apache.org at
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> 
> 
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> 
> 
> 
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
> 
> are the highlights of this release.
> 
> 1. Node Attributes Support in YARN
> 
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> 
> 3. Support service upgrade via YARN Service API and CLI
> 
> 4. HDFS Storage Policy Satisfier
> 
> 5. Support Windows Azure Storage - Blob file system in Hadoop
> 
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> 
> 7. Improvements in Router-based HDFS federation
> 
> 
> 
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> 
> I have done few testing with my pseudo cluster. My +1 to start.
> 
> 
> 
> Regards,
> 
> Sunil
> 
> 
> 
> [1]
> 
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> 
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC


Wilfred Spiegelenburg | Software Engineer
cloudera.com <https://www.cloudera.com/>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Posted by Akira Ajisaka <aa...@apache.org>.

Thanks Sunil for driving this release.

+1 (binding)

- verified signatures and checksums
- checked the documents including changes and release notes
- built from source on CentOS 7.5, Java 1.8.0_191
- ran some MapReduce jobs in pseudo-distributed cluster
- checked some web UIs (NN, DN, RM, NM)

Thanks,
Akira
2018年11月23日(金) 21:07 Sunil G <su...@apache.org>:
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org