You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@eagle.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/09/23 03:39:20 UTC
[jira] [Commented] (EAGLE-565) Because of the "type" field when has subqueue ,response of RM REST API doesn‘t match the SchedulerWrapper object
[ https://issues.apache.org/jira/browse/EAGLE-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515284#comment-15515284 ]
ASF GitHub Bot commented on EAGLE-565:
--------------------------------------
GitHub user chitin opened a pull request:
https://github.com/apache/incubator-eagle/pull/450
EAGLE-565 Because of the "type" field when has subqueue ,response of RM REST API doesn‘t match the SchedulerWrapper object
EAGLE-565 Because of the "type" field when has subqueue ,response of RM REST API doesn‘t match the SchedulerWrapper object
- add function workaroundYARN4785() to process YARN bug
https://issues.apache.org/jira/browse/EAGLE-565
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chitin/incubator-eagle eagle565
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-eagle/pull/450.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #450
----
commit 3070fe4c7e9924416a8f5af509a35c397e6038b4
Author: chitin <ch...@gmail.com>
Date: 2016-09-23T03:30:55Z
EAGLE-565 Because of the "type" field when has subqueue ,response of RM REST API doesnt match the SchedulerWrapper object
- add function workaroundYARN4785() to process YARN bug
https://issues.apache.org/jira/browse/EAGLE-565
----
> Because of the "type" field when has subqueue ,response of RM REST API doesn‘t match the SchedulerWrapper object
> ----------------------------------------------------------------------------------------------------------------
>
> Key: EAGLE-565
> URL: https://issues.apache.org/jira/browse/EAGLE-565
> Project: Eagle
> Issue Type: Bug
> Affects Versions: v0.4.0
> Environment: hadoop2.6.0-cdh5.4.3
> Reporter: Lingang Deng
> Assignee: Lingang Deng
> Priority: Critical
>
> I got the exception {code:java}Caused by: org.codehaus.jackson.map.JsonMappingException: Can not deserialize instance of java.lang.String out of START_ARRAY token
> at [Source: java.io.ByteArrayInputStream@1578809b; line: 1, column: 2350] (through reference chain: org.apache.eagle.hadoop.queue.model.scheduler.SchedulerWrapper["scheduler"]->org.apache.eagle.hadoop.queue.model.scheduler.Scheduler["schedulerInfo"]->org.apache.eagle.hadoop.queue.model.scheduler.SchedulerInfo["queues"]->org.apache.eagle.hadoop.queue.model.scheduler.Queues["queue"]->org.apache.eagle.hadoop.queue.model.scheduler.Queue["queues"]->org.apache.eagle.hadoop.queue.model.scheduler.Queues["queue"]->org.apache.eagle.hadoop.queue.model.scheduler.Queue["type"])
> {code} The response of https://some.server.address:50030/ws/v1/cluster/scheduler?anonymous=true, the "type" field maybe return Array, but the right is String, and the field is declared as String in class Queue. Then, I got the root cause, there is a YARN JIRA, [YARN-4785|https://issues.apache.org/jira/browse/YARN-4785].Let me give a case:
> {code:xml}
> {
> "scheduler": {
> "schedulerInfo": {
> "type": "capacityScheduler",
> "capacity": 100,
> "usedCapacity": 0,
> "maxCapacity": 100,
> "queueName": "root",
> "queues": {
> "queue": [
> {
> "type": "capacitySchedulerLeafQueueInfo",
> "capacity": 35,
> "usedCapacity": 0,
> "maxCapacity": 100,
> "absoluteCapacity": 35,
> "absoluteMaxCapacity": 100,
> "absoluteUsedCapacity": 0,
> "numApplications": 0,
> "queueName": "A",
> "state": "RUNNING",
> "resourcesUsed": {
> "memory": 0,
> "vCores": 0
> },
> "hideReservationQueues": false,
> "nodeLabels": [
> "*"
> ],
> "numActiveApplications": 0,
> "numPendingApplications": 0,
> "numContainers": 0,
> "maxApplications": 20000,
> "maxApplicationsPerUser": 40000,
> "maxActiveApplications": 20,
> "maxActiveApplicationsPerUser": 14,
> "userLimit": 100,
> "users": null,
> "userLimitFactor": 2
> },
> {
> "capacity": 35,
> "usedCapacity": 0,
> "maxCapacity": 35,
> "absoluteCapacity": 35,
> "absoluteMaxCapacity": 35,
> "absoluteUsedCapacity": 0,
> "numApplications": 0,
> "queueName": "B",
> "state": "RUNNING",
> "queues": {
> "queue": [
> {
> "capacity": 37,
> "usedCapacity": 0,
> "maxCapacity": 100,
> "absoluteCapacity": 12.95,
> "absoluteMaxCapacity": 35,
> "absoluteUsedCapacity": 0,
> "numApplications": 0,
> "queueName": "B1",
> "state": "RUNNING",
> "queues": {
> "queue": [
> {
> "type": "capacitySchedulerLeafQueueInfo",
> "capacity": 45,
> "usedCapacity": 0,
> "maxCapacity": 100,
> "absoluteCapacity": 5.8275,
> "absoluteMaxCapacity": 35,
> "absoluteUsedCapacity": 0,
> "numApplications": 0,
> "queueName": "B11",
> "state": "RUNNING",
> "resourcesUsed": {
> "memory": 0,
> "vCores": 0
> },
> "hideReservationQueues": false,
> "nodeLabels": [
> "*"
> ],
> "numActiveApplications": 0,
> "numPendingApplications": 0,
> "numContainers": 0,
> "maxApplications": 20000,
> "maxApplicationsPerUser": 200000,
> "maxActiveApplications": 7,
> "maxActiveApplicationsPerUser": 20,
> "userLimit": 100,
> "users": null,
> "userLimitFactor": 10
> },
> {
> "type": "capacitySchedulerLeafQueueInfo",
> "capacity": 55,
> "usedCapacity": 0,
> "maxCapacity": 100,
> "absoluteCapacity": 7.1225004,
> "absoluteMaxCapacity": 35,
> "absoluteUsedCapacity": 0,
> "numApplications": 0,
> "queueName": "B12",
> "state": "RUNNING",
> "resourcesUsed": {
> "memory": 0,
> "vCores": 0
> },
> "hideReservationQueues": false,
> "nodeLabels": [
> "*"
> ],
> "numActiveApplications": 0,
> "numPendingApplications": 0,
> "numContainers": 0,
> "maxApplications": 712,
> "maxApplicationsPerUser": 712,
> "maxActiveApplications": 2,
> "maxActiveApplicationsPerUser": 1,
> "userLimit": 100,
> "users": null,
> "userLimitFactor": 1
> }
> ]
> },
> "resourcesUsed": {
> "memory": 0,
> "vCores": 0
> },
> "hideReservationQueues": false,
> "nodeLabels": [
> "*"
> ]
> },
> {
> "type": [
> "capacitySchedulerLeafQueueInfo"
> ],
> "capacity": 63,
> "usedCapacity": 0,
> "maxCapacity": 100,
> "absoluteCapacity": 22.05,
> "absoluteMaxCapacity": 35,
> "absoluteUsedCapacity": 0,
> "numApplications": 0,
> "queueName": "B2",
> "state": "RUNNING",
> "resourcesUsed": {
> "memory": 0,
> "vCores": 0
> },
> "hideReservationQueues": false,
> "nodeLabels": [
> "*"
> ],
> "numActiveApplications": 0,
> "numPendingApplications": 0,
> "numContainers": 0,
> "maxApplications": 20000,
> "maxApplicationsPerUser": 30000,
> "maxActiveApplications": 7,
> "maxActiveApplicationsPerUser": 8,
> "userLimit": 100,
> "users": null,
> "userLimitFactor": 1.5
> }
> ]
> },
> "resourcesUsed": {
> "memory": 0,
> "vCores": 0
> },
> "hideReservationQueues": false,
> "nodeLabels": [
> "*"
> ]
> }
> ]
> }
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)