[GitHub] [hadoop] elek opened a new pull request #1256: HDDS-1937. Acceptance tests fail if scm webui shows invalid json

[GitBox <gi...@apache.org>]

elek opened a new pull request #1256: HDDS-1937. Acceptance tests fail if scm webui shows invalid json
URL: https://github.com/apache/hadoop/pull/1256
 
 
   Acceptance test of a nightly build is failed with the following error:
   
   {code}
   Creating ozonesecure_datanode_3 ... 
   
   Creating ozonesecure_kdc_1      ... done
   
   Creating ozonesecure_om_1       ... done
   
   Creating ozonesecure_scm_1      ... done
   
   Creating ozonesecure_datanode_3 ... done
   
   Creating ozonesecure_kms_1      ... done
   
   Creating ozonesecure_s3g_1      ... done
   
   Creating ozonesecure_datanode_2 ... done
   
   Creating ozonesecure_datanode_1 ... done
   parse error: Invalid numeric literal at line 2, column 0
   {code}
   
   https://raw.githubusercontent.com/elek/ozone-ci/master/byscane/byscane-nightly-5b87q/acceptance/output.log
   
   The problem is in the script which checks the number of available datanodes.
   
   If the HTTP endpoint of the SCM is already started BUT not ready yet it may return with a simple HTML error message instead of json. Which can not be parsed by jq:
   
   In testlib.sh:
   
   {code}
     37   │   if [[ "${SECURITY_ENABLED}" == 'true' ]]; then
     38   │     docker-compose -f "${compose_file}" exec -T scm bash -c "kinit -k HTTP/scm@EXAMPL
          │ E.COM -t /etc/security/keytabs/HTTP.keytab && curl --negotiate -u : -s '${jmx_url}'"
     39   │   else
     40   │     docker-compose -f "${compose_file}" exec -T scm curl -s "${jmx_url}"
     41   │   fi \
     42   │     | jq -r '.beans[0].NodeCount[] | select(.key=="HEALTHY") | .value'
   {code}
   
   One possible fix is to adjust the error handling (set +x / set -x) per method instead of using a generic set -x at the beginning. It would provide a more predictable behavior. In our case count_datanode should not fail evert (as the caller method: wait_for_datanodes can retry anyway).
   
   See: https://issues.apache.org/jira/browse/HDDS-1937

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org