Re: (Updated-7) Stabilizing our automated builds - help still needed!

[Joan Touzet <wo...@apache.org>]

Hi everyone,

Now that the initial Debian/Ubuntu packaging is done, my attention turns
to our test suite full time. I won't automate building of packages until
we have a reliable test suite.
 
# TL;DR
* 3 new failures: 1 JS, 2 EUnit.
* Total issues open: *18*
* Recurrences of 3 previously reported bugs.
* Lack of help on these issues continues to impact our ability to
  automatically roll nightly builds/packages. :(

# New Issues
COUCHDB-3402: JS: dev/run timing out starting up nodes
  I am fairly certain this is because the ASF Jenkins workers are
  heavily loaded, and we need to be more generous with dev/run's
  timeouts. However, I'd like to know for sure, and we have issues
  with Travis runs as well where the answers are buried in logfiles
  we don't have access to after the run fails. So there will be some
  groundwork done here to start saving logs out for us to analyze after
  the fact for both Travis (using after_failure) and Jenkins (using
  the build script) to a common destination (possibly itself
  Couch-flavoured.)

COUCHDB-3401: EUnit: should_not_remember_docs_in_index_after_backup_restore 500 error
  We're failing an assert on an http status code that should be 200 but
  instead returned 500. Logs would be nice. One sighting of this in the
  wild so far.

COUCHDB-3396: EUnit: failed_to_start_child
  Multiple failures in chttpd_db_doc_size_tests across multiple platforms.

An unrelated issue, COUCHDB-3398, has been added to add a new EUnit
test to the test suite.

# Continuing Open Issues (qty. 15)
COUCHDB-3384: EUnit: couch_replicator_compact_tests failure
COUCHDB-3383: EUnit: couchdb_file_compression_tests timed out
COUCHDB-3382: EUnit: couchdb_auth_tests assertEqual failure
COUCHDB-3360: Broken couch_mrview test modules
COUCHDB-3356: JS: jsonp test fail
COUCHDB-3354: JS: replicator_db_compact_rep_db failed
COUCHDB-3352: JS: couchjs SIGSEGVs
COUCHDB-3348: EUnit: global_changes_tests context setup failed
COUCHDB-3346: JS: reduce.js "JSON is not a function"
COUCHDB-3345: JS: stats.js silent failure
COUCHDB-3344: EUnit: compaction_daemon_tests timing out
COUCHDB-3343: JS: show_documents failure
COUCHDB-3342: JS: design_docs expected '"ok"', got '"ko"'
COUCHDB-3341: EUnit: config listener unknown failure
COUCHDB-3339: EUnit: couch_mrview_red_views_tests throw:error
 
# Full list of test case issues in JIRA
 
    https://issues.apache.org/jira/issues/?filter=12340503

-Joan