You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by "e-strauss (via GitHub)" <gi...@apache.org> on 2025/01/29 10:56:09 UTC
[PR] [SYSTEMDS-3823] Compression test case for bultin kmeans [systemds]
e-strauss opened a new pull request, #2194:
URL: https://github.com/apache/systemds/pull/2194
This fix adds a new test case for the builtin kmeans with compression enabled. It also adds a missing decompression handling for indexed MatrixBlock copy, which previously broke the compressed kmeans test case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SYSTEMDS-3823] Compression test case for bultin kmeans [systemds]
Posted by "Baunsgaard (via GitHub)" <gi...@apache.org>.
Baunsgaard commented on code in PR #2194:
URL: https://github.com/apache/systemds/pull/2194#discussion_r1933712684
##########
src/main/java/org/apache/sysds/runtime/matrix/data/MatrixBlock.java:
##########
@@ -1730,6 +1730,9 @@ public void putInto(MatrixBlock target, int rowOffset, int colOffset, boolean sp
* (the invoker is responsible to recompute nnz after all copies are done)
*/
public void copy(int rl, int ru, int cl, int cu, MatrixBlock src, boolean awareDestNZ ) {
+ if (src instanceof CompressedMatrixBlock){
+ src = ((CompressedMatrixBlock) src).decompress();
Review Comment:
add a string to the decompress command to indicate what is causing the decompression.
##########
src/test/java/org/apache/sysds/test/functions/compress/workload/WorkloadAlgorithmTest.java:
##########
@@ -154,19 +172,21 @@ private void runWorkloadAnalysisTest(String testname, ExecMode mode, int compres
String HOME = SCRIPT_DIR + TEST_DIR;
fullDMLScriptName = HOME + testname + ".dml";
- programArgs = new String[] {"-stats", "20", "-args", input("X"), input("y"), output("B")};
+ programArgs = new String[] {"-stats", "20", "-args", input("X"), input("y"), output("B"),
+ String.valueOf(maxIter)};
writeInputMatrixWithMTD("X", X, false);
writeInputMatrixWithMTD("y", y, false);
- String ret = runTest(null).toString();
+ ByteArrayOutputStream out = runTest(null);
+ String ret = out != null ? out.toString() : "";
LOG.debug(ret);
// check various additional expectations
long actualCompressionCount = (mode == ExecMode.HYBRID || mode == ExecMode.SINGLE_NODE) ? Statistics
.getCPHeavyHitterCount("compress") : Statistics.getCPHeavyHitterCount("sp_compress");
-
- Assert.assertEquals("Assert that the compression counts expeted matches actual: " + compressionCount + " vs "
+ System.out.println(actualCompressionCount);
Review Comment:
remove System out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SYSTEMDS-3823] Compression test case for bultin kmeans [systemds]
Posted by "e-strauss (via GitHub)" <gi...@apache.org>.
e-strauss closed pull request #2194: [SYSTEMDS-3823] Compression test case for bultin kmeans
URL: https://github.com/apache/systemds/pull/2194
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SYSTEMDS-3823] Compression test case for bultin kmeans [systemds]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #2194:
URL: https://github.com/apache/systemds/pull/2194#issuecomment-2621370264
## [Codecov](https://app.codecov.io/gh/apache/systemds/pull/2194?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 72.30%. Comparing base [(`6c4bffd`)](https://app.codecov.io/gh/apache/systemds/commit/6c4bffd9cc51aaef8a6254cb21becc6e60aeeb89?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`d4c140d`)](https://app.codecov.io/gh/apache/systemds/commit/d4c140d382cd3838b0e00e5783ca73b51cd0f26e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 3 commits behind head on main.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #2194 +/- ##
============================================
+ Coverage 71.86% 72.30% +0.43%
- Complexity 44707 44997 +290
============================================
Files 1450 1452 +2
Lines 169272 169309 +37
Branches 32996 33037 +41
============================================
+ Hits 121652 122413 +761
+ Misses 38290 37587 -703
+ Partials 9330 9309 -21
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/systemds/pull/2194?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [SYSTEMDS-3823] Compression test case for bultin kmeans [systemds]
Posted by "Baunsgaard (via GitHub)" <gi...@apache.org>.
Baunsgaard commented on PR #2194:
URL: https://github.com/apache/systemds/pull/2194#issuecomment-2621631555
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org