You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by GitBox <gi...@apache.org> on 2020/11/06 00:08:10 UTC
[GitHub] [bigtop] sekikn opened a new pull request #691: BIGTOP-3441. Suppress YCSB smoke test output.
sekikn opened a new pull request #691:
URL: https://github.com/apache/bigtop/pull/691
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #691: BIGTOP-3441. Suppress YCSB smoke test output.
Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #691:
URL: https://github.com/apache/bigtop/pull/691#issuecomment-722909677
That sounds better, because it reserves output to stderr that may be useful for debugging and its size is reasonably small. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on pull request #691: BIGTOP-3441. Suppress YCSB smoke test output.
Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #691:
URL: https://github.com/apache/bigtop/pull/691#issuecomment-722873725
@sekiken the stdout should be redirected to file. Without the redirection, output messages of ycsb are kept in memory. I think we should keep the redirection.
In addition, just removing `-s` did not reduce the file size. The contents of load_basic.dat looks like query log.
before:
```
[root@e4230e025d1a bigtop-home]# ls -lh /bigtop-home/bigtop-tests/smoke-tests/ycsb/
total 537M
drwxr-xr-x. 7 root root 84 Nov 6 05:22 build
-rw-rw-r--. 1 jenkins jenkins 1.2K Jul 30 02:56 build.gradle
-rw-r--r--. 1 root root 537M Nov 6 05:22 load_basic.dat
-rw-rw-r--. 1 jenkins jenkins 3.5K Nov 6 05:11 TestYcsbSmoke.groovy
-rw-r--r--. 1 root root 107K Nov 6 05:23 transactions_basic.dat
```
after:
```
[root@309de81c3edf /]# ls -lh /bigtop-home/bigtop-tests/smoke-tests/ycsb/
total 537M
drwxr-xr-x. 7 root root 84 Nov 6 06:00 build
-rw-rw-r--. 1 jenkins jenkins 1.2K Jul 30 02:56 build.gradle
-rw-r--r--. 1 root root 537M Nov 6 06:01 load_basic.dat
-rw-rw-r--. 1 jenkins jenkins 3.5K Nov 6 05:36 TestYcsbSmoke.groovy
-rw-r--r--. 1 root root 106K Nov 6 06:01 transactions_basic.dat
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims edited a comment on pull request #691: BIGTOP-3441. Suppress YCSB smoke test output.
Posted by GitBox <gi...@apache.org>.
iwasakims edited a comment on pull request #691:
URL: https://github.com/apache/bigtop/pull/691#issuecomment-722873725
@sekiken the stdout should be redirected to file. Without the redirection, output messages of ycsb are kept in memory.
In addition, just removing `-s` did not reduce the file size. The contents of load_basic.dat looks like query log.
before:
```
[root@e4230e025d1a bigtop-home]# ls -lh /bigtop-home/bigtop-tests/smoke-tests/ycsb/
total 537M
drwxr-xr-x. 7 root root 84 Nov 6 05:22 build
-rw-rw-r--. 1 jenkins jenkins 1.2K Jul 30 02:56 build.gradle
-rw-r--r--. 1 root root 537M Nov 6 05:22 load_basic.dat
-rw-rw-r--. 1 jenkins jenkins 3.5K Nov 6 05:11 TestYcsbSmoke.groovy
-rw-r--r--. 1 root root 107K Nov 6 05:23 transactions_basic.dat
```
after:
```
[root@309de81c3edf /]# ls -lh /bigtop-home/bigtop-tests/smoke-tests/ycsb/
total 537M
drwxr-xr-x. 7 root root 84 Nov 6 06:00 build
-rw-rw-r--. 1 jenkins jenkins 1.2K Jul 30 02:56 build.gradle
-rw-r--r--. 1 root root 537M Nov 6 06:01 load_basic.dat
-rw-rw-r--. 1 jenkins jenkins 3.5K Nov 6 05:36 TestYcsbSmoke.groovy
-rw-r--r--. 1 root root 106K Nov 6 06:01 transactions_basic.dat
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims merged pull request #691: BIGTOP-3441. Suppress YCSB smoke test output.
Posted by GitBox <gi...@apache.org>.
iwasakims merged pull request #691:
URL: https://github.com/apache/bigtop/pull/691
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on pull request #691: BIGTOP-3441. Suppress YCSB smoke test output.
Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #691:
URL: https://github.com/apache/bigtop/pull/691#issuecomment-722904140
@sekikn How about redirecting stdout to /dev/null and stderr to the file? like `"-s >/dev/null 2>transactions_basic.dat"`.
The message generated by the `-s` seems to be written to stderr and reasonably small.
```
[root@7954c75b4084 /]# ls -lh /bigtop-home/bigtop-tests/smoke-tests/ycsb/
total 16K
drwxr-xr-x. 7 root root 84 Nov 6 06:28 build
-rw-rw-r--. 1 jenkins jenkins 1.2K Jul 30 02:56 build.gradle
-rw-r--r--. 1 root root 1.2K Nov 6 06:28 load_basic.dat
-rw-rw-r--. 1 jenkins jenkins 3.5K Nov 6 06:18 TestYcsbSmoke.groovy
-rw-r--r--. 1 root root 2.2K Nov 6 06:28 transactions_basic.dat
[root@7954c75b4084 /]# cat /bigtop-home/bigtop-tests/smoke-tests/ycsb/load_basic.dat
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.272.b10-1.el8_2.x86_64/bin/java -cp /usr/lib/ycsb/basic-binding/conf:/usr/lib/ycsb/conf:/usr/lib/ycsb/lib/HdrHistogram-2.1.4.jar:/usr/lib/ycsb/lib/core-0.12.0.jar:/usr/lib/ycsb/lib/htrace-core4-4.1.0-incubating.jar:/usr/lib/ycsb/lib/jackson-core-asl-1.9.4.jar:/usr/lib/ycsb/lib/jackson-mapper-asl-1.9.4.jar com.yahoo.ycsb.Client -db com.yahoo.ycsb.BasicDB -P /usr/lib/ycsb/workloads/workloada -p recordcount=500000 -s -load
YCSB Client 0.12.0
Command line: -db com.yahoo.ycsb.BasicDB -P /usr/lib/ycsb/workloads/workloada -p recordcount=500000 -s -load
Loading workload...
Starting test.
DBWrapper: report latency for each error is false and specific error codes to track for latency are: []
2020-11-06 06:28:16:987 0 sec: 3 operations; ? current ops/sec; est completion in 0 seconds [INSERT: Count=9, Max=1235, Min=237, Avg=447.22, 90=473, 99=1235, 99.9=1235, 99.99=1235]
2020-11-06 06:28:21:130 4 sec: 500000 operations; 120249.4 current ops/sec; [CLEANUP: Count=1, Max=1, Min=1, Avg=1, 90=1, 99=1, 99.9=1, 99.99=1] [INSERT: Count=499991, Max=10471, Min=6, Avg=6.91, 90=7, 99=15, 99.9=363, 99.99=383]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #691: BIGTOP-3441. Suppress YCSB smoke test output.
Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #691:
URL: https://github.com/apache/bigtop/pull/691#issuecomment-722884897
Thank you for the comment, @iwasakims!
I'll update the PR to keep redirection and remove its output in a new cleanup method with the `@AfterClass` annotation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #691: BIGTOP-3441. Suppress YCSB smoke test output.
Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #691:
URL: https://github.com/apache/bigtop/pull/691#issuecomment-722919669
Updated the PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org