Slack digest for #dev - 2019-08-27

[Apache Pulsar Slack <ap...@gmail.com>]

2019-08-26 20:14:05 UTC - Addison Higham: huh, when running the integration tests via docker, I am not getting feedback from mvn but certainly seeing some tests run as containers go in and out of service, is there a way to get feedback to see where it is at?
----
2019-08-26 20:16:30 UTC - Ali Ahmed: @Addison Higham they are being streamed to a log file
----
2019-08-26 20:18:07 UTC - Addison Higham: `tests/integration/target/container-logs`?
----
2019-08-26 20:18:25 UTC - Ali Ahmed: I don’t remember on top of my head
----
2019-08-26 20:19:19 UTC - Addison Higham: that shows the logs for all the brokers/proxies etc, but I haven't yet found the logs of actual test progress, but will keep looking
----
2019-08-26 20:21:08 UTC - Sijie Guo: you can redirect the test output to console by specifying `-DredirectTestOutputToFile=false`
----
2019-08-26 20:32:52 UTC - Addison Higham: :thumbsup: thanks, also, one more thing that I keep noticing when building docker images... somewhere in the python client build I end up with extra wheel files in `pulsar-client-cpp/python/dist/` that if I switch versions (from 2.4 to 2.5 for example) leads to command that takes a glob that isn't specific enough and it ends up passing more args than the command handles
----
2019-08-26 20:33:25 UTC - Addison Higham: keep meaning to report it but then I once against lost if out of my scrollback
----
2019-08-26 20:43:26 UTC - Sijie Guo: I also noticed that. that’s because there are multiple wheel files from different versions.
----
2019-08-26 20:43:53 UTC - Sijie Guo: We can improve the build script to exclude wheel files from old versions.
----
2019-08-26 21:27:33 UTC - Ali Ahmed: I have pushed a test fix for one the flaky test issues <https://github.com/apache/pulsar/pull/5044>
----
2019-08-26 21:43:01 UTC - Addison Higham: random question: it seems like pulsar tests are ran one at a time, some of the tests, like the client tests, take quite a while to run and it didn't have very high system utilization which would make me thinking parallelization would be a good way to speed things up. That been something that has been explored it all?
----
2019-08-26 21:43:43 UTC - Matteo Merli: Yes, though the problem is then having the tests consistently passing :confused:
----
2019-08-26 21:44:29 UTC - Matteo Merli: there are few tests that are problematic, and some that don’t tear down all the resources  after they run
----
2019-08-26 21:44:51 UTC - Matteo Merli: that’s why we ended up running with 1fork per test
----
2019-08-26 21:46:37 UTC - Addison Higham: okay, that is what I figured. I wonder with surefire if that could be configured per module or some other way to isolate problematic tests. In scala-land with scalatest it isn't crazy hard to do a test annotation that makes it single threaded pool of tests and then everything else can be done in parallel
----
2019-08-26 21:47:35 UTC - Matteo Merli: Good point, we could use serial exec for pulsar-broker and parallel for the rest.. though it’s also the module that takes the vast majority of the time ..
----
2019-08-26 21:48:25 UTC - Matteo Merli: ```
          &lt;reuseForks&gt;false&lt;/reuseForks&gt;
          &lt;forkCount&gt;1&lt;/forkCount&gt;
```

This would have to be changed into `true` and `2C` to have multiple tests running in parallel
----
2019-08-26 22:02:58 UTC - Addison Higham: I think I finished this run but next time I need to run the tests I will try poking around
----
2019-08-26 22:04:41 UTC - Addison Higham: huh, this hasn't happened locally: <https://builds.apache.org/job/pulsar_precommit_java8/10990/console>
----
2019-08-27 02:38:01 UTC - Yi Tang: @Yi Tang has joined the channel
----