Enabled backend tests for UBSAN

[Tim Armstrong <ta...@cloudera.com>]

Hi All,
  I turned on backend tests under UBSAN in our precommit tests - Jim Apple
let me know that they were passing after he fixed the final set of issues
there.

- Tim

Re: Enabled backend tests for UBSAN

[Tim Armstrong <ta...@cloudera.com>]

I've run the impala minicluster with TSAN occasionally when trying to debug
races. I think it would be useful to run the backend tests with it. The
other annoyance with TSAN is that there are a number of false positives
with the JVM that we'd need to suppress.

It seems worth the effort.

It would also be useful to finish off enabling leak sanitizer for the
handful of backend tests that don't have it enabled. I turned it on for the
vast majority of backend tests but there were a few that still leaked
thrift servers and things like that.

On Mon, May 6, 2019 at 11:13 AM Todd Lipcon <to...@cloudera.com> wrote:

> That's great to hear. Has anyone tried running any BE tests with TSAN in
> recent years? It's been invaluable for Kudu, and while I'm sure it would
> take a bit of effort to get passing on Impala, the ongoing returns would be
> worth it IMO.
>
> One point, though, is that for TSAN to work well, you need to instrument
> thirdparty libs as well, so we'd need to start from native-toolchain and
> work our way up.
>
> -Todd
>
> On Mon, May 6, 2019 at 10:49 AM Tim Armstrong <ta...@cloudera.com>
> wrote:
>
> > Hi All,
> >   I turned on backend tests under UBSAN in our precommit tests - Jim
> Apple
> > let me know that they were passing after he fixed the final set of issues
> > there.
> >
> > - Tim
> >
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: Enabled backend tests for UBSAN

[Todd Lipcon <to...@cloudera.com>]

That's great to hear. Has anyone tried running any BE tests with TSAN in
recent years? It's been invaluable for Kudu, and while I'm sure it would
take a bit of effort to get passing on Impala, the ongoing returns would be
worth it IMO.

One point, though, is that for TSAN to work well, you need to instrument
thirdparty libs as well, so we'd need to start from native-toolchain and
work our way up.

-Todd

On Mon, May 6, 2019 at 10:49 AM Tim Armstrong <ta...@cloudera.com>
wrote:

> Hi All,
>   I turned on backend tests under UBSAN in our precommit tests - Jim Apple
> let me know that they were passing after he fixed the final set of issues
> there.
>
> - Tim
>


-- 
Todd Lipcon
Software Engineer, Cloudera

Re: Enabled backend tests for UBSAN

[Sahil Takiar <ta...@gmail.com>]

I've been running an UBSAN job for the past week that covers BE, JDBC, FE,
and custom cluster tests. So far everything has been clean.

Separate update on the enabling TSAN, I started actively working on this
in IMPALA-5904. I'm pretty close to getting dataload + the BE
tests TSAN-clean (haven't invested the time in getting third-party libs
compiled with TSAN yet though).

On Fri, Jun 28, 2019 at 9:18 PM Jim Apple <ap...@jbapple.com> wrote:

> The JDBC, FE, and custom cluster tests are now UBSAN-clean. Can you turn
> those on, too?
>
> Thank you!
>
> On Mon, May 6, 2019 at 10:49 AM Tim Armstrong <ta...@cloudera.com>
> wrote:
>
> > Hi All,
> >   I turned on backend tests under UBSAN in our precommit tests - Jim
> Apple
> > let me know that they were passing after he fixed the final set of issues
> > there.
> >
> > - Tim
> >
>


-- 
Sahil Takiar
Software Engineer
takiar.sahil@gmail.com | (510) 673-0309

Re: Enabled backend tests for UBSAN

[Jim Apple <ap...@jbapple.com>]

The JDBC, FE, and custom cluster tests are now UBSAN-clean. Can you turn
those on, too?

Thank you!

On Mon, May 6, 2019 at 10:49 AM Tim Armstrong <ta...@cloudera.com>
wrote:

> Hi All,
>   I turned on backend tests under UBSAN in our precommit tests - Jim Apple
> let me know that they were passing after he fixed the final set of issues
> there.
>
> - Tim
>